at The College Board in Springfield, Illinois, United States
Job Description
Advanced Doctoral Psychometrics Intern (10-month)College Board - Learning & Assessment/Psychometrics
Location: This is a fully remote role. At the discretion of the candidate's mentor, there could be an opportunity to attend two optional on-site one-week residencies at our New York City office.
Type:
- This is an internship position that will last 10 months from the start date beginning in September and ending in June. Interns will be expected to work 32 hours a week and the internship will coincide with the student's school schedule.
Application Deadline: Friday, August 9th at 5pm EST
About the Team
The Psychometrics Team supports College Board in serving more than seven million students a year as they prepare for a successful transition to college through programs and services in college readiness and college success. College Board's Psychometrics provides its partners with measurement solutions that set the standard for quality, efficiency and innovation through operational excellence and research leadership that advances the field of Psychometrics and supports College Board's mission of access and equity in Education.
About the Opportunity
The internship spans the entire academic year, with an expected weekly load of 32 hours per week (four days a week). This internship is designed to provide the intern with the opportunity to work closely with psychometricians and to gain hands-on working experience with College Board data and research projects in support of College Board programs. The intern is expected to perform operational psychometric tasks with the guidance of the two mentors. The tasks may include but are not limited to standard setting, test analysis, review of technical manuals, literature review and conducting research studies.
As an intern you will be part of one of the four teams in the Psychometrics department at College Board.
These teams are
- AP Psychometrics
- SAT Psychometrics, Analytics, and Research
- CLEP and ACCUPLACER Psychometrics & Research
- Psychometric Technology and Analytics
You may also work directly with the Robert L. Brennan Chair of Psychometrics who consults and works with all of the four teams on multiple projects.
Current ongoing projects for which you may have an opportunity to work on include
- Reliability for tests with polytomous items with different weights
- Multidimensional response time analysis for test security - Detecting anomalies in response time due to test-taking misbehavior while factoring out construct-irrelevant sources of variance like speededness
- Standard setting methods - Exploratory analysis of cognition and test-taking strategy to improve ALD development
- Improving cut score decisions via machine learning - Using ranking algorithms to improve precision and consistency of cut score decisions in standard setting
- Expanding item parameter drift detection tool into a full R package
- Exploring maximization methods for IRT scoring
- Investigation of psychometric qualities of reported subscores or domain scores using both simulated and real data
- Exploring methods to detect invalid essay responses using deep learning models
- IRT equating for mixed-format tests with weights
- Standard setting using AI
- Detection of AI usage in free response questions
- Automated essay scoring
- Prediction of item statistical properties using large language models (LLM),
- Using timing data in predicting student scores
- Investigating different test security analytics
About You
To be eligible:
- The intern must be a full-time doctoral student at an accredited 4-year university
- A strong preference will be made toward advanced students in the process of completing their dissertation
- Graduate students in psychometrics, measurement, educational psychology, industrial-organizational psychology, or related fields are invited to submit an application. Students in adjacent fields such as data science, computational linguistics, and areas of computer science such as artificial intelligence (AI) can also apply
- Experience with statistical software (SAS and/or R), psychometrics software (e.g., flexMIRT, Mplus), and/or data science tool such as Python is required
- Working knowledge of Classical Test Theory and Item Response Theory is required
- Students must be eligible to be legally employed in the United States
- Good communication skills (oral and written) are required
About Our Compensation
College Board offers a competitive compensation program that attracts top talent looking to make a difference in education.
- The salary for this position is $32 per hour. You will be paid this rate hourly for time worked
- As a temporary internship position, this role is not eligible for College Board benefits including medical benefits or paid time off
- Interns are required to be in the United States for the duration of the internship.
You can expect to have transparent conversations about compensation with our recruiters throughout your application process. To view full details and how to apply, please login or create a Job Seeker account
