ACT, Inc. Senior Research Scientist in Iowa City, Iowa
ACT is a nonprofit organization helping people achieve educational and workplace success. Our programs are designed to boost lifelong learning in schools and workplaces around the world. Whether it's guiding students along their learning paths, enabling companies to develop their workforce, fostering parent, teacher, and counselor understanding of student progress, guiding job seekers toward career success, or informing policymakers about education and workforce issues. ACT is passionate about making a difference in all we do.
Learn more about working at ACT at
ACT is looking for a seasoned measurement professional to join our Assessment Transformation team. The Assessment Transformation team has as its mission to lead the research, design, and development of next generation assessments that facilitate adaptability and personalization by leveraging digital technologies, new measurement models, and digital data sources. This position will independently: provide psychometric support including test construction, field test design and item analysis, equating design and analysis, and score comparability research; incorporate the use of process data (e.g. response time, eye-tracking, log file data) to enhance assessment modeling and support for adaptive/personalized assessment feedback; conduct cognitive labs to provide validity evidence for response processes; conduct research on next generation psychometric models at scale.
ESSENTIAL JOB RESPONSIBILITIES
Typical work-related activities include:
Independently design and conduct psychometric and research analyses for ACT products and services
Interpret and synthesize results from psychometric analyses and make recommendations to ACT assessment product owners based on those results
Lead project teams, maintain quality control of all projects, and guide junior staff in completing assignments and acquiring skills
Create and maintain technical specifications and documentation for the psychometric processes and procedures related to the ACT and related assessment programs
Conduct cognitive labs, usability studies, and eye-tracking research to provide validity evidence based on response processes
Incorporate process data (e.g., response time, eye-tracking, log file data) into psychometric procedures to support adaptive and personalized assessment solutions
Provide input to technology teams and data engineering teams on capabilities needed for test delivery and scoring as well as data outputs and formats for analysis for both paper based and digital assessments
Present research study plans and results to the ACT Technical Advisory Committee for feedback
Provide support for proposal development in response to domestic RFP’s, international tenders, and similar requests for services
Collaborate closely with Test Development on development of overall assessment design and content specifications for current and new assessment products
Support the development of a research agenda to transition existing large scale educational assessments to new assessment modes and measurement models as well as the development of new assessments (both formative and summative) which utilize next generation psychometric methods
Participate within ACT Research and across ACT in creating an open, innovative, and collaborative environment
Increase visibility of ACT Research through staff publications, presentations, and presence across relevant national and international research communities
- Doctoral degree in educational measurement, educational psychology, quantitative psychology or related field
Six years of experience in the development and publishing of high-quality, large scale educational assessments
Demonstrated experience implementing program and project management methods for complex assessment programs (e.g., creating/managing schedules, creating/managing program documentation and analysis specifications, etc.)
Experience in working with technology delivered assessments including knowledge of processes for authoring, publishing, delivery, and data acquisition
Experience with transitioning large scale assessment programs to new modes of delivery and new measurement models
Experience in developing scoring methodologies and applying psychometric models for technology enhanced item types
Experience with developing proposal responses for state assessment programs
Experience applying Item Response Theory in support of field testing, test construction, and operational scaling and equating practices
Demonstrated experience working with IRT software packages including Bilog, FlexMIRT, Winsteps, etc.
Experience presenting to Technical Advisory Committees
TravelUp to 25% Travel
ACT is an Equal Opportunity Employer/Minorities/Females/Protected Veterans/Disabled.