Test Development and Scoring Process

Each ABPN examination is developed by a committee, chaired by an ABPN director, and comprised of subject matter experts (SMEs) who have been trained in the principles of item writing. In addition to content expertise, the ABPN strives to achieve a balance in terms of other factors such as involvement in residency and fellowship training, geographic location, gender, and cultural background. A typical test development cycle is one year. Test scoring typically takes 8-12 weeks, depending on the number of examinations administered in the same time frame.

Content Outline Development
  • Test committee develops content outline that specifies both the topics to be covered and the number (percent) of items devoted to each.
  • Posted on the ABPN web site for examinees.
  • Periodically reviewed and updated.

Item Writing and Review
  • Item writers receive assignments and submit items to ABPN.
  • ABPN editors edit items and return to authors for revision if needed.
  • Test committee meets to review items.

Test Assembly
  • ABPN test development staff prepare draft exam, including pre-test items.
  • Test committee reviews draft and selects final items.
  • ABPN staff publish and proof final exam.
  • Test committee chair reviews and approves final exam.

Test Administration
  • All certification and MOC examinations are administered under secure, standardized test conditions.
  • Examination data are delivered electronically to the ABPN.

Test Scoring
  • Psychometric staff conduct preliminary scoring to ensure that all data were transmitted accurately.
  • Psychometric staff review the scoring results to verify that each question on the examination is psychometrically valid.
  • Test development staff review all examinee comments to identify problematic items.
  • Based on these analyses, items that do not meet certain statistical criteria and/or have generated examinee comments are referred to the committee chair for review.
  • Items that are deemed to be flawed by the committee chair are deleted, and the examination is rescored.
  • This phase typically takes 4-6 weeks, depending on the number of test items and the number of tests being scored at the same time.

Standard Setting
  • The ABPN utilizes a criterion-referenced approach to standard setting. That is, there are no preset pass/fail rates for any group of examinees (referred to as norm-referenced approaches). It is always possible for all examinees to pass an examination.
  • Periodically (typically every 7-8 years) a standard setting study is conducted by the test committee members and other subject matter experts. During a 2-day meeting, they review all of the items on the examination and use well-established, evidence-based procedures to set the passing standard for the examination.
  • After the initial standard has been set, in subsequent years statistical procedures are used to equate the pass/fail standard across different versions of the same test. This removes any advantage associated with taking a test with easier items or disadvantage associated with taking a test with harder items.
  • This phase typically takes 2-4 weeks, depending on whether a standard setting study is conducted or whether equating is used.

Score Reports
  • Score reports are generated, and several quality control procedures are performed to verify the accuracy of the scores.
  • Scores reports are posted electronically in your portfolio (specialty certification examinations) or mailed to you.
  • This phase typically takes 2 weeks, depending on the number of examinees and the number of tests administered at the same time.

American Board of Psychiatry and Neurology, Inc.

7 Parkway North, Deerfield, IL 60015

Phone 847.229.6500 | Fax 847.229.6600