General responsibilities and timeline for ASBMB exam scorers

Who are the ASBMB exam scoring volunteers?

Since the ASBMB exam was first offered in 2014, over 130 scientist–educators from institutions ranging from R1 universities to small colleges have volunteered their time and expertise in service of the biochemistry and molecular biology community by scoring student responses. Preferred qualifications for scorers include a master’s degree or Ph.D. in biochemistry and/or molecular biology or a related field, together with prior experience in BMB education as an instructor or, perhaps, teaching assistant. Ph.D. candidates, i.e. those who have advanced to candidacy by passing their qualifying exam, are also eligible for consideration as exam scorers.

Enthusiasm for working collaboratively to fairly and robustly assess student performance on the ASBMB exam is highly valued! Volunteer scorers play an integral role in both exam development and evaluation. They work in teams to review and refine prospective questions and their associated answer keys in addition to reading and classifying student responses to their assigned question. A good resource for understanding the origins of the exam and the community-driven process behind its construction and testing is a recently published manuscript from CBE—Life Sciences Education.

What are the benefits of being a volunteer?

Our volunteers are members of a dynamic and interactive community who, through their collective efforts, are influencing undergraduate education in biochemistry and molecular biology on a national scale. Volunteers not only get an inside look at the operation of the program and the exam, they also provide input that helps shape these activities to better serve the community. Volunteers' contributions are acknowledged through the awarding of the title of ASBMB Education Associate, a certification that can be cited in biographical sketches and tenure and promotion dossiers.

Many scorers report that their participation has proven to be a highly rewarding experience that has improved their skills as question writers and evaluators and connected them with a stimulating and dynamic group of colleagues. The sincerity of this feedback is reflected by the fact that most scorers continue to volunteer year after year. From 2020–2021, 97% of scorers were returning volunteers. This article in ASBMB Today describes one volunteer’s positive experience.

What do scorers do?

Organization 

Scorers are organized into four teams, based on expressed interest area and overall need, under the direction of an experienced team leader. Each team is responsible for the questions pertaining to one of ASBMB’s four core concept and skill areas (CAs):

• CA1: Energy and Metabolism
• CA2: Structure and Function
• CA3: Information Storage and Transfer
• CA4: Scientific Skills/Quantitative Analysis

Review and refinement of questions and rubrics (estimated 3 hours)

In late fall, each of the four teams is sent the questions nominated for inclusion in the next year’s exam — usually two to four in number — that fall within their particular core concept/skill area. The group leader then schedules a conference call in which the members of the scoring team discuss and, where appropriate, suggest changes to the question and/or its associated answer key.

We have found that the scoring teams exert a profound impact on the quality of questions and keys. Scorers bring an independent point of view and a clear stake in ensuring that questions are clear and unambiguous and require concrete, definable responses. This input is amplified by the opportunity to engage in discussion with other, independent scientist–educators.

Scoring trial (4–6 hours)

All exam scoring is completed online on a user-friendly platform. Prior to the beginning of scoring, ASBMB support staff hold an informational webinar on navigating the platform.

In approximately late March, scorers are divided into teams of three, each of which is assigned to score student responses to a single question. As the number of students taking the exam has grown, we routinely assign more than one team per question to further distribute the workload.

In the month of April, we initiate a scoring trial to ensure that scorers are properly “calibrated” (score in a consistent and accurate manner) and to address any unanticipated student responses. This is done by asking the scorers assigned to a particular question to evaluate approximately 50 sample student responses from that year's exam. 

These preliminary scores are then used to calculate inter-rater reliability, a measure of consistency among the members of the scoring team; scoring team leaders and ASBMB staff review any responses that received diametrically opposite scores, e.g. both “Not yet proficient” and “Highly proficient”. The scoring team then receives a report summarizing the degree of consistency exhibited during the scoring trial, together with an analysis of any problems with the key or its interpretation that became apparent. A conference call is subsequently used to discuss suggested changes in the key or to clarify misconceptions.

Bulk scoring (8–10 hours)

In May–June, the now-calibrated scoring teams are typically given between 200–300 student responses to score using the online grader platform. If needed, an additional conference call may be scheduled or an email exchange used to address any inconsistencies observed in scoring before students’ scores are averaged and finalized.

Debriefing (1 hour)

To ensure that any future versions of these questions that might be used in subsequent exams are of the highest quality possible, scoring team leaders may invite members of their team to discuss how specific questions can be improved in light of the experience gained. 

How can you volunteer to be a scorer?

Thank you for your interest!  Please complete this form to let ASBMB staff know of your interest and your preferred concept/skill area(s). Scoring volunteers are typically recruited during the fall of each year, in advance of the next exam cycle.