General responsibilities and timeline for ASBMB exam question writers

Who are the ASBMB question writing volunteers?

Since the ASBMB exam was first offered in 2014, scientist–educators from institutions ranging from R1 universities to small colleges have volunteered their time and expertise in service of the biochemistry and molecular biology community by writing and scoring questions for the exam. Preferred qualifications for these volunteers include a master’s degree or Ph.D. in biochemistry and/or molecular biology or a related field, together with prior experience in BMB education as an instructor or, perhaps, teaching assistant. Ph.D. candidates, i.e. those who have advanced to candidacy by passing their qualifying exam, are also eligible for consideration as exam question writers.

Enthusiasm for working collaboratively to create assessment items that fairly and robustly measure student learning in ASBMB-accredited programs is highly valued! Question writers play an integral role in ASBMB’s ability to produce a new exam every year. Before being utilized for assessment, each new exam question is drafted, reviewed, piloted, analyzed and further polished. Question writers contribute at each stage of this process. They draft new questions and associated rubrics and then gather remotely to revise and refine these questions in preparation for piloting on the exam. Question writers also analyze student answers collected in response to piloted questions and make recommendations for further improvements and/or inclusion on a future exam. A good resource for understanding the origins of the exam and the community-driven process behind its construction and testing — including the development and use of pilot questions on the exam — is this 2021 publication in CBE—Life Sciences Education.

What are the benefits of being a volunteer?

Our volunteers are members of a dynamic and interactive community who, through their collective efforts, are influencing undergraduate education in biochemistry and molecular biology on a national scale. Volunteers not only get an inside look at the operation of the program and the exam but also provide input that helps shape these activities to better serve the community. Volunteers' contributions are acknowledged through the awarding of the title of ASBMB Education Associate, a certification that can be cited in biographical sketches and tenure and promotion dossiers.

Question writing is an opportunity both to hone one’s skills in developing assessment items and to connect with a stimulating and dynamic group of colleagues. Question writers have shared that they take the best practices they learned in writing for the ASBMB exam back into their own classrooms. This article in ASBMB Today describes one volunteer’s positive experience.

What do question writers do?

Organization

Question writers are organized into four teams, based on expressed interest area and overall need, under the direction of an experienced team leader. Each team is responsible for questions pertaining to one of ASBMB’s four core concept and skill areas (CAs):

• CA1: Energy and Metabolism
• CA2: Structure and Function
• CA3: Information Storage and Transfer
• CA4: Scientific Skills/Quantitative Analysis

Creation of pilot questions and rubrics (estimated 1-2 hours)

In early fall, question writers are recruited to draft questions to be included as pilot questions on a future exam. Question writers are provided a template — with notes of best practices — on which to articulate a learning objective, draft a question and draft a scoring rubric. (Writing guidelines are outlined, for example, in the Supplemental Material of this publication describing the ASBMB exam.) The scoring rubric contains sample responses and defines the criteria for scoring an answer as “Highly Proficient,” Proficient,” or “Not Yet Proficient.”

Question development is scheduled as a fall activity, and the results of piloting are analyzed in August/September, after the completion of the exam cycle in July. Conference calls for review and refinement are usually reserved for late fall/early winter. However, submissions of new draft questions are welcome year-round.

Preliminary review and refinement of questions and rubrics (estimated 2-3 hours)

In late fall, each of the four teams is sent the questions that were developed within their particular core concept/skill area in the current cycle. The group leader then schedules a conference call in which the members of the team discuss and, where appropriate, suggest changes to each question and/or its associated rubric.

Further review and refinement of questions and rubrics (estimated 1-3 hours)

In early winter, the four teams meet via conference call to revisit the revised questions and associated rubrics and offer final suggestions for improving the quality of the question, the scoring rubric, or both. By the end of the conference call, questions are recommended for piloting or are recycled for further revision in the following question-writing cycle.

All new exam questions are first piloted on an ASBMB exam. Answers are collected for internal assessment of the quality of the item and its associated rubric, but the score is not reflected in the exam taker’s final assessment. Rather, question writers use these data to further refine a question before its formal adoption on a future exam. Question writers often report positively on the opportunity to discuss assessment development with other, independent scientist–educators.

Analysis of responses to piloted questions and debriefing (estimated 3-4 hours)

In late summer, question writers analyze approximately 75–150 student answers for each piloted question, in order to evaluate the exam question and associated rubric (approximately 2–3 hours). Based on these answers, writers either recommend further revisions and re-piloting or submit the question for consideration for an upcoming exam. Debriefing discussions to share these recommendations are held remotely, usually in September–October.

How can you volunteer to be a question writer?

Thank you for your interest! Please complete this form to let ASBMB staff know of your interest and your preferred concept/skill area(s). Question writing volunteers are typically recruited during the fall of each year, in advance of the next exam cycle.