Wednesday, July 11, 2012

The First Step Is To Ask The Right Questions


Six months into my tenure as BRPT President, I’ve spent considerable time delving into exam statistics and data, trying to understand what influences the pass rate and perception of difficulty of the RPSGT exam.  Unfortunately, my investigation just leads me to more questions instead of answers.  For instance, why is it, time after time, RPSGT candidates who have another health care credential have the highest overall pass rate on the RPSGT exam?  Is it because they have more education or work experience in health care, or they are better test-takers, or could it be they know how to prepare better for a certification examination?  Why is it two individuals who go through the same associate’s level education program, and receive the same information, can perform differently on the exam, one passing and another failing?   Why is it students from one education program can have a consistently high pass rate while students from another can have a consistently low pass rate?  Could it be there is a difference in education courses, or instructors, or maybe the motivation or educational readiness of students accepted into the programs?   Why is it that individuals with on-the-job training, supported by on-line education modules, consistently have the lowest pass rate on the RPSGT exam?  And why is it that nearly 50% of test takers find the RPSGT exam fair or even too easy, while the other 50% say the exam is too difficult?

An investigator should have a hypothesis.  My hypothesis is that the RPSGT exam is not unrealistically difficult.  I think there are many factors contributing to the perception of difficulty and the pass rate of the exam.  I am not convinced there are significant flaws in the design or content of the RPSGT exam.  As a reputable certification agency, the BRPT builds an examination that is relevant to large numbers of people with varied backgrounds and education.  It offers examinations using strict security measures.  It recruits subject matter experts from all around the country, with differing backgrounds, education, and work experience.  BRPT brings subject matter experts together to work face-to-face, sometimes spending-literally-hours on one exam question before it is approved for use on the examination.  BRPT uses psychometricians and statisticians to balance the level of difficulty for each exam form, and to evaluate each and every exam question for reliability and validity, looking for items that need to be double checked for accuracy or even re-worked for better clarity.  As a nationally – and internationally –respected credentialing organization, the BRPT builds the exam against a blueprint of a job description that is created from a statistically reliable survey of technologists working day to day in the field as RPSGTs, a survey that is repeated at least every 5 years, sooner as the field changes rapidly, to keep the exam relevant.

BRPT works very hard to put together the RPSGT certification examination based on current testing guidelines and standards.  Is the exam difficult?  For some it is, for others it is not.  Can the RPSGT exam be improved?  Absolutely!  BRPT is continually evaluating and reevaluating exam options and processes to incorporate positive changes and enhancements.  The RPSGT exam is a good, sound exam.  It is an exam prepared for sleep technologists by sleep technologists.  During my term as President I hope to prove or disprove my hypothesis.  I think I can do this by drilling down deeper into the data we collect, by looking at similar certification organizations, issues of professional readiness, and factors that affect learning.   As we begin the process to conduct a new Job Task Analysis for the RPSGT exam, I can assure you these questions will remain at the forefront of our discussions.

Cindy Altman, RPSGT, R. EEG/EP T.
BRPT President