The Canadian Organization of Paramedic Regulators (COPR) promotes and advances best practices in paramedicine nationwide. Founded in 2009, COPR is comprised of self-regulating colleges and government regulators. With the help of nine provinces, COPR develops the entry-to-practice exams for the paramedic profession and administers them in seven provinces. COPR also conducts assessments of internationally educated paramedic professionals for all Canadian jurisdictions. The organization administers the following exams each year:
- Primary Care Paramedic (PCP) Entry to Practice Examination
- Advanced Care Paramedic (ACP) Entry to Practice Examination
- Emergency Medical Responder (EMR) Entry to Practice Examination
Delivers exams for 3 different practice levels 4 times per year
Assesses 1,200+ candidates per year
- Itematic Automated Item Generation Software
- ProctorU Online Proctoring Platform
- Measure Exam Delivery Software
Exam Scoring & Marking
In 2010, paramedicine was considered a noncompliant industry in Canada for labor mobility. To help move its candidates forward, COPR needed to develop pan-Canadian exams for paramedic professionals.
COPR had three goals for its new exams:
- Ensure the exams are valid, reliable, and fair
- Standardize the licensing process across the paramedicine industry
- Develop a robust item bank as quickly as possible
The process of item development is notorious for being time- and resource-intensive, resulting in one of the costliest components of the testing process. To develop pan-Canadian exams with enough content to comprehensively assess its candidates, COPR needed to move quickly, deliberately, and cost effectively.
COPR ultimately put its trust in Yardstick. And before long, Yardstick set its sights on supplementing the invaluable work of COPR’s subject matter experts (SMEs). To do so, Yardstick and COPR worked closely on piloting Itematic—an automated item generation (AIG) tool. Unlike other AIG tools, Itematic allows SMEs to specify how many items they want to generate and then creates only the most unique subset. After that, a Yardstick psychometrician can seamlessly import approved items into an existing item bank. The goal of using Itematic was not to eliminate the need for COPR’s SMEs but to give them an extra tool to save time, increase productivity, and enjoy the item-writing process.
To get started, COPR SMEs designed the structure of new items using cognitive models that define the knowledge and skills required of candidates. Next, the SMEs created item models, specifying which item features could be changed. After that, Itematic took over. Its dynamic algorithms matched the cognitive models to the item models, rapidly generating dozens of unique items. Because Itematic slashed the most time-consuming part of the development process, COPR SMEs were able to spend more time reviewing the items to make sure they were valid, reliable, and fair.
Traditional Item Development VS. Itematic Development Process
COPR’s SMEs create templates so algorithms can develop many unique items.
COPR’s per-item cost is decreased by 30%-50% while maintaining quality.
The SME’s item-writing output can be ramped up significantly based on item-banking needs and the quality of the template.