Richard Rathe, M.D.; Richard Smith, Ph.D.
University of Florida College of Medicine, Gainesville, Florida, USA

Intended Audience: This workshop is intended for medical educators, information technology support staff and others who have interest in advanced assessment techniques.

Prerequisites: Participants should have a basic knowledge of computers and some familiarity with assessment techniques (i.e., multiple choice tests). While some of the material presented will be technical, no prior knowledge of statistics or computer programming is necessary.

Objectives:Workshop participants will: Understand the pros and cons of traditional assessment techniques (TAT) Understand the major components of computer adaptive testing (CAT) Understand how CAT overcomes the major problems with TAT Experience an adaptive test first hand Experience prescriptive feedback based on test performance Directly observe the various dynamic processes within the CAT system Understand and use advanced item analysis for quality improvement

Workshop Format:The first portion will cover essential concepts including item banks, score reporting, feedback, and item analysis. Participants will then experience an adaptive test first hand using software developed at the University of Florida. The final portion will use the results of this live demonstration to illustrate important concepts and the adaptive component of the CAT system.

Overview:Computer adaptive testing has the potential to transform all aspects of medical education. The availability of inexpensive, reliable microcomputers and Internet technologies finally puts this important tool within the reach of most institutions. This workshop will serve as an introduction to computer adaptive testing as we attempt to demonstrate how this technique solves many of the problems encountered with traditional testing techniques.Computer adaptive tests rely on calibrated banks of questions rather than a single test form for test administration. Each examinee sees a different set of questions. The difficulty of the questions is adapted to match the ability of the examinee in real time - each correct or incorrect answer defines a different item selection tree. A test is finished when the examinee's performance meets certain accuracy requirements. There is no wasted effort. On average adaptive tests require half the number of questions for the material as traditional methods for the same accuracy. The system also gives the examinee immediate detailed feedback to guide future study. In most cases the examinee may be allowed to take the test more than once! At the end of each round, item analysis data are fed back into the system and adjustments are made based on the actual performance of the questions in the bank.

Better outcomes? Fewer questions? Tests that help you teach? Come learn what adaptive testing is all about!

Instructor Qualifications:

Dr. Rathe is the Associate Dean for Information Technology and an Associate Professor of Family Medicine at the University of Florida. He has worked extensively with instructional technology for the past ten years. He is the coauthor of four popular CD-ROM titles: Radiologic Anatomy, Human Anatomy,Microscopic Anatomy (all published by Gold Standard Multimedia, http://www.gsm.com/) and Musculoskeletal Pathology (http://www.medinfo.ufl.edu/cme/mspath/). His eval CGI program (http://medinfo.ufl.edu/omi/docs/neweval.html) has allowed the UF College of Medicine to eliminate over 90% of the paper forms once used for evaluation.

Dr. Smith is an Associate in Medical Education at the University of Florida.He has worked extensively with the family of Rasch measurement models for 25 years. He is the author of two books on Rasch measurement and the editor of two other books on Rasch measurement. He is also the founding editor of the Journal of Applied Measurement.

Richard Rathe, MD, rrathe@ufl.edu
University of Florida
http://rathe.medinfo.ufl.edu/