Breakout session
Sharing of Lessons Learned from years of CBI Development, Use, and Evaluation
Dr. Harold C. Lyon, Jr., Notre Dame College, Manchester, New Hampshire
Here are some of the Lessons I learned. The session will urge the participants to share their lessons or ask the group to suggest solutions to common problems
1. Use CBI as a supplementary tool for curriculum reform (to replace some traditional lectures and to transform others into more interactive learning by projecting CBI cases during lectures). The serendipity which invariably results is that faculty members always rethink and improve their teaching as they author new CBI courseware.
2. Though there has been a paucity of carefully controlled studies comparing CBI with traditional teaching (and meager financial support for these costly studies), several well conducted studies provide sufficiently convincing evidence of the efficiency, economic, and performance benefits of CBI to convince many medical educators to increase their efforts in developing and integrating CBI into the curricula. However regardless of convincing summative evaluation studies, always make ongoing formative on-line evaluation an integral component of the development process of CBI.
3. If and when you do a summative evaluation using post tests, remember that your CBI was designed to teach clinical reasoning and problem solving (hypothetical deductive reasoning) as well as pattern recognition, so try to create post tests (as challenging as that is!) which measure both pattern recognition (lots of images in tests) and problem solving. (Most multiple choice tests measure only factual knowledge and pattern recognition, and, at best, simple concepts, but short modified essay questions can measure problem solving.)
4. Most students and many faculty (once they have authored their own CBI) enjoy using CBI for self-paced, problem solving learning. However, given their heavy work load, students often "triage" their work , and our experience indicates that CBI will be more utilized if it is assigned as "required" and even more heavily used when students are informed that final exam questions will be taken from CBI content.
5. Encourage students to work on CBI cases in small groups of 2-4 to take advantage of the pedagogical advantages which result from small group learning.
6. Support and acknowledgment from faculty leaders in institutions is needed, such as publicly expressing that authors of peer reviewed CBI will be given credit toward promotion equivalent to what they would receive for peer reviewed, published research papers.
7. Exceptionally user-friendly authoring shells are required to facilitate faculty authoring. In the U.S. several excellent and complex authoring shells have been developed for which expectations were high that many cases soon would be authored. However, this optimism was not warranted as most busy clinicians will not take weeks, or even days, to master authoring tools. A good authoring shell should enable, with some professional pedagogical support, the authoring of a first case by a novice author in less than 10 hours and a second case in 5 hours or less.
8. Capturing content expertise and diagnostic strategies from busy clinicians is a non-trivial but vitally important task, as their knowledge constitutes the most valuable component in CBI. Before they author their case have them: a) think through what learning objectives they want the student to accomplish in the case; b) write the case out as a word processing document (patient presentation, complaint, age, sex, signs, symptoms, history, physical exam, differentail diagnosis, appropriate & inappropriate lab tests and results, leading the student through the reasoning process to a diagnostic decision and treatment. Let the patient tell his/her story on the computer. "Studies of medical outpatient consultations show that 86 % of diagnosis depends entirely on what the patients say, their own story. What doctors find on examination adds a further 6 %; and technical investigations such as x-ray, blood tests, etc. add another 8 %."); c) mark notes in the document where they want images, video, and quizzes; d) locate sources of images and video clips (slides, Internet search, colleagues); and e) write out quizzes.
9. Introduce the content expert/author with images and sound at the beginning and end of each case so students will associate the teaching with him/her.
10. Imitate, in CBI pedagogy, the techniques used by the most effective human teachers. It poses a challenge to put into the "cold" technology of the computer the very "warm" and human qualities found in research to be present in the most effective teachers (empathy, prizing, humor, frequent summaries, quizzes, and reviews with prompt feedback and encouragement), but this effort will pay dividends in learner motivation. Congratulate and encourage the students at the end of the case.
11. Learners are multi-sensory and learn best and most indelibly when teaching uses all the senses (sound, sight, touch, text.)
12. Faculty prefer to use CBI with their own personal imprint and/or teaching styles rather than commercial off-the-shelf software prepared by others, so if you want others to use your software, provide flexible ways for them to add some of their own teaching ideas and quizes, and a way to personalize it by adding their names and phone numbers in the credits section of the software
13. Try not to put more text on a page than one can see without scrolling and make it large and bold enough for those with glasses.
14. Use a map or flow chart in CBI to enable the user to always know where they are and how to navigate.
15. Humans think "by association" so utilize the opportunities hypermedia offers for students to jump around and create their own learning pathways through the content.
16. Include a short final exam at the end of each case to stress the main teaching points.
17. Have content experts peer review and critique CBI, giving constructive feedback to authors followed by revisions and improvements. This also results in motivating those faculty conducting the peer reviews to author their own CBI.
18. Have students review and critique CBI followed by author revisions before integrating into the curriculum.
19. Integration of CBI into the curriculum is a non-trivial task which is facilitated by institutional leaders supporting course directors in the use of CBI and by having teaching faculty author cases which they can use in their teaching.
20. Ten years ago the little CBI which was available was stored on computer hard drives in learning centers or libraries. In the past few years it has become available on central servers or CD-ROMS. The most prevalent direction for the present and future is to present CBI on the Internet (or local intranets) making it available from servers and databases on all computer platforms and locations, including from homes via modem. This greatly diminishes the tedious and costly problems of updating software and providing enough workstations of the same configuration.
21. Content experts usually over estimate the capability of students and often create CBI above their level of expertise.
22. CBI is not a replacement for an empathic teacher but is another tool available for the resourceful teacher. When we eliminated 40 hours of traditional teaching by substituting CBI in its place, faculty were encouraged to establish "office hours" to facilitate one-on-one mentoring of students.
23. Always enable within CBI a way for students to communicate with the content expert/author by providing phone or electronic communication.
24. Tech support in the form of a programmer, web master, or librarian is usually needed to maintain and update CBI for medical education within the curriculum.
25. Professional pedagogical and evaluation support is needed in CBI development, use, and evaluation. Physicians spend 80% of their time teaching colleagues, students, and patients; but usually they have had no training in pedagogy!
26. Collaboration with other departments, institutions, and countries in CBI development, use, and evaluation prevents "reinvention of the wheel" and is a very cost-effective practice, given the initial high cost of CBI development and evaluation and given that many creative minds are better than one!
27. Challenge your students and engage them by creating in your CBI cases a genuine problem to be solved by them -- one containing a real life story with pathos and "heart!"
This work was partially sponsored by a grant (1 FO6 TWO1953-01A2) from the Fogarty International Center of the National Institutes
Harold C. Lyon, Ed.D Professor of Health Sciences & Informatics Notre Dame College and Guest Prof.of Medical Education & Informatics Ludwig Maximilians University Munich, Germany 100040.266@compuserve.com