Using the Assessment Process to Improve Evidence-Based Information Gathering Skills for Future Audiologists
Success of students in Doctor of Audiology programs depends on the ability of the learner to find and evaluate scholarly evidence. The objective of this study is to determine if an information literacy training session on evidence-based information gathering will increase four student measures: (1) Attitudes toward gathering and evaluating scholarly evidence, (2) Knowledge about evidence-base information gathering practices, (3) Perceived value of the training regarding evidence-based searching methods, and (4) Ability to gather evidence for clinical research questions. 23 first-year audiology doctoral students (AuD) at the University of the Pacific in San Francisco participated in this study. Pre- and post-test assessments were given to students before and after a two-hour information literacy training on how to find and evaluate resources during the evidence-based information gathering process. The results support the hypothesis that understanding evidence-based practice (EBP) early in the first year of doctoral education allows the learner to gain confidence in information gathering and produce higher quality scholarly output.
Student success in graduate education is dependent on the learner’s ability to find and evaluate scholarly information. For doctoral programs in the communication sciences and disorders, high quality evidence is required to support clinic practices, research, and other scholarly activities.1 It is essential to create optimum evidence information gathering skills in doctor of audiology students at an early stage of evidence-based practice (EBP) professional education. By understanding the importance of evidence, the professional will build a strong foundation in treating patients in the future, develop lifelong critical thinking skills, and increase the quality of research in the professional field.2 The purpose of this study is to measure changes in student confidence, knowledge and behaviour in gathering evidence for research projects. This study’s hypothesis states that training in evidence-based information gathering increases confidence and improves the quality of information gathering behaviours.
An information literacy training session was conducted for 23 first-year audiology students at the University of the Pacific in San Francisco. Pre-test and post-test assessments were given to the students. Questions on the assessment instruments were created by combining traditional concepts in information literacy and evidence-based practice3 with an adaptation of Kirkpatrick’s Four-Level Training Evaluation Model suitable for higher education.4 A pre-test was given to students containing thirteen questions, with four questions adapted from an information literacy assessment developed by University of the Pacific health sciences faculty.5 The pre-test instrument contained four questions on information gathering usage and behavior (USE), four questions on confidence and attitude (ATT), four questions on knowledge (KNW), and one free writing portion: a list of resources to order in terms of how likely the student would reference them in an essay.
The students were then given a two-hour information literacy training on how to find and evaluate resources during the evidence-based searching process, using databases inclined towards research in communication disorders.6 The post-test assessment contained the same questions as the pre-test assessment. An additional four questions were provided on the post-test instrument assessing the students’ reactions and values regarding the training session.
Information Gathering Usage and Behaviour
The pre- and post-test assessments consisted of four questions designed to determine how students were gathering and using evidence. Median values showed a wide range of information gathering practices (Figure 1). These data suggest room for improvement in the students’ practices before completing the information literacy training session.
One of the session’s goals was to shift student attitudes in regards to information gathering, with the assumption that as new information was acquired, attitudes would shift and practices would follow. In order to assess shifts in attitude toward evidence-based information gathering, we used a Wilcoxon signed-rank test to compare ratings on the four CNF questions before and after the information session.
Results of these comparisons showed significant differences (p<0.05) in three out of four CNF questions. Figure 2 shows histograms for student responses from the pre- (black bars) and post- (red bars) session questionnaires. Positive measures of attitude change are suggested by shifts toward the “Agree” response.
The second goal was to increase student knowledge about evidence-based practices. To determine whether this occurred, the Wilcoxon signed-rank test was used to compare pre- and post-session ratings on the four KNW questions.
In this section, all four comparisons showed significant shifts towards positive learning outcomes (p<0.05).
Change in Behaviour
The third goal was to improve student behaviour in regards to evidence-based information gathering. In order to assess behaviour, we asked students to rank how likely they were to use each of the seven reference sources. The sources were ranked by an expert (the teacher of the course) from one to seven, with one being the best information source and seven being the worst.
Figure 4 shows the median student rankings before (pre) and after (post) the information gathering education class. These data show that the student rankings more closely follow the expert rankings after the information gathering class, suggesting that behavior regarding referencing shifted.
Mean values of the four questions relating to student assessment of the session’s value and clarity were very high, with one being the highest rating (Figure 5). This suggests that the students perceived the material to be relevant, important, and clearly presented.
Evidence-based practice is one of the most valuable strategies mastered by not only AuD students, but also audiologists worldwide. Data from this study reveals that trainings designed to improve students’ evidence-based information gathering skills significantly improves confidence and proficiency in this area. As most research method classes and capstone projects are planned later in the AuD curriculum, students will need to rely upon evidence-based practice skills, especially information gathering skills. The results of this study suggest that a training session on evidence-based information gathering can significantly improve EBP skills, can be accomplished in a short time frame, and conducted early in AuD education. The impact of future research in this area may guide education practices in fields that rely on evidence-based practice, such as neuroscience, pre-med, pharmacy, and patient-based professional practices. As such, students will build confidence with evidence-based learning strategies, resulting not only in improved awareness, but also in the rigor of scholarly output.
- Nail-Chiwetalu B and Ratner N. An assessment of the information-seeking abilities and needs of practicing speech-language pathologists. J Med Library Assoc 2007;95(2):182–88.
- Dollaghan C. Evidence-based practice in communication disorders: What do we know, and when do we know it? J Communicat Dis 2004;37(5):391–400. doi: 10.1016/j.jcomdis.2004.04.002
- Nail-Chiwetalu B and Ratner N. Information literacy for speech-language pathologists: A key to evidence-based practice. Lang Speech Hear Serv Schools 2006;37(3):157–67.
- Praslova L. Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in higher education. Educat Assess Eval Account 2010;22(3):215–25.
- Stark R and Oza P. Information literacy pre-test questionnaire. Unpublished instrument; 2015.
- Grabowsky A. Library instruction in communication disorders: Which databases should be prioritized? Issues Sci Technol Librarian 2015. doi:10.5062/F4707ZFB