Currently viewing Vol. 3 • Issue 6 • 2016

Using the Assessment Process to Improve Evidence-Based Information Gathering Skills for Future Audiologists

By Mickel Paris, MLIS Jiong Hu, PhD, AuD Veronica Koo, BS Susanna Marshall, BA Gabriella Musacchia, PhD

This page as PDF

ABSTRACT

Success of students in Doctor of Audiology programs depends on the ability of the learner to find and evaluate scholarly evidence. The objective of this study is to determine if an information literacy training session on evidence-based information gathering will increase four student measures: (1) Attitudes toward gathering and evaluating scholarly evidence, (2) Knowledge about evidence-base information gathering practices, (3) Perceived value of the training regarding evidence-based searching methods, and (4) Ability to gather evidence for clinical research questions. 23 first-year audiology doctoral students (AuD) at the University of the Pacific in San Francisco participated in this study. Pre- and post-test assessments were given to students before and after a two-hour information literacy training on how to find and evaluate resources during the evidence-based information gathering process. The results support the hypothesis that understanding evidence-based practice (EBP) early in the first year of doctoral education allows the learner to gain confidence in information gathering and produce higher quality scholarly output.

INTRODUCTION

Student success in graduate education is dependent on the learner’s ability to find and evaluate scholarly information. For doctoral programs in the communication sciences and disorders, high quality evidence is required to support clinic practices, research, and other scholarly activities.1 It is essential to create optimum evidence information gathering skills in doctor of audiology students at an early stage of evidence-based practice (EBP) professional education. By understanding the importance of evidence, the professional will build a strong foundation in treating patients in the future, develop lifelong critical thinking skills, and increase the quality of research in the professional field.2 The purpose of this study is to measure changes in student confidence, knowledge and behaviour in gathering evidence for research projects. This study’s hypothesis states that training in evidence-based information gathering increases confidence and improves the quality of information gathering behaviours.

METHODS

An information literacy training session was conducted for 23 first-year audiology students at the University of the Pacific in San Francisco. Pre-test and post-test assessments were given to the students. Questions on the assessment instruments were created by combining traditional concepts in information literacy and evidence-based practice3 with an adaptation of Kirkpatrick’s Four-Level Training Evaluation Model suitable for higher education.4 A pre-test was given to students containing thirteen questions, with four questions adapted from an information literacy assessment developed by University of the Pacific health sciences faculty.5 The pre-test instrument contained four questions on information gathering usage and behavior (USE), four questions on confidence and attitude (ATT), four questions on knowledge (KNW), and one free writing portion: a list of resources to order in terms of how likely the student would reference them in an essay.

The students were then given a two-hour information literacy training on how to find and evaluate resources during the evidence-based searching process, using databases inclined towards research in communication disorders.6 The post-test assessment contained the same questions as the pre-test assessment. An additional four questions were provided on the post-test instrument assessing the students’ reactions and values regarding the training session.

RESULTS

Information Gathering Usage and Behaviour

The pre- and post-test assessments consisted of four questions designed to determine how students were gathering and using evidence. Median values showed a wide range of information gathering practices (Figure 1). These data suggest room for improvement in the students’ practices before completing the information literacy training session.

Figure 1

Figure 1. Initial assessment of evidence-based information usage.

Confidence Shift

One of the session’s goals was to shift student attitudes in regards to information gathering, with the assumption that as new information was acquired, attitudes would shift and practices would follow. In order to assess shifts in attitude toward evidence-based information gathering, we used a Wilcoxon signed-rank test to compare ratings on the four CNF questions before and after the information session.

Results of these comparisons showed significant differences (p<0.05) in three out of four CNF questions. Figure 2 shows histograms for student responses from the pre- (black bars) and post- (red bars) session questionnaires. Positive measures of attitude change are suggested by shifts toward the “Agree” response.

Figure 2

Figure 2. Confidence in information literacy.

Increased Knowledge

The second goal was to increase student knowledge about evidence-based practices. To determine whether this occurred, the Wilcoxon signed-rank test was used to compare pre- and post-session ratings on the four KNW questions.

Figure 3

Figure 3. Information literacy knowledge.

In this section, all four comparisons showed significant shifts towards positive learning outcomes (p<0.05).

Change in Behaviour

The third goal was to improve student behaviour in regards to evidence-based information gathering. In order to assess behaviour, we asked students to rank how likely they were to use each of the seven reference sources. The sources were ranked by an expert (the teacher of the course) from one to seven, with one being the best information source and seven being the worst.

Figure 4 shows the median student rankings before (pre) and after (post) the information gathering education class. These data show that the student rankings more closely follow the expert rankings after the information gathering class, suggesting that behavior regarding referencing shifted.

Figure 4

Figure 4. Evidence and reference usage behaviour.

Educational Value

Mean values of the four questions relating to student assessment of the session’s value and clarity were very high, with one being the highest rating (Figure 5). This suggests that the students perceived the material to be relevant, important, and clearly presented.

Figure 5

Figure 5. Perceived class value and satisfaction.

CONCLUSION

Evidence-based practice is one of the most valuable strategies mastered by not only AuD students, but also audiologists worldwide. Data from this study reveals that trainings designed to improve students’ evidence-based information gathering skills significantly improves confidence and proficiency in this area. As most research method classes and capstone projects are planned later in the AuD curriculum, students will need to rely upon evidence-based practice skills, especially information gathering skills. The results of this study suggest that a training session on evidence-based information gathering can significantly improve EBP skills, can be accomplished in a short time frame, and conducted early in AuD education. The impact of future research in this area may guide education practices in fields that rely on evidence-based practice, such as neuroscience, pre-med, pharmacy, and patient-based professional practices. As such, students will build confidence with evidence-based learning strategies, resulting not only in improved awareness, but also in the rigor of scholarly output.

REFERENCES

  1. Nail-Chiwetalu B and Ratner N. An assessment of the information-seeking abilities and needs of practicing speech-language pathologists. J Med Library Assoc 2007;95(2):182–88.
  2. Dollaghan C. Evidence-based practice in communication disorders: What do we know, and when do we know it? J Communicat Dis 2004;37(5):391–400. doi: 10.1016/j.jcomdis.2004.04.002
  3. Nail-Chiwetalu B and Ratner N. Information literacy for speech-language pathologists: A key to evidence-based practice. Lang Speech Hear Serv Schools 2006;37(3):157–67.
  4. Praslova L. Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in higher education. Educat Assess Eval Account 2010;22(3):215–25.
  5. Stark R and Oza P. Information literacy pre-test questionnaire. Unpublished instrument; 2015.
  6. Grabowsky A. Library instruction in communication disorders: Which databases should be prioritized? Issues Sci Technol Librarian 2015. doi:10.5062/F4707ZFB
About the authors

Mickel Paris, MLIS

Mickel Paris is an assistant professor and the Health Sciences Librarian for University of the Pacific.

Jiong Hu, PhD, AuD

Jiong Hu, PhD, AuD is an Assistant Professor at the University of the Pacific in San Francisco, CA. He is a fellow of the American Academy of Audiology. Most recently his research has focused on examining the impact of bilirubin-induced neurotoxicity on the auditory system in newborns and infants, as well as the effect of long-term auditory exposure on pitch coding ability at the brainstem level in adults with different language backgrounds.

Veronica Koo, BS

Veronica Koo is an AuD student at the University of the Pacific in San Francisco.

Susanna Marshall, BA

Susanna Marshall is an AuD student at the University of the Pacific in San Francisco.

Gabriella Musacchia, PhD

Dr. Gabriella Musacchia, PhD, is an Associate Professor and Department Chair of Audiology at the University of the Pacific and a Research Scholar in the Department of Otolaryngology Head and Neck Surgery at Stanford University. Dr. Musacchia’s area of scholarly expertise is in the central auditory nervous system. She enjoys sharing scholarly activities, experiences, and publications with her students and colleagues. Dr. Musacchia married Dr. Matthew Fitzgerald, fellow Northwestern grad and current Chief of Audiology at Stanford University. They have a beautiful son and live in San Francisco.