The Use of Audience Response System Technology With Limited-English-Proficiency, Low-Literacy, and Vulnerable Populations

Matthew C. Keifer, MD, MPH; Iris Reyes, MPH; Amy K. Liebman, MA, MPA; Patricia Juarez-Carrillo, PhD, MPH.  Abstract. Audience response systems (ARS) have long been used to improve the interactivity of educational activities. Most studies of ARS have addressed education of literate trainees. How well these devices work with low-literacy subjects is not well studied. Information gathering on the training audience is an important use of ARS and helpful in improving the targeting of training information. However, obtaining demographic information from vulnerable populations with reasons to be concerned about divulging information about themselves has not been tested. In addition, a culturally competent method to effectively collect demographic and evaluation data of this growing population is essential. This project investigated the use of ARS to gather information from Hispanic immigrant workers, many of whom are socially vulnerable and have limited English proficiency (LEP) and low-literacy. Workers attended focus groups and were asked to use ARS devices or clickers to respond to questions. Questions were both catergorical (multiple choice) and open-ended numerical (text entry), and varied from simple queries to more sensitive points regarding immigration. Most workers answered the one-key response categorical questions with little difficulty. In contrast, some participants struggled when responding to numerical questions, especially when the response required pressing multiple clicker keys. An overwhelming majority of participants reported that the clickers were comfortable and easy to use despite the challenges presented by the more complex responses. The error rate increased as question complexity increased and the trend across three ordered categories of response complexity reached statistical significance. Results suggest that ARS is a viable method for gathering dichotomous or higher-order categorical information from LEP and low-literacy populations in a group setting while assuring anonymity. However, it is recommended that clickers be developed and tested with fewer, bigger, and more widely separated buttons, and less printing on the buttons for these populations. Further research is needed to determine the effectiveness of using clickers with simplified configurations in the workplace as a tool to collect data for surveys and assessments and to better engage LEP and low-literacy workers in training sessions.