Noa Segall

The objective of this study was to assess the plausibility of using computational cognitive models for evaluating the usability of human–machine interfaces in supervisory control of high-throughput (biological) screening (HTS) operations. Usability evaluations of new interface prototypes were conducted by comparisons with existing technologies. Model assessment occurred through comparison with human test results. Task completion times and the number of errors were recorded during human performance trials, and task time was predicted in cognitive model trials in tests with two HTS interfaces. Computational GOMSL (Goals, Operators, Methods, and Selection rules Language) models were constructed based on a combination of cognitive task analyses (abstraction hierarchy modeling and goal-directed task analysis). The usability tests revealed improvements in task performance with the new prototypes. The cognitive model outputs were correlated with actual human performance, and the approach was considered useful for evaluating the usability of new interfaces in life sciences automation in the future.

ABSTRACT Various IOM reports highlight urgency for health professions education reform, most notably the increase of cross-disciplinary education. With an estimated 44,000-98,000 preventable annual deaths in the US due to medical errors; research indicates that two factors, teamwork and communication, contribute to 70% of such errors. AHRQ developed the TeamSTEPPS curriculum to improve patient safety by promoting improved communication and leadership among all members of a healthcare team. The TeamSTEPPS curriculum was tested in two successive years of medical-nursing student teamwork exercises. In Year 1, n =438 students participated in a one-day training, which included TeamSTEPPS training, four alternative educational interventions, and outcomes teamwork measures. In Year 2, n = 397 students participated in a similar training but with more intensified exercises. In both years, students were pre- and post-tested with knowledge and attitudes instruments. The pre- and posttest attitudes instrument had strong validity with internal consistencies of .824 and .859, respectively, and an eight factor with R2 = .545. In both years, experimental cohorts significantly improved on teamwork attitudes from pre-to-posttest (F = 48.7, p = .000). Likewise, teamwork knowledge significantly improved for all cohorts. In Year 2, students who received more intensified training significantly improved on both attitudes and knowledge compared to students who had not received such training. The teamwork training significantly improved student knowledge and attitudes on interdisciplinary teamwork for improving patient safety. We found that more intensified training was more effective in achieving student learning and communication skills than brief interventions involving different instructional modalities.

Eye tracking can be a valuable tool for collecting data about perception and attention in task performance, but its use in human factors research has been limited. This may be due to the fact that the coding and visualization of eye tracking data can be difficult and time-...

ABSTRACT Introduction/Background: Over 370,000 cardiopulmonary resuscitations are attempted each year in US hospitals.1 For many, pulseless ventricular tachycardia or ventricular fibrillation (VT/VF) is the first monitored arrhythmia, which may be treated successfully with prompt defibrillation. Yet for 30% of patients, defibrillation is delayed more than 2 minutes, reducing their chance of survival to hospital discharge by half.2 To increase the potential for timely detection of cardiac events, many at-risk patients are monitored by telemetry technicians. However, decisions regarding the appropriate number of patients that a single technician may safely and effectively monitor are primarily based on technological capabilities and not on our understanding of human information processing limitations. Simulation provides an opportunity to measure responses to life-threatening cardio-respiratory events in a time frame and with a degree of accuracy not feasible through assessment of response to true events. Our objective is to determine the impact of increasing the number of patients monitored on response time to cardio-respiratory events. To achieve this objective, we designed simulation of cardiac telemetry monitoring Methods: Currently, we are carrying out a randomized controlled trial to compare response times to VF across five number-of-patient conditions. The simulation replicates the work of cardiac telemetry technicians using a combination of real patient data and a simulated patient experiencing VF. We video and audio-recorded true patient data with a single simulated patient embedded in the patient set. The technical implementation involved connecting an ECG rhythm simulator into the hospital's network that transmits physiological signals to remote telemetry monitors. The signal appears in exactly the same way as it would appear for a real patient. Study participants are randomly assigned to one of the 5 patient loads and complete a four-hour monitoring session, during which they perform tasks similar to real monitoring work, such as calling patients' providers to report events and documenting changes. After about 3 hours, the simulated patient sustains VF and the time required for the participants to report this arrhythmia is recorded. Study participants are asked to complete a survey regarding the realism of the simulation. 28 technicians and nurses from cardiac units have completed the study to date. As expected, there is a trend of increasing response time to the lethal arrhythmia as the number of patients monitored increases. Survey Results are presented in the Table. Results: Conclusion: We simulated the work of remote cardiac telemetry technicians to study the effect of increasing the number of patients monitored on response time to a simulated arrhythmia. Overall, study participants perceived the experience to be relatively realistic. The knowledge to be gained will inform efforts to study this problem in real-world cardiac telemetry. References: 1. Ballew KA, Philbrick JT. Causes of variation in reported in-hospital CPR survival: A critical review. Resuscitation 1995;30:203-15. 2. Chan PS, Krumholz HM, Nichol G, Nallamothu BK. Delayed time to defibrillation after in-hospital cardiac arrest. N Engl J Med 2008;358:9-17. Disclosures: Abbott, Pfizer Guidepoint, Maven Applied Research Associates-proprietary interest in jointly developed projects BMS, Pfizer, The Medicines Company, Merck, Novartis, Medtronic, Boehringer Ingelheim, Astra Zeneca BMS, Pfizer, The Medicines Company, Boehringer Ingelheim, Daiitchi, Astra Zeneca.

The electronic personal health record (PHR) has been championed as a mediator of patient-centered care, yet its usability and utility to patients, key predictors of success, have received little attention. Human-centered design (HCD) offers validated methods for studying systems effects on users and their cognitive tasks. In HCD, user-centered activities allow potential users to shape the design of the end product and enhance its usability. We sought to evaluate the usability and functionality of HealthView, the PHR of the Duke University Health System, using HCD methods. Study participants were asked to think aloud as they carried out tasks in HealthView. They then completed surveys and interviews eliciting their reactions to the web portal. Findings were analyzed to generate redesign recommendations, which will be incorporated in a future release of HealthView.

Sudden cardiac arrest is the leading cause of death in the United States. Despite new therapies, progress in this area has been slow, and outcomes remain poor even in the hospital setting, where providers, drugs, and devices are readily available. This is partly attributed to the quality of resuscitation, which is an important determinant of survival for patients who experience cardiac arrest. Systems problems, such as deficiencies in the physical space or equipment design, hospital-level policies, work culture, and poor leadership and teamwork, are now known to contribute significantly to the quality of resuscitation provided. We describe an in situ simulation-based quality improvement program that was designed to continuously monitor the cardiac arrest response process for hazards and defects and to detect opportunities for system optimization. A total of 72 simulated unannounced cardiac arrest exercises were conducted between October 2010 and September 2013 at various locations throughout our medical center and at different times of the day. We detected several environmental, human-machine interface, culture, and policy hazards and defects. We used the Systems Engineering Initiative for Patient Safety (SEIPS) model to understand the structure, processes, and outcomes related to the hospital's emergency response system. Multidisciplinary solutions were crafted for each of the hazards detected, and the simulation program was used to iteratively test the redesigned processes before implementation in real clinical settings. We describe an ongoing program that uses in situ simulation to identify and mitigate latent hazards and defects in the hospital emergency response system. The SEIPS model provides a framework for describing and analyzing the structure, processes, and outcomes related to these events.

Remotely monitored patients may be at risk for a delayed response to critical arrhythmias if the telemetry watchers who monitor them are subject to an excessive patient load. There are no guidelines or studies regarding the appropriate number of patients that a single watcher may safely and effectively monitor. Our objective was to determine the impact of increasing the number of patients monitored on response time to simulated cardiac arrest. Randomized trial. Laboratory-based experiment. Forty-two remote telemetry technicians and nurses from cardiac units. Number of patients monitored in a simulation of cardiac telemetry monitoring work. We carried out a study to compare response times to ventricular fibrillation across five patient loads: 16, 24, 32, 40, and 48 patients. The simulation replicated the work of telemetry watchers using a combination of real recorded patient electrocardiogram signals and a simulated patient experiencing ventricular fibrillation. Study participants we...

ABSTRACT As capabilities for storing and sharing electronic health data expand, it becomes increasingly important to classify, prioritize, and contextually link patient health data. Providers using electronic health records (EHR) complain that it is difficult for them to rapidly find information they need and that key information can be missed. Unfortunately, phases of human-centered design that support defining information needs and developing an understanding of the work environment are sometimes shortchanged. We present a knowledge elicitation approach for the purposes of supporting EHR design that combines the rich contextual data gained through recording information use activities during actual patient care followed by retrospective verbal protocol interviews of the provider as they watch and listen to playback of their information use activities. As a means of analyzing qualitative data collected through techniques such as these, we apply grounded theory based content analysis techniques. The goal of our research is the development of principles to support better organization and prioritization in the presentation of electronic health data. As a secondary objective, we also seek to identify criteria that, with computational algorithms, may be used to prioritize specific data in context to support intelligent contextualized information display. In this paper, we present details of our methods (data collection and analyses), preliminary findings that have resulted from these efforts, and specific advantages and disadvantages to our approach.

The objective of this study was to assess the plausibility of using computational cognitive models for evaluating the usability of human–machine interfaces in supervisory control of high-throughput (biological) screening (HTS) operations. Usability evaluations of new interface prototypes were conducted by comparisons with existing technologies. Model assessment occurred through comparison with human test results. Task completion times and the number of

The authors developed a Standardized Assessment for Evaluation of Team Skills (SAFE-TeamS) in which actors portray health care team members in simulated challenging teamwork scenarios. Participants are scored on scenario-specific ideal behaviors associated with assistance, conflict resolution, communication, assertion, and situation assessment. This research sought to provide evidence of the validity and feasibility of SAFE-TeamS as a tool to support the advancement of science related to team skills training. Thirty-eight medical and nursing students were assessed using SAFE-TeamS before and after team skills training. The SAFE-TeamS pretraining and posttraining scores were compared, and participants were surveyed. Generalizability analysis was used to estimate the variance in scores associated with the following: examinee, scenario, rater, pretraining/posttraining, examinee type, rater type (actor-live vs. external rater-videotape), actor team, and scenario order. The SAFE-TeamS scores reflected improvement after training and were sensitive to individual differences. Score variance due to rater was low. Variance due to scenario was moderate. Estimates of relative reliability for 2 raters and 8 scenarios ranged from 0.6 to 0.7. With fixed scenarios and raters, 2 raters and 2 scenarios, reliability is greater than 0.8. Raters believed SAFE-TeamS assessed relevant team skills. Examinees' responses were mixed. The SAFE-TeamS was sensitive to individual differences and team skill training, providing evidence for validity. It is not clear whether different scenarios measure different skills and whether the scenarios cover the necessary breadth of skills. Use of multiple scenarios will support assessment across a broader range of skills. Future research is required to determine whether assessments using SAFE-TeamS will translate to performance in clinical practice.