Abstract
Background/Aims Distributed research networks (DRNs) represent a mechanism by which electronic health care data can be securely shared and analyzed across multiple sites. DRNs may feature a menu-driven interface that allows researchers to query structured data sets that reside behind each participating site’s firewall. These queries return aggregated data tables that contain simple frequencies, such as the number of patients by age, gender, diagnosis, treatment or procedure. It is hypothesized that researchers would use a menu-driven interface to conduct preparatory-to-research queries for power analyses that require data from multiple sites. The objective of this study was to evaluate the usability of a menu-driven interface developed for the Scalable Partnering Network for Comparative Effectiveness Research (SPAN).
Methods To evaluate the functionality of the SPAN menu-driven interface, we conducted twelve 90-minute usability testing sessions with researchers from 4 SPAN sites. In each testing session, the user was asked to perform sample queries prepared by the research team. As the user performed a sample query, a moderator asked questions about the functionality of the interface. At the end of each testing session, user satisfaction was also measured with the System Usability Scale (SUS), a widely used survey tool for quantifying subjective assessments of usability. The usability testing sessions were audiotaped and analyzed using standard qualitative coding techniques.
Results In general, users thought the interface would be a useful and efficient tool for conducting multi-site preparatory-to-research queries. Users also found the layout of the interface to be logical, easy to navigate and visually pleasing. Some investigators claimed they would use the tool themselves, while others would go through an analyst. Some users expressed concern that the interface was a “black box,” where they did not have enough information about the nuances of the underlying data. There were several requests to increase the interface’s functionality, such as incorporating denominator data to calculate rates. The mean SUS score was 77.7, which indicates an above average level.
Conclusions The SPAN menu-driven interface performed well in testing and appears to be a useful tool for multi-site research. Future enhancements are being considered.
