Abstract
Background/Aims Clinical research trials often rely on automated data systems, such as the electronic medical record to create and maintain a patient registry, to determine a participant’s progress through the study, or for analytic purposes. Data might come from a variety of sources and include such elements as procedures, lab results, hospitalization, vital status, enrollment and participant responses from survey software. An automated process for adding outside data to a study tracking database can save significant time over abstraction or manual data import processes, and can allow determination of an individual’s study status in close to real time.
Methods There are a number of ways of setting up an automated import process, depending on data needs and availability. Over the course of several clinical trials, we have developed a process with the following steps, all occurring automatically on a scheduled basis: 1. A SAS program performs Extract/Transform/Load (ETL) tasks: 1a. Extract and transform the desired data from each source. 1b. Load new data into a SAS dataset for historical purposes and into a staging table in the tracking database preparatory to actual use. 1c. Generate automated emails when specific milestones occur (e.g., study end date reached or first use of a given lab test code). 2. Another SAS program checks the ETL logs and sends email about any errors occurring in the ETL process. 3. A database job calls stored procedures to insert data from the staging tables into the main study events table and send email detailing how many records were processed. 3a. If manual intervention is needed before the final data is loaded, the user can call the stored procedure once data entry is complete. 3b. A master stored procedure can call individual procedures in order (e.g., importing a positive lab result may trigger randomization).
Results The ETL process described above has proven itself to be robust and adaptable in a variety of study contexts.
Conclusions An automated system for adding clinical data to a patient registry or tracking database can save staff time, allow access to near real-time data, and facilitate integration of data from different sources.
