High-level Architecture

OpenClinica

1. Users create study datasets. The dataset consists of all the study items that the users would like to import into OpenSpecimen database.
2. Subsequently, users create one scheduled job per dataset to export the corresponding study data. The scheduled job can be configured to export the data either on nightly or weekly basis depending upon the study activities. The scheduled job should be configured to export the data in ODM XML 1.3 format.

OpenSpecimen

1. The CP administrator configures the OpenClinica studies whose data needs to be imported or synchronised into OpenSpecimen database. Among other details, this configuration specifies how to pick the ODM XML data files from OpenClinica. The data files could be residing either on a local filesystem or remote filesystem. OpenSpecimen can handle both scenarios.
2. A single pre-created scheduled job runs at pre-determined intervals and performs the following tasks for each configured study in the OpenSpecimen:
   1. downloads the study data files exported by OpenClinica using either SFTP or local filesystem copy command.
   2. parses the data files into in-memory structures representing the CRF metadata and data records.
   3. validates the CRF metadata and records.
   4. creates/updates corresponding forms and records in OpenSpecimen DB.
   5. notifies users with metrics like how many forms, participants, visits, specimen, and form records were created/updated.
3. Unlike OpenClinica, there is only one scheduled job in OpenSpecimen that is responsible for importing the data. Depending on the requirements, the frequency of the scheduled job can be changed to run on hourly or daily or weekly basis.