Got feedback or spotted a mistake?

Leave a comment at the end of this page or email contact@krishagni.com

REDCap-FAQs and Errors

Owned by Saurav Singh
Last updated: Apr 17, 2024 by Ankita Chavan
9 min read

Important

Which version of REDCap is required for the plugin to work?

Ans: The plugin supports most of the latest versions of REDCap. We usually keep up to date with the new releases of REDCap.

Will the interface be real-time? Is the data entered into REDCap will be available in OpenSpecimen right away?

Ans: You can configure the auto-synchronization run on a nightly, hourly, or minutely basis. You will find details about it here.

If you configure it to a very high frequency (like every minute) and have many CRFs and CPs, REDCap might block the OpenSpecimen IP address for excessive API calls.

What is the difference in handling classical vs longitudinal REDCap projects?

Ans: For classical REDCap projects, there is no concept of visits. Therefore all REDCap records are tied at the participant level in OpenSpecimen.

For longitudinal projects, users are provided with an option to map the REDCap project events to corresponding events in OpenSpecimen. As a result, all the instruments and their data from REDCap are duplicated at the visit level in OpenSpecimen.

Deploying

Why does the plugin use the redcap_log_events table instead of REDCap’s export records API?

Ans: This is by design. The REDCap’s export records API gives a snapshot of ALL records of the project. This is undesirable for the following reasons - 

  1.  Fetching all project records for a medium to a large project can be quite taxing on REDCap and OpenSpecimen server resources.

  2.  The plugin (frequently) synchronizes records between REDCap and OpenSpecimen. The plugin requires only those changes that have occurred in the REDCap project since the last synchronization run. There is no need to fetch all records every time the plugin run synchronization.

  3. The plugin has been designed to fetch changes in the REDCap project in manageable chunks of 100 changes. It ensures that there is no undue memory pressure either on the REDCap server or OpenSpecimen server.

The redcap_log_events table will have no changes recorded for copied REDCap projects. How is this scenario handled?

Ans: Although REDCap provides an option to copy records, we believe users copy REDCap projects mostly to replicate the metadata (instruments, arms, events, etc.) and not the data records. Also, please note, by default, REDCap does not copy records when a project is copied. That being said, we will consider enhancing the plugin to provide an option to import the REDCap records CSV files.

Why is a custom REDCap PHP plugin required?

Ans: Refer above question for the preamble. The custom REDCap PHP plugin is required for the following reasons:

  1. There is no REDCap API to fetch changes in data that have occurred in a project since a given time point. The API always responds with ‘ALL changes’. The custom PHP plugin solves the problem by implementing an API that provides options like time point from when the changes need to be picked, how many changes, etc.

  2. For longitudinal projects, the REDCap events API does not provide event IDs. It poses a problem in enforcing mapping between REDCap and OpenSpecimen events when there is a change in the REDCap event name (including unique_event_name). The custom PHP plugin implements an API to export project events along with their IDs. The OpenSpecimen plugin then uses these event IDs to create events mapping between REDCap and OpenSpecimen. The event IDs remain the same even when there is a change of event name. Significantly, data change records fetched from REDCap contain references to event ID.

Synchronization Run

How are visits synchronized between REDCap and OpenSpecimen?

>>If the user has multiple visit dates in redcap, and we mapped them into OS, but

>>there is no visit names (required field in OS) in redcap.  After we map the visit date into

>>OpenSpecimen, can OS automatically generate visit names and required visit fields in

>>OpenSpecimen to be able to let the user start to enter the specimen information?

Ans: Yes, visits will be created automatically using the default visit naming scheme. You can then collect specimens under that visit.

How to find the participant and visit records added from REDCap project synchronization?

Ans: Below, SQL can be used to prepare the participant list and visit records that were added from REDCap project synchronization.

Below SQLs can be used to prepare the list of participants and visit records that were added from REDCap project synchronization.

Participants

select cp.short_title as "CP", p.name as "Project", a.protocol_participant_id as "PPID" from os_redcap_records r inner join os_redcap_projects p on p.identifier = r.project_id inner join catissue_coll_prot_reg_aud a on a.identifier = r.cpr_id inner join catissue_collection_protocol cp on cp.identifier = a.collection_protocol_id where -- 0 for insert, 1 for update a.revtype = 0 order by r.identifier;

Visits

select cp.short_title as "CP", p.name as "Project", a.identifier as "Visit ID", a.name as "Visit Name" from os_redcap_records r inner join os_redcap_record_visits vr on vr.record_id = r.identifier inner join os_redcap_projects p on p.identifier = r.project_id inner join cat_specimen_coll_group_aud a on a.identifier = vr.visit_id inner join catissue_coll_prot_reg reg on reg.identifier = a.collection_protocol_reg_id inner join catissue_collection_protocol cp on cp.identifier = reg.collection_protocol_id where -- 0 for insert, 1 for update a.revtype = 0 order by r.identifier;

Is it possible to get a subset of participants from REDCap?

Ans: No. Example case: There are 1000 participants in REDCap, but only 100 are to be registered to OpenSpecimen. It is not possible to fetch a subset of participants from REDCap. But if you want to register the only specific types of participants such as participants who are female and above 18 years old, then you can use the 'Record Inclusion Rule' as shown in the section here.

Is it possible to prefix values fetched from REDCap?

Ans: No. Example case: Append RC project ID to the PPID/MPI in OpenSpecimen.

Forms and Mapping

What if the fields are not available in OpenSpecimen's default setting? Does it mean that we first need to create the custom form to store these fields?

Ans: No, the missing fields are automatically created as custom forms/fields by the plugin.

Can the plugin map the REDCap fields into existing custom form fields? 

Ans: No, the plugin cannot map the REDCap fields into existing custom forms. The plugin will create new forms for each REDCap instrument being synced.

Why are events mapping required?

Ans: Events mapping is exclusively for longitudinal projects. It specifies the anticipated or existing visits to which the instrument records from the REDCap are linked.

Is it mandatory to specify events mapping? What happens when events mapping is not specified?

Ans: No, it is not required to specify events mapping. Unplanned visits are created in the absence of such mapping, and the records are linked to those visits.

Is it mandatory to map all REDCap events?

Ans: No, it is not required to map all REDCap events to the CP visits. However, unplanned visits are created for each unmapped REDCap event.

Is it mandatory to specify the visit fields mapping?

Ans: No, it is not mandatory. However, it is good practice to at least map the visit date.

How are mandatory visit fields populated when visit fields are not mapped?

Ans: The mandatory visit fields are populated as follows:

  1. When the visit date field is not mapped, its value is set to the date/time of the corresponding REDCap event data entry.

  2. When the visit site field is not mapped, its value is picked either from the mapped event's default site or the project site.

What sites are available for selection in the project site dropdown?

Ans: The project site dropdown is restricted to the sites linked with the collection protocol.

REDCap Data

If we remove forms or fields in REDCap after the go-live date, would OS remove them for us automatically during the overnight job?

Ans: Yes, it will. However, in the regular runs, it will only pull the records created from the previous run. If you want ALL the data for the new form, you have to re-run the full synchronization.

If we create a new form in REDCap after the go-live date, would OS sync it for us automatically during the overnight job?

Ans: It depends on the type of configuration you set during the REDCap configuration:

  1. The new instrument would be configured automatically if the forms were not defined in the 'Form inclusion/exclusion list'.

  2. If any forms are included in the list, you will have to add the new form in the include list.

Why does OpenSpecimen does not delete participants that are deleted in REDCap?

Ans: Since specimen information such as collection, storage, distribution, etc., might be still crucial to the researcher, we do not delete the participant forcefully. We expect that the end-user to login to the OpenSpecimen server and manually delete the participant.

Why does OpenSpecimen pull deleted participants from REDCap?

Ans: OpenSpecimen process the REDCap audit logs in chronological order. Since it first encounters the insert participant audit log, it creates the participant in OpenSpecimen. Later, when it encounters the delete participant audit log, it ignores it since (as per the previous question) OpenSpecimen does not delete participants forcefully. These participants have to be deleted manually.

How to handle data deletes in OpenSpecimen?

Ans: OpenSpecimen maintains a few mapping tables to maintain the mapping between OS-RC objects. To do a total cleanup and resync, you must either truncate or delete rows from these tables.

From v6.3 onwards, the mapping tables can be cleaned from the user interface itself. To do a full resync of the data, delete the data from OpenSpecimen using bulk import. Then navigate to the REDCap settings and click on ‘Delete Data’ under ‘More’. It will clear all the mapping tables, and now the data can be resynced into OpenSpecimen.

Before version v6.3, the cleanup needs to be done at the database level by truncating or deleting rows from these tables.

  1. os_redcap_projects - Table that stores all the RC project details.

  2. os_redcap_records - Table that stores all the RC participant mapping.

  3. os_redcap_instrument_instances - Table that stores all the RC visit form mapping

  4. os_redcap_record_visits - Table that stores all the RC visit mapping (only for longitudinal studies)

If you delete a participant or visit in OpenSpecimen and resync ALL data from RC, it will not pull the data for deleted OS objects from RC. However, in the odd case, where you want the resync to fetch ALL data, you will need to delete the entries from the below table:

  1. os_redcap_records: delete participant mapping

  2. os_redcap_record_visits: delete visit mapping

  3. os_redcap_instrument_instances: join with #1 (and/or #2) above and delete the instrument entries

  4. If you want to resync all: delete all the entries for a project by joining with os_redcap_projects.

Errors

“java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String"

Solution: OpenSpecimen versions older than v6.1 are not compatible with REDCap 8.10. Due to which when setting up the project, this error is displayed on the user interface. Hence, you need to update to the higher version of OpenSpecimen.

Error Connecting to REDCap API

This error is raised when the OpenSpecimen server is unable to contact the REDCap server. It can happen due to any of the following reasons:

  1. UnknownHostException: The REDCap server hostname could not be resolved to a valid IP address.

    1. Ensure the hostname specified in the server config is correct. 

    2. Contact system/network admin to configure the appropriate DNS entries to resolve the REDCap hostname to the corresponding IP address.

  2. ConnectException - Connection Refused

    1. Firewall rules might be preventing connections from the OpenSpecimen VM to the REDCap server. Contact system/network admin to fix the firewall rules.

  3. CertificateExpiredException: This occurs when the REDCap server certificate has expired.

    1. Contact the REDCap server administrator to renew the server certificate.

  4. SSLHandshakeException: PKIX path building failed. This occurs when the REDCap server is either using a self-signed certificate or using a certificate issued by CA not trusted by every client.

    1. Contact the REDCap server administrator to share the CA root certificate for importing into OpenSpecimen's trusted certificate store.

    2. The REDCap public certificate can be imported into the JVM trust store as below. This will require reboot of OpenSpecimen app.

      1. keytool -importcert -alias <server_name> -keystore $JAVA_HOME/jre/lib/security/cacerts -file <public_certificate.crt>

MysqlDataTruncation:Data truncation: Incorrect datetime

The error occurs because the MySQL timestamp columns cannot store date/time prior to 1/1/1970. Error message in the log file is like this - 'Error: OpenSpecimenException:Error: MysqlDataTruncation:Data truncation: Incorrect datetime value: '1969-04-18 00:00:00' for column 'DE_A_29' at row 1'.

Steps to fix the error

  1. Retrieve the form ID for which the records are failing. This can be obtained from the URL of the view where participant form data is displayed.

  2. Identify the table used to store the form information using the below SQL.

    select substr(xml, 1, 256) from dyextn_containers where identifier = 151;

  3. The name of the column to modify is printed in the error log file.

  4. Using #2 and #3, execute below DDL to change the column data type to ‘DateTime.

You do not have permission to use the API or Invalid token.

There could be two reasons why this error occurred. The REDCap user to whom the token is issued: 

  • Do not have the export/import API rights for the REDCap project.

  • Do not have the right to access the project data logs.

To fix the error, ensure the token user has a tick (check) mark under the "Logging" and "API" columns of the project's user rights table. The below table illustrates the basic minimum rights the token user need to possess. 

NoClassDefFoundError:Could not initialize class sun.text.normalizer

This error is raised when the JAVA_HOME is pointing to an unstable or incorrect installation. Unless the JAVA_HOME variable is fixed, the user is not navigated to the next page.

Row size too large

 

Got feedback or spotted a mistake?

Leave a comment at the end of this page or email contact@krishagni.com