Defining Data Validation

Data validations can be defined using JSON. The ‘editChecks’ section of the code must be inserted after the ‘dictionary’ section in the workflow JSON.

The code has three parts - records, forms and rules
1. records - Whenever any of these specified records are modified; the constraint rules should be evaluated/enforced. Refers to the fields at the data entry level for which the validation is to be applied. The value for records can be cpr, specimen, visit or a combination of these.
2. forms - Optional field. If present, specifies the list of forms whose data is used to enforce the constraints.
3. rules - List of integrity rules that should be satisfied by the records and forms.
A ‘rule’ is made up of three attributes when, expr, and description.
1. when: The attribute 'when' is optional and specifies when the rule is applicable. For example, the rule is applicable only for primary specimens
2. expr: The attribute 'expr' specifies the check constraint that should be satisfied.
3. description: The attribute 'description' describes the constraint in user-friendly language. This is also used in error messages when the rule is broken or not satisfied.
Aliases for rules are - “cpr, visit, primarySpecimen, specimen”, which are self-explanator.
Aliases for accessing the form records are cprForms, visitForms, primarySpecimenForms, specimenForms. These are maps that can be indexed by the form name to access the relevant form data. For example, specimenForms['SpecimenFrozenEvent'] can be used to access the latest frozen event. Similarly, specimenForms['SpecimenFrozenEvent$Array'] can be used to access the list of all the frozen events for the specimen in question.

Example: Visit date should be same or later than the registration date

For more examples, please refer to the wiki page Data Validation Examples

{
  "name" : "editChecks",
  "data" : {
    "constraints" : [ 
      {
        "records" : [ "cpr", "visit" ],
        "rules" : [
          {
            "expr" : "#cpr.registrationDate != null && #visit.visitDate != null && !#cpr.registrationDate.after(#visit.visitDate)",
            "description" : "Visit date should be same or later than the registration date!"
          } 
        ]
      }
    ]
  }
}

Data Validation Fields

We can set up data validation on these 3 types of fields and their combinations:

Core fields
Custom fields
Custom form fields
Combination of the core fields, custom fields, custom form fields

Level	Field Name	Field Expression

Level	Field Name	Field Expression
Participant	PPID	`#cpr.ppid`
Participant	First Name	`#cpr.participant.firstName`
Participant	Middle Name	`#cpr.participant.middleName`
Participant	Last Name	`#cpr.participant.lastName`
Participant	Registration Date	`#cpr.registrationDate`
Participant	External Subject ID	`#cpr.externalSubjectId`
Participant	Registration Site	`#cpr.site`
Participant	Birth Date	`#cpr.participant.birthDate`
Participant	Social Security Number	`#cpr.participant.uid`
Participant	eMPI	`#cpr.participant.empi`
Participant	Gender	`#cpr.participant.gender`
Participant	Vital Status	`#cpr.participant.vitalStatus`
Participant	Death Date	`#cpr.participant.deathDate`
Participant	Races	`#cpr.participant.races`
Participant	Ethnicity	`#cpr.participant.ethnicities`
Participant	MRN	`#cpr.participant.pmis`
Visit	Name	`#cpr.participant.pmis`
Visit	Name	`#visit.name`
Visit	Status	`#visit.status`
Visit	Missed By	`#visit.missedBy`
Visit	Missed Reason	`#visit.missedReason`
Visit	Visit Date	`#visit.visitDate`
Visit	Visit Site	`#visit.site`
Visit	Clinical Diagnosis	`#visit.clinicalDiagnoses`
Visit	Clinical Status	`#visit.surgicalPathologyNumber`
Visit	SPR	`#visit.clinicalStatus`
Visit	Visit Comments	`#visit.comments`
Specimen	Label	`#specimen.label`
Specimen	Barcode	`#specimen.barcode`
Specimen	Lineage	`#specimen.lineage`
Specimen	Collection Status	`#specimen.status`
Specimen	Type	`#specimen.type`
Specimen	Anatomic Site	`#specimen.anatomicSite`
Specimen	Laterality	`#specimen.laterality`
Specimen	Pathology Status	`#specimen.pathology`
Specimen	Initial Qty	`#specimen.initialQty`
Specimen	Available Qty	`#specimen.availableQty`
Specimen	Concentration	`#specimen.concentration`
Specimen	Parent Specimen Label	`#specimen.parentLabel`
Specimen	Biohazard	`#specimen.biohazards`
Specimen	Location	`#specimen.storageLocation`
Specimen	Created On	`#specimen.createdOn`
Specimen	Freezer Thaw Cycle	`#specimen.freezeThawCycles`
Specimen	Increment Freeze Thaw Cycle	`#specimen.incrParentFreezeThaw`
Specimen	Comments	`#specimen.comments`
Specimen	Collection Date	`#specimen.collectionEvent.time`
Specimen	Collector	`#specimen.collectionEvent.user`
Specimen	Received Date	`#specimen.receivedEvent.time`
Specimen	Receiver	`#specimen.receivedEvent.user`
Specimen	Collection Container	`#specimen.collectionEvent.container`
Specimen	Collection Procedure	`#specimen.collectionEvent.procedure`
Specimen	Received Quality	`#specimen.receivedEvent.receivedQuality`
Specimen	Frozen Time	`#specimen.events.SpecimenFrozenEvent.time`
Specimen	Frozen Method	`#specimen.events.SpecimenFrozenEvent.frozenMethod`
Specimen	Frozen Comments	`#specimen.events.SpecimenFrozenEvent.comments`

Adding Multiple Data Validations

In case there are many data validations to be performed, here is a tip that could help in avoiding errors:

Check and test the data entry workflow after adding each validation. This will allow you to identify if any validation fails and will be useful in rectifying the failed one. If all the validation are added together, it will be difficult to identify the validation in case the above error ‘Cannot index into a null value’ occurs.

OpenSpecimen

Defining Data Validation

Data Validation Fields

Adding Multiple Data Validations

Related content