Introduction

OpenSpecimen supports bulk/batch importing of many entities such as Collection Protocols, Specimen, Visits, Users, etc. Bulk Importing is used in high throughput biobanks or when the user needs to add/update multiple entities to/from OpenSpecimen. (Follow this page for an overview on the BulkImport module)

This page documents the details of bulk import API. Follow this wiki-page for a working example of CP import via API.

Goals for Bulk Import API

Provide an easy and efficient way to perform a single operation (Insert/Update) on multiple entities.

Steps for Bulk importing

Download the CSV template file
Upload the CSV file
Create and run the import job
Check the job status
Download the job report

Download the CSV template file

Follow the below steps to download the template file:

Login to OpenSpecimen → Navigate To → Collection Protocols → More → Import Biospecimen Data (CSV) → Choose the Record Type → Click on Download Template

Upload the CSV file

Attribute	Value
URL	http://<host>:<port>/openspecimen/rest/ng/import-jobs/input-file
Method	POST
Request Type	form-data
Response	{ "fileId": "fa8e7401-0495-443b-bedc-99a43e0738f5" }

Note down the 'fileId' as this will be used in step #3

Create and run the import job

Attribute	Value
URL	http://<host>:<port>/openspecimen/rest/ng/import-jobs
Method	POST
Request Type	application/json
Request	{ "objectType" : <objectType>, "importType" : <operationType>, "inputFileId" : <fileId>, "dateFormat": <dateFormat>, "timeFormat": <timeFormat> }

Replace '<objectType>' with the schemaName for the corresponding Bulk Import entity. (Find here the schema name to entity mapping)
Replace '<operationType>' with 'CREATE' or 'UPDATE' for bulk creating or bulk updating entities respectively.
Replace '<fileId>' with the fileId retrieved from the step #2
Date and time formats are optional. When not specified the system configured formats are used. Specify dateFormat and timeFormat when your input CSV uses date and times that are incompatible with the system configuration.

Check the job status

Attribute	Value
URL	http://<host>:<port>/openspecimen/rest/ng/import-jobs/{job-id}
Method	GET
Request Type	application/json

Download the job report/output

Attribute	Value
URL	http[s]://<host>:<port>/openspecimen/rest/ng/import-jobs/{job-id}/output
Method	GET

Example of Bulk Importing CP(s) using API

Follow this wiki-page to see an example of bulk importing CP(s) using bulk import API.

Error Cases

Code	Applies to	Status Message
200	All requests	Bulk Import request was successfully processed.
401	All requests	Authorisation failed, user doesn’t have the authority.
500	All requests	Internal server error, encountered server error while performing operations.

Sample Code

The below is the sample python code to do bulk import for Specimen Events:

How to run?

python3 bulk_import.py <CSV_FILE_NAME>

Browser not supported