Versions Compared

Old Version 5

changes.mady.by.user Swapnil Ingle (Unlicensed)

Saved on

compared with

New Version Current

changes.mady.by.user Kaustubh Wadagavi

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Introduction

OpenSpecimen supports bulk/batch importing of many entities such as Collection Protocols, Specimen, Visits, Users, etc. Bulk Importing is used in high throughput biobanks or when the user needs to add/update multiple entities to/from OpenSpecimen. (Follow this page for an overview on the BulkImport module)

This page documents the details of bulk import API. Follow this wiki-page for a working example of CP import via API.

Goals for Bulk Import API

  1. Provide an easy and efficient way to perform a single operation (Insert/Update) on multiple entities.

Steps for Bulk importing

  1. Download the input CSV template file.
  2. Register your input file.
  3. Deploy the input file
  4. Upload the CSV file
  5. Create and run the import - job.
  6. Check the job status of
  7. Download the import- job .report

Download the

...

CSV template file

...

application/json

Note

Replace <schemaName> with the schema name for the entity you wish to bulk import. (Find here the schema name to entity mapping)

...

Follow the below steps to download the template file:

Login to OpenSpecimen → Navigate To → Collection Protocols → More → Import Biospecimen Data (CSV) → Choose the Record Type → Click on Download Template

Image Added 

Upload the CSV file

AttributeValue
URLhttp://<host>:<port>/openspecimen/rest/ng/import-jobs/input-file
MethodPOST
Request Type

form-data

Response


Code Block
languagejs
themeEclipse
linenumberstrue
{
    "fileId": "fa8e7401-0495-443b-bedc-99a43e0738f5"
}


...

Note

Note down the 'fileId' as this will be used in step #3

...

Create and run the import

...

job

AttributeValue
URLhttp://<host>:<port>/openspecimen/rest/ng/import-jobs
MethodPOST
Request Type

application/json

Request


Code Block
languagejs
themeEclipse
linenumberstrue
{
  "objectType" : "<objectType>",
  "importType" : "<operationType>",
  "inputFileId" : "<fileId>"<fileId>,
  "dateFormat": <dateFormat>,
  "timeFormat": <timeFormat>
}



Note
  1. Replace '<objectType>' with the schemaName for the corresponding Bulk Import entity. (Find here the schema name to entity mapping)
  2. Replace '<operationType>' with 'CREATE' or 'UPDATE' for bulk creating or bulk updating entities respectively.
  3. Replace '<fileId>' with the fileId retrieved from the step #2

...

  2. Date and time formats are optional. When not specified the system configured formats are used. Specify dateFormat and timeFormat when your input CSV uses date and times that are incompatible with the system configuration.

Check the job status

AttributeValue
URLhttp://<host>:<port>/openspecimen/rest/ng/import-jobs/{job-id}
MethodGET
Request Type

application/json

Download the job report/output

AttributeValue
URLhttp[s]://<host>:<port>/openspecimen/rest/ng/import-jobs/{job-id}/output
MethodGET

Example of Bulk Importing CP(s) using API

Follow this wiki-page to see an example of bulk importing CP(s) using bulk import API.

Error Cases

CodeApplies toStatus Message
200All requestsBulk Import request was successfully processed.
401All requestsAuthorisation failed, user doesn’t have the authority.
500All requestsInternal server error, encountered server error while performing operations.

Sample Code

The below is the sample python code to do bulk import for Specimen Events:

View file
namebulk_import.py
height250

How to run?

Info

python3 bulk_import.py <CSV_FILE_NAME>