CHORD¶
This project was funded by CANARIE.
What is CHORD?¶
“CHORD is a project to build a federated, Canadian, national data service for privacy- sensitive genomic and related health data. It allows the technologies and services being built by CanDIG [a GA4GH driver project sharing PIs] and its international partners to be made available to the Canadian health research community more broadly.”
Quote from the CHORD SOW; square brackets added for clarity.
In more technical terms, CHORD is a set of microservices that allow the sharing, management, and discovery of genomic and health data across a federated network. CHORD heavily benefits from and utilizes GA4GH’s efforts to create standard APIs and formats for genomic and health data transfer and storage.
Project Provenance¶
Releases are authorized by a committee composed of CHORD and shared platform software developers and project managers.
Before publication, release candidates currently go through the following validation process:
Comprehensive service- and library-level testing suites
Testing by hand using synthetic datasets on local machines
Advance deployment on select instances that opt into testing newer versions
As part of the release process, documentation is included in the form of tagged
versions of the this website, and service-level README files for
service-specific technical details.
The developers of the platform are constantly monitoring for the latest patches to dependencies used in the project. Any updates that are of critical importance (bug fixes, security flaws) will warrant a patch release of the software itself, which will pass through the standard release vetting process.
CHORD Singularity Server¶
Overview¶
To run on the GenAP platform, CHORD services can be bundled together into a single Singularity image, which provides a portable and easy-to-run format for the CHORD suite of microservices.
See the repository for additional details: https://github.com/c3g/chord_singularity
What’s Bundled?¶
The CHORD Singularity container includes several services, tools, and libraries other than what’s provided by CHORD services themselves:
NodeJS 10
Python 3.7
Java 11
A Redis instance (for message-passing and fast key-value storage) at
/chord/tmp/redis.sockA PostgreSQL 11 instance at
/chord/tmp/postgresql/.s.PGSQL.5433with service-specific credentials stored in environment variables.
htslib
chord_services.json¶
Overview¶
The chord_services.json file specifies a list of services for the single-Singularity container server, and is
akin to Docker files for each service, although in this case each service is loaded into the same container.
Justification¶
CHORD is designed to be ran as an application on the GenAP platform. GenAP applications are instances of one Singularity container, meaning that although CHORD is microservice-based, all microservices must run in the same container.
The chord_services.json file specifies configuration for each of these services to allow them to co-operate within
the same container.
Configuration Parameters¶
TODO
CHORD Search Format¶
TODO
chord_workflows.json¶
TODO
CHORD Service Types¶
Artifacts should be unique by themselves within a node, since it allows for a more readable overall API / service file system layout and prevents confusion.
Organization |
Artifact |
Repository |
|---|---|---|
ca.c3g.chord |
service-registry |
|
ca.c3g.chord |
drop-box |
|
ca.c3g.chord |
drs |
|
ca.c3g.chord |
wes |
|
ca.c3g.chord |
federation |
|
ca.c3g.chord |
notification |
|
ca.c3g.chord |
event-relay |
|
ca.c3g.chord |
metadata |
|
ca.c3g.chord |
variant |
Creating a Service¶
TODO
Bundled Services¶
TODO
Adding Data to the Drop Box¶
Currently, no upload mechanism is available for CHORD. Files should be copied
via some out-of-band mechanism to the CHORD node and placed in whichever folder
is bound to /chord/data/drop-box inside the container. These files and
folders will appear in the front end’s “Files” section.