... | ... | @@ -210,23 +210,17 @@ Datasets tha havec to be stored are listed in the corresponding ***Analysis*** y |
|
|
|
|
|
* Create the file '<provider_slug>_to_source_data.py' inside dbnomics-fetchers/<provider_slug>-fetcher/
|
|
|
|
|
|
Some useful tips:
|
|
|
|
|
|
* Read the analysis that specify which dataset we want to store and how to access it
|
|
|
|
|
|
* Define the targeted datasets and make assertion check to detect if there is change in the access to the datasets
|
|
|
|
|
|
|
|
|
* Specify the **data-source repository** for your provider into your `<provider_slug>_to_source_data.py`, this script will be executed from CLI by gitlab-CI so it should take at least one argument : the destination for the datasets i.e the specific path of source-data repository corresponding to your provider
|
|
|
|
|
|
|
|
|
* Add you file commit and push:
|
|
|
```bash
|
|
|
(nomics_env) me@mylaptop:~/dbnomics-fetchers/<provider_slug>-fetcher$ git add .
|
|
|
(nomics_env) me@mylaptop:~/dbnomics-fetchers/<provider_slug>-fetcher$ git commit -m "ADD Fetcher: <provider_slug>_to_source_data.py "
|
|
|
(nomics_env) me@mylaptop:~/dbnomics-fetchers/<provider_slug>-fetcher$ git push
|
|
|
```
|
|
|
|
|
|
### JSON DATA
|
|
|
Correspond to our script to_dbnomics.py in your fetcher that will populate JSON DATA and convert from source data to dbnomics format
|
|
|
|
|
|
Correspond to your script to_dbnomics.py in your fetcher that will populate JSON DATA and convert from source data to dbnomics format
|
|
|
|
|
|
|
|
|
`<provider_slug>_to_dbnomics.py` is a script that:
|
... | ... | @@ -236,24 +230,9 @@ Correspond to our script to_dbnomics.py in your fetcher that will populate JSON |
|
|
* with the selected and converted data as mentionned in the Analysis
|
|
|
* by using the most appropriate method and dbnomics-convertors builtins functions to help and validate
|
|
|
|
|
|
* Open the corresponding Analysis and define the targeted time series
|
|
|
you will need to extract from raw data
|
|
|
|
|
|
#### PROVIDER
|
|
|
|
|
|
* define the provider
|
|
|
* declare the **categories** following the Analysis instructions
|
|
|
|
|
|
#### CATEGORIES
|
|
|
> (hard coding seems to be the best option for now...)
|
|
|
|
|
|
multiples levels of categories are possible in a tree mode
|
|
|
|
|
|
#### DATASETS
|
|
|
|
|
|
|
|
|
#### SERIES & OBSERVATIONS
|
|
|
|
|
|
Useful tips:
|
|
|
* Open the corresponding Analysis that define the structure and the targeted time series
|
|
|
you will need to extract from raw data stored in source_data
|
|
|
|
|
|
|
|
|
### Requirements
|
... | ... | |