... | ... | @@ -197,7 +197,7 @@ The fetcher will be composed of two mandatory files: |
|
|
|
|
|
* given a provider
|
|
|
* populate the **source-data** repository
|
|
|
* with the raw data of the provider
|
|
|
* with the raw data of the provider (specified datasets mentionned in the Analysis)
|
|
|
* by using the most appropriate method
|
|
|
|
|
|
|
... | ... | @@ -209,10 +209,12 @@ Datasets tha havec to be stored are listed in the corresponding ***Analysis*** y |
|
|
|
|
|
* Read the analysis that specify which dataset we want to store and how to access it
|
|
|
|
|
|
* Define the targeted datasets
|
|
|
* Define the targeted datasets and make assertion check to detect if there is change in the access to the datasets
|
|
|
|
|
|
|
|
|
* Specify the **data-source repository** for your provider into your `<provider_slug>_to_source_data.py`, this script will be executed from CLI by gitlab-CI so it should take at least one argument : the destination for the datasets i.e the specific path of source-data repository corresponding to your provider
|
|
|
|
|
|
|
|
|
* Add you file commit and push:
|
|
|
```bash
|
|
|
(nomics_env) me@mylaptop:~/dbnomics-fetchers/<provider_slug>-fetcher$ git add .
|
... | ... | @@ -220,8 +222,15 @@ Datasets tha havec to be stored are listed in the corresponding ***Analysis*** y |
|
|
(nomics_env) me@mylaptop:~/dbnomics-fetchers/<provider_slug>-fetcher$ git push
|
|
|
```
|
|
|
|
|
|
### Data-source
|
|
|
##### to_dbnomics
|
|
|
|
|
|
`<provider_slug>_to_dbnomics.py` is a script that:
|
|
|
|
|
|
* given a data_source
|
|
|
* populate the **json-data** repository
|
|
|
* with the selected and converted data as mentionned in the Analysis
|
|
|
* by using the most appropriate method and dbnomics-convertors builtins functions to help and validate
|
|
|
|
|
|
|
|
|
### |
|
|
* Open the corresponding Analysis and define the targeted time series
|
|
|
you will need to extract from raw data |