... | ... | @@ -186,12 +186,15 @@ Inside https://git.nomics.world/dbnomics-fetchers click on `New Project` |
|
|
(nomics_env) me@mylaptop:~/dbnomics-fetchers/$ cd <provider_slug>-fetcher
|
|
|
```
|
|
|
|
|
|
The fetcher will be composed of two mandatory files:
|
|
|
The fetcher will be composed of three mandatory files:
|
|
|
|
|
|
* <provider_slug>_to_source_data.py
|
|
|
* <provider_slug>_source_data_to_dbnomics.py
|
|
|
* requirements.txt
|
|
|
|
|
|
##### to_source_data
|
|
|
### Source data
|
|
|
|
|
|
Correspond to script to_source_data.py in your fetcher that populate the source data repoistory.
|
|
|
|
|
|
`<provider_slug>_to_source_data.py` is a script that:
|
|
|
|
... | ... | @@ -222,7 +225,9 @@ Datasets tha havec to be stored are listed in the corresponding ***Analysis*** y |
|
|
(nomics_env) me@mylaptop:~/dbnomics-fetchers/<provider_slug>-fetcher$ git push
|
|
|
```
|
|
|
|
|
|
##### to_dbnomics
|
|
|
### JSON DATA
|
|
|
Correspond to our script to_dbnomics.py in your fetcher that will populate JSON DATA and convert from source data to dbnomics format
|
|
|
|
|
|
|
|
|
`<provider_slug>_to_dbnomics.py` is a script that:
|
|
|
|
... | ... | @@ -231,6 +236,34 @@ Datasets tha havec to be stored are listed in the corresponding ***Analysis*** y |
|
|
* with the selected and converted data as mentionned in the Analysis
|
|
|
* by using the most appropriate method and dbnomics-convertors builtins functions to help and validate
|
|
|
|
|
|
|
|
|
* Open the corresponding Analysis and define the targeted time series
|
|
|
you will need to extract from raw data
|
|
|
|
|
|
#### PROVIDER
|
|
|
|
|
|
* define the provider
|
|
|
* declare the **categories** following the Analysis instructions
|
|
|
|
|
|
#### CATEGORIES
|
|
|
> (hard coding seems to be the best option for now...)
|
|
|
|
|
|
multiples levels of categories are possible in a tree mode
|
|
|
|
|
|
#### DATASETS
|
|
|
|
|
|
|
|
|
#### SERIES & OBSERVATIONS
|
|
|
|
|
|
|
|
|
|
|
|
### Requirements
|
|
|
|
|
|
Put in requirements the external packages to run the script
|
|
|
|
|
|
`requirements.txt` example:
|
|
|
|
|
|
```
|
|
|
requests
|
|
|
xlrd
|
|
|
bs4
|
|
|
``` |