management issueshttps://git.nomics.world/dbnomics-fetchers/management/-/issues2021-05-12T21:13:35Zhttps://git.nomics.world/dbnomics-fetchers/management/-/issues/971Error fetching series from R2021-05-12T21:13:35ZChristophe Benzchristophe.benz@nomics.worldError fetching series from R## User message
> Pour votre information, j’ai rencontré un problème de téléchargement avec la série suivante de l’Acoss.
> Est-ce que vous répliquez l’erreur de votre côté ?
https://db.nomics.world/ACOSS/ECONOMIC_BAROMETER/EMPLOI_IND...## User message
> Pour votre information, j’ai rencontré un problème de téléchargement avec la série suivante de l’Acoss.
> Est-ce que vous répliquez l’erreur de votre côté ?
https://db.nomics.world/ACOSS/ECONOMIC_BAROMETER/EMPLOI_IND_MENS.nb_employees_industry_companies_10+_employees.SA.M
```R
series <- "ACOSS/ECONOMIC_BAROMETER/EMPLOI_IND_MENS.nb_employees_industry_companies_10+_employees.SA.M"
df <- rdb(ids = series)
df
NULL
```https://git.nomics.world/dbnomics-fetchers/management/-/issues/690Refactor the Python client2022-06-28T12:44:03ZThomas BrandRefactor the Python client## Description
As a user of the python plugin, I want to use the same function as available in the R plugin.
For instance, the functions `rdb_datasets`, `rdb_dimensions` and `rdb_series` are very useful.
## Tasks
- [ ] list the diffe...## Description
As a user of the python plugin, I want to use the same function as available in the R plugin.
For instance, the functions `rdb_datasets`, `rdb_dimensions` and `rdb_series` are very useful.
## Tasks
- [ ] list the differences between R and Python client APIs
- [ ] make a proposition of evolution for Python client API, to be discussed and validated with the team
- [ ] implement it
- [ ] create a documentation either in `API.md` or in a sphinx doc, more idiomatic than a notebook
## Resources
- https://cran.r-project.org/web/packages/rdbnomics/news/news.html
- use case: https://cran.r-project.org/web/packages/rdbnomics/vignettes/rdbnomics.html#8_fetch_the_available_datasets_of_a_providerChristophe Benzchristophe.benz@nomics.worldChristophe Benzchristophe.benz@nomics.worldhttps://git.nomics.world/dbnomics-fetchers/management/-/issues/661Python client - tests fail2022-04-05T09:34:31ZBruno DuyéPython client - tests failRelated to #660
https://git.nomics.world/dbnomics/dbnomics-python-client/-/jobs/177300
-> 2 failures:
- test_fetch_series_by_ids_in_different_datasets
- test_fetch_series_by_api_linkRelated to #660
https://git.nomics.world/dbnomics/dbnomics-python-client/-/jobs/177300
-> 2 failures:
- test_fetch_series_by_ids_in_different_datasets
- test_fetch_series_by_api_linkChristophe Benzchristophe.benz@nomics.worldChristophe Benzchristophe.benz@nomics.worldhttps://git.nomics.world/dbnomics-fetchers/management/-/issues/546Consume data locally2019-10-28T18:49:00ZChristophe Benzchristophe.benz@nomics.worldConsume data locallyEPIC: #519
- As a data consumer
- I want to consume data locally
- in order to save bandwidth (and save server resources).
## Acceptance criteria
- [ ] a client MUST be able to build a data-frame from a local dataset (downloaded with...EPIC: #519
- As a data consumer
- I want to consume data locally
- in order to save bandwidth (and save server resources).
## Acceptance criteria
- [ ] a client MUST be able to build a data-frame from a local dataset (downloaded without using the web API)
## Description
Problems:
- Using web API to download a big amount of series uses too much server resources
- We must find a way to let the clients replicate data locally and work on it without relying on DBnomics infrastructure
Goals:
- download a single dataset locally
Ideas:
- load DataFrame from a Git bare repo
```python
# `./ameco-json-data.git` has been cloned from DBnomics GitLab instance in bare mode
ameco = DBnomics(provider_dir="./ameco-json-data.git")
ameco.to_df(dataset="ZUTN")
ameco.to_df(dataset="ZUTN", series="AUS.1.0.0.0.ZUTN")
# access revisions
ameco.to_df(dataset="ZUTN", series="AUS.1.0.0.0.ZUTN", revisions=True)
ameco.to_df(dataset="ZUTN", series="AUS.1.0.0.0.ZUTN", revision="{SHA1}")
```
- instrument `git clone` step also
```python
# `./data` is a black box directory
client = DBnomics(data_dir="./data")
ameco = client.fetch(provider="AMECO")
ameco.to_df(dataset="ZUTN", series="AUS.1.0.0.0.ZUTN")
```
Questions:
- if the server is restarted while a client is downloading a dataset from the API, using pagination, will the client miss some data pages, or will the request fail?
## Tasks
- [ ] ...