|
|
# DB.nomics Technical Committee meeting
|
|
|
# DBnomics Technical Committee meeting
|
|
|
September 29, 2017 16:00-17:00
|
|
|
|
|
|
## Attendees
|
|
|
|
|
|
Christophe Benz, DB.nomics
|
|
|
Thomas Brand, Cepremap
|
|
|
Michel Juillard, Banque de France
|
|
|
Julien Lasselot, Banque de France
|
|
|
Constance de Quatrebarbes, DB.nomics
|
|
|
Johan Richer, DB.nomics
|
|
|
Christophe Benz, DBnomics
|
|
|
Thomas Brand, Cepremap
|
|
|
Michel Juillard, Banque de France
|
|
|
Julien Lasselot, Banque de France
|
|
|
Constance de Quatrebarbes, DBnomics
|
|
|
Johan Richer, DBnomics
|
|
|
|
|
|
[Meeting preparations](https://git.nomics.world/dbnomics-fetchers/management/issues/22)
|
|
|
[Meeting preparations](https://git.nomics.world/dbnomics-fetchers/management/issues/22)
|
|
|
|
|
|
## Outstanding issues
|
|
|
Decisions or propositions of solutions
|
|
|
Decisions or propositions of solutions
|
|
|
|
|
|
### Can we factorize code for Excel file parsing?
|
|
|
The developer has the last word on this issue. Not a matter treated during Analysis.
|
|
|
|
|
|
### ONS
|
|
|
To be discussed during the next Technical Committee
|
|
|
To be discussed during the next Technical Committee
|
|
|
|
|
|
### Destatis
|
|
|
API : 50€ per year to get access to tables ; 500€ to get access to linear files. Metadata not guaranteed in English.
|
|
|
Questions: do we want to spend this kind of money and in the end have a segment of the database in German?
|
|
|
Decisions:
|
|
|
- Look into the feasability of using just the website (scraping)
|
|
|
- Contact Destatis to know exactly what we get access to by paying 500€ per year.
|
|
|
API : 50€ per year to get access to tables ; 500€ to get access to linear files. Metadata not guaranteed in English.
|
|
|
Questions: do we want to spend this kind of money and in the end have a segment of the database in German?
|
|
|
Decisions:
|
|
|
- Look into the feasability of using just the website (scraping)
|
|
|
- Contact Destatis to know exactly what we get access to by paying 500€ per year.
|
|
|
|
|
|
### What to do with missing and unknown values?
|
|
|
Problem: How do the API know which value should be interpreted as missing or unknown?
|
|
|
Propositions:
|
|
|
Problem: How do the API know which value should be interpreted as missing or unknown?
|
|
|
Propositions:
|
|
|
- Store in the metadata of a series the values that should be interpreted as 'missing' or 'unknown (e.g. NaN, N/A, Null, -1, 9999, etc.)
|
|
|
- Keep as is (period and symbol of value)
|
|
|
- Convert to a standard value for unknown (e.g. NaN)
|
... | ... | @@ -40,8 +40,8 @@ Propositions: |
|
|
### Should we store web pages for categories?
|
|
|
To be decided fetcher by fetcher. The developer has the last word on this issue. Mettre en dur les informations ou les extraire du source (HTML ou fichier).
|
|
|
|
|
|
### Should we use a numbering for `categories_code` like AMECO or let each fetcher choose?
|
|
|
Take number given by provider if existing, or make up one or use label slug.
|
|
|
### Should we use a numbering for `categories_code` like AMECO or let each fetcher choose?
|
|
|
Take number given by provider if existing, or make up one or use label slug.
|
|
|
|
|
|
### Do users read the JSON repositories ?
|
|
|
|
... | ... | |