|
|
# DB.nomics Technical Committee meeting
|
|
|
# DBnomics Technical Committee meeting
|
|
|
[November 21, 2017 17:30-18:30](https://git.nomics.world/dbnomics-fetchers/management/wikis/Timeline#-2017-11-21-1730-technical-committee-meeting)
|
|
|
|
|
|
## Attendees
|
|
|
|
|
|
Christophe Benz, DB.nomics
|
|
|
Thomas Brand, Cepremap
|
|
|
Michel Juillard, Banque de France
|
|
|
Julien Lasselot, Banque de France
|
|
|
Johan Richer, DB.nomics
|
|
|
Christophe Benz, DBnomics
|
|
|
Thomas Brand, Cepremap
|
|
|
Michel Juillard, Banque de France
|
|
|
Julien Lasselot, Banque de France
|
|
|
Johan Richer, DBnomics
|
|
|
|
|
|
## Agenda
|
|
|
|
... | ... | @@ -19,16 +19,16 @@ Johan Richer, DB.nomics |
|
|
- Planning for end of [Release 1](https://git.nomics.world/dbnomics-fetchers/management/milestones/1)
|
|
|
- Planning for [Release 2](https://git.nomics.world/dbnomics-fetchers/management/milestones/2) and [Release 3](https://git.nomics.world/dbnomics-fetchers/management/milestones/3) (distribution of developer/days)
|
|
|
- Testing of finished fetchers
|
|
|
- #127: Merge all DB.nomics projects into one "dbnomics" group
|
|
|
Proposition: place all current [DB.nomics groups](https://git.nomics.world/dbnomics-fetchers/documentation/wikis/repositories-organization) (/dbnomics-fetchers, /dbnomics-api, etc.) under one unique group [git.nomics.world/dbnomics](https://git.nomics.world/dbnomics)). They will become subgroups which can have their own set of subgroups and [projects](https://docs.gitlab.com/ee/user/project/index.html).
|
|
|
- Git sparse checkout
|
|
|
- GitLab Wiki notifications
|
|
|
- #127: Merge all DBnomics projects into one "dbnomics" group
|
|
|
Proposition: place all current [DBnomics groups](https://git.nomics.world/dbnomics-fetchers/documentation/wikis/repositories-organization) (/dbnomics-fetchers, /dbnomics-api, etc.) under one unique group [git.nomics.world/dbnomics](https://git.nomics.world/dbnomics)). They will become subgroups which can have their own set of subgroups and [projects](https://docs.gitlab.com/ee/user/project/index.html).
|
|
|
- Git sparse checkout
|
|
|
- GitLab Wiki notifications
|
|
|
|
|
|
## Update
|
|
|
|
|
|
The efficiency of the DB.nomics team improved during this iteration. Part of the development process becomes increasingly systematic.
|
|
|
[BLS](https://next.nomics.world/bls) and [ILO](https://next.nomics.world/ilo) were delivered in time. [Fetcher for Eurostat](https://git.nomics.world/dbnomics-fetchers/eurostat-fetcher) is done but data has yet to be indexed and integrated into the UI.
|
|
|
More work has to be done on the infrastructure to adapt it to bigger datasets and files (cf. Eurostat). #133
|
|
|
The efficiency of the DBnomics team improved during this iteration. Part of the development process becomes increasingly systematic.
|
|
|
[BLS](https://next.nomics.world/bls) and [ILO](https://next.nomics.world/ilo) were delivered in time. [Fetcher for Eurostat](https://git.nomics.world/dbnomics-fetchers/eurostat-fetcher) is done but data has yet to be indexed and integrated into the UI.
|
|
|
More work has to be done on the infrastructure to adapt it to bigger datasets and files (cf. Eurostat). #133
|
|
|
|
|
|
## Action points
|
|
|
- Set the iteration dates, with regard to holidays and the deadline for Release 1 ([Timeline](https://git.nomics.world/dbnomics-fetchers/management/wikis/Timeline))
|
... | ... | @@ -36,7 +36,7 @@ More work has to be done on the infrastructure to adapt it to bigger datasets an |
|
|
- At least 1 full iteration in January
|
|
|
- Testing: Michel started to work on it, a process is beginning to emerge, will be discussed at the [next meeting](https://git.nomics.world/dbnomics-fetchers/management/wikis/steering-committee/17-12-05).
|
|
|
- Use a prod server
|
|
|
- GitLab DB.nomics group: test the feature before making the decision
|
|
|
- GitLab DBnomics group: test the feature before making the decision
|
|
|
- Git sparse checkout: @cbenz ?
|
|
|
- GitLab Notifications: use [custom notifications](https://git.nomics.world/profile/notifications) for each GitLab group. [It is not possible to "watch" wiki changes natively in GitLab](https://groups.google.com/forum/#!msg/gitlabhq/v887P52tMXI/MqaCWztc7sAJ), though [a solution exists](https://github.com/datamachines/gitwatch) (GitLab wikis are stored in a separate repo).
|
|
|
- Use the provider's API to download data whenever it is simpler than using the bulk download. It may be slower for the initial acquisition of data, but more efficient for future updates if the API provides a way to obtain only modified data. This applies to IMF and World Bank that are considered for the next iteration. |
|
|
\ No newline at end of file |