DBnomics Technical Committee meeting
Christophe Benz, DBnomics Thomas Brand, Cepremap Michel Juillard, Banque de France Julien Lasselot, Banque de France Johan Richer, DBnomics
- Brief review of iteration 08/11-21/11
- Resistance of infrastructure to integrating Eurostat (#11 (closed) #90 (closed) #91 (closed) #92 (closed))
- IMF (#8 (closed) #131 (closed))
- World Bank (#12 (closed) #130 (closed))
- Rosstat (#7 (closed))
- Planning for end of Release 1
- Planning for Release 2 and Release 3 (distribution of developer/days)
- Testing of finished fetchers
- #127 (closed): Merge all DBnomics projects into one "dbnomics" group Proposition: place all current DBnomics groups (/dbnomics-fetchers, /dbnomics-api, etc.) under one unique group git.nomics.world/dbnomics). They will become subgroups which can have their own set of subgroups and projects.
- Git sparse checkout
- GitLab Wiki notifications
The efficiency of the DBnomics team improved during this iteration. Part of the development process becomes increasingly systematic. BLS and ILO were delivered in time. Fetcher for Eurostat is done but data has yet to be indexed and integrated into the UI. More work has to be done on the infrastructure to adapt it to bigger datasets and files (cf. Eurostat). #133 (closed)
- Set the iteration dates, with regard to holidays and the deadline for Release 1 (Timeline)
- Testing: Michel started to work on it, a process is beginning to emerge, will be discussed at the next meeting.
- Use a prod server
- GitLab DBnomics group: test the feature before making the decision
- Git sparse checkout: @cbenz ?
- GitLab Notifications: use custom notifications for each GitLab group. It is not possible to "watch" wiki changes natively in GitLab, though a solution exists (GitLab wikis are stored in a separate repo).
- Use the provider's API to download data whenever it is simpler than using the bulk download. It may be slower for the initial acquisition of data, but more efficient for future updates if the API provides a way to obtain only modified data. This applies to IMF and World Bank that are considered for the next iteration.