GitLab

This wiki documents how to write a fetcher.

• DBnomics tutorial notebook
• Glossary
• Fetcher design rules
• Write a new fetcher
• Write a new converter
• Things to think about when finishing writing a converter
• General acceptance criteria
• JSON data model
• Web API

Tooling

• Development tools
• Code style
• Git and gitlab workflow
• Code Optimization (profiling...)

Maintainance & Operations

• Maintaining fetchers
• Publishing Python packages
• Solr indexation
• Failure handling procedures
• Troubleshooting (what to do if a problem occurs)

Infrastructure

• Servers and services (architecture)
• Setup CI jobs: see https://git.nomics.world/dbnomics/dbnomics-gitlab-ci/blob/master/README.md
• CI jobs and runners
• [download-convert runners migration(runners-migration)
• Repositories organization
• Test a fetcher on the pre-production server
• Test the API or the UI on the development server
• Download big git repositories statically
• Production configuration
• Monitoring
• E-mails

Data

• Special datasets

Other fetcher libraries

SDMX libraries:

• https://github.com/amattioc/SDMX
• https://pandasdmx.readthedocs.io/
• https://github.com/26fe/jsonstat.py
• https://github.com/sdmx-twg/sdmx-rest

Fun

• DBnomics pearls

Project management

See this project and wiki