This wiki documents how to write a fetcher.
- DBnomics tutorial notebook
- Glossary
- Fetcher design rules
- Write a new fetcher
- Write a new converter
- Things to think about when finishing writing a converter
- General acceptance criteria
- JSON data model
- Web API
Tooling
- Development tools
- Code style
- Git and gitlab workflow
- Code Optimization (profiling...)
Maintainance & Operations
- Maintaining fetchers
- Publishing Python packages
- Solr indexation
- Failure handling procedures
- Troubleshooting (what to do if a problem occurs)
Infrastructure
- Servers and services (architecture)
- Setup CI jobs: see https://git.nomics.world/dbnomics/dbnomics-gitlab-ci/blob/master/README.md
- CI jobs and runners
- download-convert runners migration
- Repositories organization
- Test a fetcher on the pre-production server
- Test the API or the UI on the development server
- Download big git repositories statically
- Production configuration
- Monitoring
- E-mails
Data
Other fetcher libraries
SDMX libraries:
- https://github.com/amattioc/SDMX
- https://pandasdmx.readthedocs.io/
- https://github.com/26fe/jsonstat.py
- https://github.com/sdmx-twg/sdmx-rest
Fun
Project management
See this project and wiki