Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
D
documentation
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 0
    • Merge Requests 0
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI / CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • dbnomics-fetchers
  • documentation
  • Wiki
  • Home

Last edited by Christophe Benz Dec 22, 2020
Page history
This is an old version of this page. You can view the most recent version or browse the history.

Home

This wiki documents how to write a fetcher.

  • DBnomics tutorial notebook
  • Glossary
  • Fetcher design rules
  • Write a new fetcher
  • Write a new converter
  • Things to think about when finishing writing a converter
  • General acceptance criteria
  • JSON data model
  • Troubleshooting
  • Web API

Tooling

  • Development tools
  • Code style
  • Git and gitlab workflow
  • Code Optimization (profiling...)

Maintainance

  • Maintaining fetchers

UI

  • User scenarios
  • UI Design

Infrastructure

  • Setup CI jobs: see https://git.nomics.world/dbnomics/dbnomics-gitlab-ci/blob/master/README.md
  • CI jobs
  • Repositories organization
  • Test a fetcher on the pre-production server
  • Download big git repositories statically
  • Servers
  • Production configuration
  • Monitoring
  • E-mails

Data

  • Special datasets

Other fetcher libraries

SDMX libraries:

  • https://github.com/amattioc/SDMX
  • https://pandasdmx.readthedocs.io/
  • https://github.com/26fe/jsonstat.py
  • https://github.com/sdmx-twg/sdmx-rest

Fun

  • DBnomics pearls

Project management

See this project and wiki

Clone repository
  • Code style
  • Git and Gitlab workflow
  • acceptance criteria
    • fetchers
  • ci jobs and runners
  • code optimization
  • dev tools
  • e mails
  • failure handling procedures
  • Home
  • librairies
  • maintaining fetchers
  • monitoring
  • presentation
  • production configuration
  • publishing python packages
View All Pages