Skip to content
Closed
Milestone expired on Jan 31, 2018

Fetchers Batch #1

Total in production: 22

  1. BLS (US) #5 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. LBMA #141 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. WTO #2 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. Pôle emploi #6 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. DARES #3 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. DREES #4 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. ROSSTAT – Federal State Statistics, Russie #7 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. IMF – datasets CPIS, MONA, RFCL, Global Housing Watch #8 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. ONS (UK) #9 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. Bank of England #10 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. Eurostat – autres datasets #11 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. World Bank – datasets QEDS, Joint External Debt Hub, Public Sector Debt Statistics #12 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. BCEAO #13 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. ILO #14 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board

Fetchers ported from MongoDB instance

Sorted by priority.

  1. AMECO #155 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. FED #157 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. ECB #158 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. OECD #142 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. BIS #159 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. ESRI #161 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. INSEE #156 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board
  1. BEA #160 (closed)
  • data model 0.7
  • CI setup
  • visible on UI
  • in review column of the issues board

Put into production

  • New infrastructure

    • GitLab (source data and converted data stored in Git repositories, instead of MongoDB)
    • GitLab-CI (automation of download, conversion and indexation jobs, instead of Cron jobs)
    • Solr (new indexation engine, allowing full-text search, instead of MongoDB queries)

  • New API

    • Swagger-UI

  • New UI

    • Just using DB.nomics colors in Bootstrap-CSS theme (no ad hoc design)

  • Plugins

    • Python
    • R

Other tasks

  • Blog post

    • Announce the deprecating of old.nomics.world at Milestone 2
      • No big deal, everything is supported with new UI (except INSEE)
    • Explain techno choices and what they allow
    • Detail architecture, main components
    • Before / After
    • Showcase documentation
    • Present next steps
      • Oh no! more fetchers...
      • P2P sharing (DAT): maybe? to be experimented!

  • Write documentation

    • API reference (Swagger-UI)
    • Guides (Readthedocs)
      • How to download datasets?
      • How do use DB.nomics from Python, R, etc.
    • FAQ?

Questions

  • Should we even keep the old website (on old.nomics.world)?