Optimize time to availability of provider data
Description
- strategic issue: users need to access INSEE data as soon as it's published
- for now we do our best to reduce the time to availability
Tasks
-
setup a dedicated runner for INSEE jobs: download, convert, index (?), deploy (git pull) -
associate that runner to insee-fetcher in fetchers.yml
(declaratively) -
make dbnomics-fetcher-ops configuration script take that declarative setting into account -
ensure that INSEE jobs actually are executed via that runner
Questions
- could index and deploy job order be inverted?
- or could index and deploy be parallelized?