Commits · d776580377aa0b601085314ed96ac3a8ca16eb50 · dbnomics-fetchers / eurostat-fetcher · GitLab

Snippets Groups Projects

Nov 30, 2017
- Read source data from Git repo · d7765803
  Christophe Benz authored 7 years ago
  
  d7765803
Nov 28, 2017
- Spare a "git ls-tree" for blob ID, relying on yields order · f6da856e
  Christophe Benz authored 7 years ago
  
  f6da856e
Nov 27, 2017
- If a pack is invalid, skip · c767ba46
  Christophe Benz authored 7 years ago
  
  c767ba46
- Don't index already indexed packs · 89bdc692
  Christophe Benz authored 7 years ago
  
  89bdc692
- Add option to exclude datasets · 7ac5c38a
  Christophe Benz authored 7 years ago
  
  7ac5c38a
- Write series JSON back to dataset JSON · 8e2863ca
  Christophe Benz authored 7 years ago
  
  8e2863ca
- Rewrite observations rows using an iterator · ecccb582
  Christophe Benz authored 7 years ago
  
  ecccb582
- Spare memory by writing series.json files again · 6086836f
  Christophe Benz authored 7 years ago
  
  6086836f
Nov 25, 2017
- Rewrite avoiding sub-functions + handle datasets in many categories · 82beb52b
  Christophe Benz authored 7 years ago
  
  82beb52b
Nov 24, 2017
- Use fast_iter to spare RAM · be2a0869
  Christophe Benz authored 7 years ago
  
  be2a0869
Nov 23, 2017
- Do not use kwargs for Python 3.5 compatibility · 1f63708e
  Christophe Benz authored 7 years ago
  
  1f63708e
- Write data model version in JSON repo, add comments · 0ad8f99a
  Christophe Benz authored 7 years ago
  
  0ad8f99a
- Access ".id" property only once · b2a25f2a
  Christophe Benz authored 7 years ago
  
  b2a25f2a
- Remove validator which does not exist anymore · 1e71ca1c
  Christophe Benz authored 7 years ago
  
  1e71ca1c
- Set Git packs compression level to 0 · ebd49cc7
  Christophe Benz authored 7 years ago
  
  ebd49cc7
- Dedupe observations TSV blobs directly to spare RAM · 80e781cf
  Christophe Benz authored 7 years ago
  
  80e781cf
- Remove unused import · ad73429a
  Christophe Benz authored 7 years ago
  
  ad73429a
- Follow data model evolutions: remove provider name, which is redundant · e7c730ba
  Christophe Benz authored 7 years ago
  
  e7c730ba
- Follow data model evolution: provider has no more slug · c4d6ca87
  Christophe Benz authored 7 years ago
  
  c4d6ca87
Nov 22, 2017
- Update dependencies · 87bb1bd0
  Christophe Benz authored 7 years ago
  
  87bb1bd0
- Remove branches in categories tree without datasets · e7fe93ab
  Christophe Benz authored 7 years ago
  
  e7fe93ab
- Keep branches in categories tree containing converted datasets · d46be73d
  Christophe Benz authored 7 years ago
  
  d46be73d
- Don't add "type" property to categories tree nodes... · 29e08305
  Christophe Benz authored 7 years ago
```
a dataset can be recognized because it has no children
```
  29e08305
- Fix index creation · 57482d2f
  Christophe Benz authored 7 years ago
  
  57482d2f
- Log index creation · 88ae9300
  Christophe Benz authored 7 years ago
  
  88ae9300
- Do not use f-strings, prod has Python < 3.6 · 5aa9b5f0
  Christophe Benz authored 7 years ago
  
  5aa9b5f0
- Reduce number of Git objects written in pack and... · 4f37193f
  Christophe Benz authored 7 years ago
```
- write categories_tree.json
- write datasets index
```
  4f37193f
- Store series in a list property of dataset.json · f5ca987b
  Christophe Benz authored 7 years ago
  
  f5ca987b
- Do not store empty keys in dataset.json · 39e6a083
  Christophe Benz authored 7 years ago
  
  39e6a083
- Do not create series sub-directories · 3a8e3cdd
  Christophe Benz authored 7 years ago
  
  3a8e3cdd
- Store dataset name, description and doc_href · 0d989033
  Christophe Benz authored 7 years ago
  
  0d989033
- Add debug info · 32f518fa
  Christophe Benz authored 7 years ago
  
  32f518fa
- If pack exists, and no source is found, add it anyway · 32397800
  Christophe Benz authored 7 years ago
  
  32397800
Nov 21, 2017
- Update to new index format · 220fb37e
  Christophe Benz authored 7 years ago
  
  220fb37e
- Index dataset.json blobs too · ab260156
  Christophe Benz authored 7 years ago
  
  ab260156
- Import categories tree · 66edc75e
  Christophe Benz authored 7 years ago
  
  66edc75e
Nov 20, 2017
- Use git CLI to generate packs info file · 34df5607
  Christophe Benz authored 7 years ago
  
  34df5607
- Add cwd to subprocess · 2b71706c
  Christophe Benz authored 7 years ago
  
  2b71706c
- Reimplement update_server_info to avoid error · cf516c22
  Christophe Benz authored 7 years ago
  
  cf516c22
- Use line based index · 842fe5ca
  Christophe Benz authored 7 years ago
  
  842fe5ca