Simplify JSON data model

Related to #93 (closed), #122 (closed)

  • As a fetcher developer
  • I want to manipulate a data model as simple as possible
  • in order to gain time and have a quicker learning curve.

Acceptance criteria

  • The majority of developers MUST agree on the following questions.

Resources

Given this dataset data model:

{
  "code": string,
  "name": string,
  "dimensions_values_labels": dict of dict,
  "dimensions_labels": dict
  "dimension_codes": list of strings,
  "attributes_values_labels": dict of dict,
  "attributes_labels": dict
  "attributes_codes": list of strings,
  "series": list of strings
}

Remaining questions:

  1. add timestamp for first download
  2. add timestamp for last update by provider
  3. add timestamp for last update by DB.nomics
  4. add timestamp for last visit by DB.nomics
  5. do we need to store all texts found in XLS files?

Solved questions:

  1. add number of series
  2. add doc_href for information about the dataset on the provider web site
  3. add notes for unstructured information about the dataset to display under Infos in the UI
  4. do we need dimensions_codes and attributes codes? There are just keys of dimensions_labels and attributes_labels
  5. do we need list of series? Some datasets have tens of thousand of series. Is a query to indexer not better?

Technical tasks

  • Update JSON data model wiki page
Edited Nov 07, 2017 by Christophe Benz
Assignee Loading
Time tracking Loading