dbnomics-api issueshttps://git.nomics.world/dbnomics/dbnomics-api/-/issues2023-06-30T14:53:25Zhttps://git.nomics.world/dbnomics/dbnomics-api/-/issues/5Accept series code masks in "/series?series_ids" endpoint2023-06-30T14:53:25ZChristophe Benzchristophe.benz@nomics.worldAccept series code masks in "/series?series_ids" endpoint## Context
In https://git.nomics.world/dbnomics-fetchers/ine-spain-fetcher/-/commit/b53bf463082a71b84f3de822444548929fffd280 we changed the series codes from generated ones based on dimension codes (e.g.72.b101.311118.n.s1.00.euros), to...## Context
In https://git.nomics.world/dbnomics-fetchers/ine-spain-fetcher/-/commit/b53bf463082a71b84f3de822444548929fffd280 we changed the series codes from generated ones based on dimension codes (e.g.72.b101.311118.n.s1.00.euros), to INE-SPAIN provided ones (e.g. CTNFSI6514).
This is because we spotted a bug: some generated dimension codes corresponded to many series, which is not acceptable since a series code must be unique.
Using INE-SPAIN provided series codes solved that issue.
However using the generated series codes should still work. Indeed, the web page at URL https://db.nomics.world/INE-SPAIN/30906/72.b101.311118.n.s1.00.euros opens correctly the series CTNFSI6514, and the URL https://db.nomics.world/INE-SPAIN/30906/CTNFSI6514 works too.
In the same vein, the API URL https://api.db.nomics.world/v22/series/INE-SPAIN/30906/72.b101.311118.n.s1.00.euros is equivalent to the URL https://api.db.nomics.world/v22/series/INE-SPAIN/30906/CTNFSI6514
Bug: the API URL "/series?series_ids=" does not accept both forms:
https://api.db.nomics.world/v22/series?series_ids=INE-SPAIN%2F30906%2F72.b101.311118.n.s1.00.euros does not return anything, but https://api.db.nomics.world/v22/series?series_ids=INE-SPAIN%2F30906%2FCTNFSI6514 does.https://git.nomics.world/dbnomics/dbnomics-api/-/issues/4Solr raises an error when a dimension value code is a reserved keyword2022-10-07T16:34:31ZChristophe Benzchristophe.benz@nomics.worldSolr raises an error when a dimension value code is a reserved keywordSometimes dimension value codes match Solr reserved words (e.g. `AND`, `NOT`, etc).
Example: the dataset `IMF/WEO:2022-04` has `country=AND` (Andorra).
The corresponding [API link](https://api.db.nomics.world/v22/series/IMF/WEO:2022-04...Sometimes dimension value codes match Solr reserved words (e.g. `AND`, `NOT`, etc).
Example: the dataset `IMF/WEO:2022-04` has `country=AND` (Andorra).
The corresponding [API link](https://api.db.nomics.world/v22/series/IMF/WEO:2022-04?dimensions=%7B%22weo-country%22%3A%5B%22AND%22%5D%7D&facets=1&limit=10&observations=1&offset=0&q=) responds a server error, and the logger error (Solr response to the API) is:
```
{
error: {
code: 400,
metadata: [
'error-class',
'org.apache.solr.common.SolrException',
'root-error-class',
'org.apache.solr.parser.ParseException'
],
msg: 'org.apache.solr.search.SyntaxError: Cannot parse \'dimension_value_code.weo-country:AND\': Encountered " <AND> "AND "" at line 1, column 33.\nWas expecting one of:\n <BAREOPER> ...\n "(" ...\n "*" ...\n <QUOTED> ...\n <TERM> ...\n <PREFIXTERM> ...\n <WILDTERM> ...\n <REGEXPTERM> ...\n "[" ...\n "{" ...\n <LPARAMS> ...\n "filter(" ...\n <NUMBER> ...\n '
},
responseHeader: {
params: {
json: '{"query": "provider_code:IMF AND dataset_code:WEO\\\\:2022\\\\-04", "filter": ["type:series", "{!tag=DIMENSION.weo-country}dimension_value_code.weo-country:AND"], "facet": {"dimension.weo-country": {"type": "terms", "field": "dimension_value_code.weo-country", "limit": -1, "mincount": 1, "domain": {"excludeTags": "DIMENSION.weo-country"}}, "dimension.weo-su...
},
QTime: 0
}
}
```
Cf [Sentry issue](https://sentry.io/organizations/dbnomics/issues/3064955551)Christophe Benzchristophe.benz@nomics.worldChristophe Benzchristophe.benz@nomics.worldhttps://git.nomics.world/dbnomics/dbnomics-api/-/issues/3Could not return series (server error)2022-09-29T14:50:39ZChristophe Benzchristophe.benz@nomics.worldCould not return series (server error)Some series trigger a server error.
Example: https://api.db.nomics.world/v22/series/NBB/REGP3S13S15/S13.D63.254.CU.A
```json
{
"_meta": {
"version": "22.1.5"
},
"message": "500 Internal Server Error: The server encountered an...Some series trigger a server error.
Example: https://api.db.nomics.world/v22/series/NBB/REGP3S13S15/S13.D63.254.CU.A
```json
{
"_meta": {
"version": "22.1.5"
},
"message": "500 Internal Server Error: The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application."
}
```
The Sentry application monitoring has recorded thousands of similar errors: https://sentry.io/organizations/dbnomics/issues/3141320627/events/6c5af2eb14924e128a6e0eb68cab7862/events/?project=5548941&statsPeriod=30dChristophe Benzchristophe.benz@nomics.worldChristophe Benzchristophe.benz@nomics.worldhttps://git.nomics.world/dbnomics/dbnomics-api/-/issues/2SDMX key is not the same as variable key2018-01-29T09:58:59ZMichel JuillardSDMX key is not the same as variable keyIn the API SDMX, the key used for the queries is a transformation of the key of the variable (i.e. underscore are changed in dash), probably because of the way the slugs are done even if it shouldn't be directly related. This is confusin...In the API SDMX, the key used for the queries is a transformation of the key of the variable (i.e. underscore are changed in dash), probably because of the way the slugs are done even if it shouldn't be directly related. This is confusing for the userhttps://git.nomics.world/dbnomics/dbnomics-api/-/issues/1Use slugs as references between collections2017-10-20T16:13:52ZChristophe Benzchristophe.benz@nomics.worldUse slugs as references between collectionsCurrently in some collections like `datasets` the documents have a key `provider_name` which reference other documents of the `provider` collection.
This key should be `provider_slug` as it is more common to manipulate slugs (ie in URL ...Currently in some collections like `datasets` the documents have a key `provider_name` which reference other documents of the `provider` collection.
This key should be `provider_slug` as it is more common to manipulate slugs (ie in URL params) than names. As for now it leads to extraneous code just to find the name of a provider given its slug.
This impacts the database, and a migration script may be written.
Occurs in:
- `datasets.provider_name`
- `series.provider_name`
- maybe in other collections