Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
E
eurostat-fetcher
Manage
Activity
Members
Labels
Plan
Issues
0
Issue boards
Milestones
Wiki
Code
Merge requests
0
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
dbnomics-fetchers
eurostat-fetcher
Commits
3a8e3cdd
Commit
3a8e3cdd
authored
7 years ago
by
Christophe Benz
Browse files
Options
Downloads
Patches
Plain Diff
Do not create series sub-directories
parent
0d989033
No related branches found
No related tags found
No related merge requests found
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
eurostat_to_dbnomics.py
+3
-5
3 additions, 5 deletions
eurostat_to_dbnomics.py
with
3 additions
and
5 deletions
eurostat_to_dbnomics.py
+
3
−
5
View file @
3a8e3cdd
...
...
@@ -108,7 +108,6 @@ def iter_git_objects_in_sdmx_file(sdmx_file_path, category_tree, dataset_json_st
elif
series_element_name
is
not
None
and
element
.
tag
==
series_element_name
and
action
==
"
end
"
:
# Exit a series.
series_element
=
element
series_tree
=
Tree
()
# Ignore some specific XML element attributes corresponding to series SDMX attributes,
# because series SDMX attributes do not exist in DB.nomics.
...
...
@@ -171,7 +170,7 @@ def iter_git_objects_in_sdmx_file(sdmx_file_path, category_tree, dataset_json_st
json
.
dumps
(
series_json
,
ensure_ascii
=
False
,
indent
=
2
,
sort_keys
=
True
).
encode
(
'
utf-8
'
)
)
yield
series_json_blob
series
_tree
.
add
(
b
"
series.json
"
,
git_blob_filemode
,
series_json_blob
.
id
)
dataset
_tree
.
add
(
"
{}.json
"
.
format
(
series_code
).
encode
(
'
utf-8
'
)
,
git_blob_filemode
,
series_json_blob
.
id
)
# Write observations.tsv
...
...
@@ -238,9 +237,8 @@ def iter_git_objects_in_sdmx_file(sdmx_file_path, category_tree, dataset_json_st
observations_tsv_blob
=
Blob
.
from_string
(
observations_tsv_str
.
encode
(
'
utf-8
'
))
yield
observations_tsv_blob
series_tree
.
add
(
b
"
observations.tsv
"
,
git_blob_filemode
,
observations_tsv_blob
.
id
)
yield
series_tree
dataset_tree
.
add
(
series_code
.
encode
(
'
utf-8
'
),
git_tree_filemode
,
series_tree
.
id
)
dataset_tree
.
add
(
"
{}.tsv
"
.
format
(
series_code
).
encode
(
'
utf-8
'
),
git_blob_filemode
,
observations_tsv_blob
.
id
)
# From https://stackoverflow.com/questions/12160418/why-is-lxml-etree-iterparse-eating-up-all-my-memory
# It's safe to call clear() here because no descendants will be accessed
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment