Skip to content

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
    • Help
    • Support
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
D
destatis-fetcher
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
    • Cycle Analytics
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Charts
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Labels
    • Milestones
  • Merge Requests 0
    • Merge Requests 0
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Charts
  • Packages
    • Packages
    • Container Registry
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Charts
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • dbnomics-fetchers
  • destatis-fetcher
  • Wiki
  • Home

Home

Last edited by Pierre Dittgen May 04, 2018
Page history

DeStatis fetcher

Website

To get a preview of a datacube (ex: 51000BJ001), follow:

  • click on "Themes" link (in "Available data" menu)
  • Select given theme (here 51 "Foreign Trade")
  • Click on "Time series icon"
  • Click on chosen datacube (here 51000BJ001)
  • Click on "Value Retrieval" button

Warning : After a few minutes, the user is automatically redirected to the home page :-(

Glossary

  • Wert: value
  • Fall: default value
  • Ganz: whole
  • Jahr: Year

Categories

Chosen by Thomas

  • 51 Foreign Trades
  • 61 Prices
  • 62 Earnings, Labour costs
  • 63 Income and expediture of households
  • 71 Public finances
  • 81 National accounts of the federation

Download

Download operation needs an account (username, password).

For each category, a datacubes file is downloaded from URL

curl "https://www-genesis.destatis.de/genesisWS/web/RechercheService_2010?method=DatenKatalog&kennung={USER_NAME}&passwort={PASSWORD}&filter={CAT}*&bereich=Alle&listenLaenge=&sprache=en" -o {CAT}/{CAT}.datacubes.xml

Parsing the datacubes file, we extract datacube ids for this category and retrieve all datacube structure file:

curl "https://www-genesis.destatis.de/genesisWS/web/ExportService_2010?method=DatenAufbau&kennung={USER_NAME}&passwort={PASSWORD}&namen={DATACUBE_CODE}&bereich=Alle&sprache=en" -o {CAT}/{DATACUBE_CODE}.structure.xml

datacube value file:

curl "https://www-genesis.destatis.de/genesisWS/web/ExportService_2010?method=DatenExport&kennung={USER_NAME}&passwort={PASSWORD}&namen={DATACUBE_CODE}&bereich=Alle&format=csv&werte=true&metadaten=false&zusatz=false&startjahr=&endjahr=&zeitscheiben=&inhalte=&regionalmerkmal=&regionalschluessel=&sachmerkmal=&sachschluessel=&sachmerkmal2=&sachschluessel2=&sachmerkmal3=&sachschluessel3=&stand=&sprache=en" -o {CAT}/{DATACUBE_CODE}.xml

Some datacube value file contain no value. Checks if contains data.

attribute info file (attribute code are extracted from datacube value file)

curl https://www-genesis.destatis.de/genesisWS/web/ExportService_2010?method=AuspraegungInformation&kennung={USER_NAME}&passwort={PASSWORD}&name={ATT_CODE}&bereich=Alle&sprache=en" -o att.{ATT_CODE}.xml

Convert

Gets all unique unit values from structure xml files:

xmlstarlet sel -t -m "//merkmale[art='I']" -v "concat('=',masseinheit,'=')" */*.structure.xml | sed -e $'s/==*/\\\n/g' | sort -u

Gets all unique types of period values:

xmlstarlet sel -t -m "//zeitraum" -v "concat('=',.,'=')" */*.structure.xml | sed -e $'s/==*/\\\n/g' | sort -u

Gets all unique frequency values:

xmlstarlet sel -t -m "//merkmale[typ='ZI']/code" -v "concat('=',.,'=')" */*.structure.xml | sed -e $'s/==*/\\\n/g' | sort -u
Clone repository
  • Home
More Pages