Skip to content

D
destatis-fetcher

Home

Last edited by Pierre Dittgen
Page history

DeStatis fetcher

Website

To get a preview of a datacube (ex: 51000BJ001), follow:

  • click on "Themes" link (in "Available data" menu)
  • Select given theme (here 51 "Foreign Trade")
  • Click on "Time series icon"
  • Click on chosen datacube (here 51000BJ001)
  • Click on "Value Retrieval" button

Warning : After a few minutes, the user is automatically redirected to the home page :-(

Glossary

  • Wert: value
  • Fall: default value
  • Ganz: whole
  • Jahr: Year

Categories

Chosen by Thomas

  • 51 Foreign Trades
  • 61 Prices
  • 62 Earnings, Labour costs
  • 63 Income and expediture of households
  • 71 Public finances
  • 81 National accounts of the federation

Download

Download operation needs an account (username, password).

For each category, a datacubes file is downloaded from URL

curl "https://www-genesis.destatis.de/genesisWS/web/RechercheService_2010?method=DatenKatalog&kennung={USER_NAME}&passwort={PASSWORD}&filter={CAT}*&bereich=Alle&listenLaenge=&sprache=en" -o {CAT}/{CAT}.datacubes.xml

Parsing the datacubes file, we extract datacube ids for this category and retrieve all datacube structure file:

curl "https://www-genesis.destatis.de/genesisWS/web/ExportService_2010?method=DatenAufbau&kennung={USER_NAME}&passwort={PASSWORD}&namen={DATACUBE_CODE}&bereich=Alle&sprache=en" -o {CAT}/{DATACUBE_CODE}.structure.xml

datacube value file:

curl "https://www-genesis.destatis.de/genesisWS/web/ExportService_2010?method=DatenExport&kennung={USER_NAME}&passwort={PASSWORD}&namen={DATACUBE_CODE}&bereich=Alle&format=csv&werte=true&metadaten=false&zusatz=false&startjahr=&endjahr=&zeitscheiben=&inhalte=&regionalmerkmal=&regionalschluessel=&sachmerkmal=&sachschluessel=&sachmerkmal2=&sachschluessel2=&sachmerkmal3=&sachschluessel3=&stand=&sprache=en" -o {CAT}/{DATACUBE_CODE}.xml

Some datacube value file contain no value. Checks if contains data.

attribute info file (attribute code are extracted from datacube value file)

curl https://www-genesis.destatis.de/genesisWS/web/ExportService_2010?method=AuspraegungInformation&kennung={USER_NAME}&passwort={PASSWORD}&name={ATT_CODE}&bereich=Alle&sprache=en" -o att.{ATT_CODE}.xml

Convert

Gets all unique unit values from structure xml files:

xmlstarlet sel -t -m "//merkmale[art='I']" -v "concat('=',masseinheit,'=')" */*.structure.xml | sed -e $'s/==*/\\\n/g' | sort -u

Gets all unique types of period values:

xmlstarlet sel -t -m "//zeitraum" -v "concat('=',.,'=')" */*.structure.xml | sed -e $'s/==*/\\\n/g' | sort -u

Gets all unique frequency values:

xmlstarlet sel -t -m "//merkmale[typ='ZI']/code" -v "concat('=',.,'=')" */*.structure.xml | sed -e $'s/==*/\\\n/g' | sort -u