Skip to content
Snippets Groups Projects

Download data

Passed Started by
Christophe Benz
@cbenz
1Running with gitlab-runner 17.8.3 (690ce25c)
2 on Run fetchers JBzuiMazz, system ID: s_33ef491069c6
3 Preparing the "docker" executor 00:08
4Using Docker executor with image git.nomics.world:4567/dbnomics-fetchers/nar-fetcher:latest ...
5Authenticating with credentials from job payload (GitLab Registry)
6Pulling docker image git.nomics.world:4567/dbnomics-fetchers/nar-fetcher:latest ...
7Using docker image sha256:0863e2dd094d341a4ccdb2cfa44c70ca0b781cf375a20c47d90c07bdf674b2f3 for git.nomics.world:4567/dbnomics-fetchers/nar-fetcher:latest with digest git.nomics.world:4567/dbnomics-fetchers/nar-fetcher@sha256:fbe18e301f2a1dcd8cf87971c60a57e4d48377bf07a6c383e6de5d297745a137 ...
8 Preparing environment 00:04
9Running on runner-jbzuimazz-project-236-concurrent-0 via tyche...
10 Getting source from Git repository 00:03
11Skipping Git repository setup
12Skipping Git checkout
13Skipping Git submodules setup
14 Executing "step_script" stage of the job script 00:09
15Using docker image sha256:0863e2dd094d341a4ccdb2cfa44c70ca0b781cf375a20c47d90c07bdf674b2f3 for git.nomics.world:4567/dbnomics-fetchers/nar-fetcher:latest with digest git.nomics.world:4567/dbnomics-fetchers/nar-fetcher@sha256:fbe18e301f2a1dcd8cf87971c60a57e4d48377bf07a6c383e6de5d297745a137 ...
16$ PROVIDER_DIR="${WORKSPACES_ROOT_DIR}/${PROVIDER_SLUG}" # collapsed multi-line command
17$ mkdir -p "$WORKSPACE_DEBUG_DATA_DIR" "$WORKSPACE_SOURCE_DATA_DIR"
18$ if [ "$DOWNLOAD_MODE" == "incremental" ]; then # collapsed multi-line command
19$ echo "Downloading data to $WORKSPACE_SOURCE_DATA_DIR ..."
20Downloading data to /data/dbnomics-prod/workspaces/nar/pipelines/485492/source-data ...
21$ export DEBUG_DATA_DIR="$WORKSPACE_DEBUG_DATA_DIR"
22$ cd /app
23$ time python download.py "$WORKSPACE_SOURCE_DATA_DIR"
24['https://reg.realtor.org/leadrshp.nsf/webassoc?OpenView&source=realtor', '/research-and-statistics/research-data-faq', 'https://store.realtor.org/search?search_api_views_fulltext=historical', '/newsroom/nar-statistical-news-release-schedule', '/research-and-statistics/citation-guidelines-for-nar-research-statistics']
25$ if [ -f download_report.json ]; then # collapsed multi-line command
26real 0m4.264s
27user 0m0.854s
28sys 0m0.044s
29$ if [ -f "$WORKSPACE_SOURCE_DATA_DIR/errors.json" ]; then # collapsed multi-line command
30$ rmdir --ignore-fail-on-non-empty "$WORKSPACE_DEBUG_DATA_DIR"
31 Uploading artifacts for successful job 00:03
32Uploading artifacts...
33WARNING: download_report.json: no matching files. Ensure that the artifact path is relative to the working directory (/builds/dbnomics-fetchers/nar-fetcher)
34WARNING: errors.json: no matching files. Ensure that the artifact path is relative to the working directory (/builds/dbnomics-fetchers/nar-fetcher)
35ERROR: No files to upload
36 Cleaning up project directory and file based variables 00:03
37Job succeeded