Download data

Passed Started 1 week ago by
1Running with gitlab-runner 17.8.3 (690ce25c)
2  on Run fetchers JBzuiMazz, system ID: s_33ef491069c6
 3  Preparing the "docker" executor  
  00:27
4Using Docker executor with image git.nomics.world:4567/dbnomics-fetchers/ilo-fetcher:latest ...
5Authenticating with credentials from job payload (GitLab Registry)
6Pulling docker image git.nomics.world:4567/dbnomics-fetchers/ilo-fetcher:latest ...
7Using docker image sha256:ecb03e317c0397066dd65165ce99f32f632d8e1f44713276524f53da0d206e8a for git.nomics.world:4567/dbnomics-fetchers/ilo-fetcher:latest with digest git.nomics.world:4567/dbnomics-fetchers/ilo-fetcher@sha256:a5ddfd6e33de3fb5edd1d000ca19f5169bd0137d9305d81038cecee8fa856c7d ...
 8  Preparing environment  
  00:11
9Running on runner-jbzuimazz-project-78-concurrent-0 via tyche...
 10  Getting source from Git repository  
  00:05
11Skipping Git repository setup
12Skipping Git checkout
13Skipping Git submodules setup
 14  Executing "step_script" stage of the job script  
  00:14
15Using docker image sha256:ecb03e317c0397066dd65165ce99f32f632d8e1f44713276524f53da0d206e8a for git.nomics.world:4567/dbnomics-fetchers/ilo-fetcher:latest with digest git.nomics.world:4567/dbnomics-fetchers/ilo-fetcher@sha256:a5ddfd6e33de3fb5edd1d000ca19f5169bd0137d9305d81038cecee8fa856c7d ...
16$ PROVIDER_DIR="${WORKSPACES_ROOT_DIR}/${PROVIDER_SLUG}" # collapsed multi-line command
17$ mkdir -p "$WORKSPACE_DEBUG_DATA_DIR" "$WORKSPACE_SOURCE_DATA_DIR"
18$ if [ "$DOWNLOAD_MODE" == "incremental" ]; then # collapsed multi-line command
19$ echo "Downloading data to $WORKSPACE_SOURCE_DATA_DIR ..."
20Downloading data to /data/dbnomics-prod/workspaces/ilo/pipelines/485229/source-data ...
21$ export DEBUG_DATA_DIR="$WORKSPACE_DEBUG_DATA_DIR"
22$ cd /app
23$ time python download.py "$WORKSPACE_SOURCE_DATA_DIR"
242025-03-21 02:34:23 URL:https://webapps.ilo.org/ilostat-files/WEB_bulk_download/html/bulk_indicator.html [590177/590177] -> "/data/dbnomics-prod/workspaces/ilo/pipelines/485229/source-data/bulk_indicator.html" [1]
25FINISHED --2025-03-21 02:34:23--
26Total wall clock time: 1.1s
27Downloaded: 1 files, 576K in 0.07s (7.77 MB/s)
28$ if [ -f download_report.json ]; then # collapsed multi-line command
29real	0m2.500s
30user	0m0.243s
31sys	0m0.059s
32$ if [ -f "$WORKSPACE_SOURCE_DATA_DIR/errors.json" ]; then # collapsed multi-line command
33$ rmdir --ignore-fail-on-non-empty "$WORKSPACE_DEBUG_DATA_DIR"
 34  Uploading artifacts for successful job  
  00:05
35Uploading artifacts...
36WARNING: download_report.json: no matching files. Ensure that the artifact path is relative to the working directory (/builds/dbnomics-fetchers/ilo-fetcher) 
37WARNING: errors.json: no matching files. Ensure that the artifact path is relative to the working directory (/builds/dbnomics-fetchers/ilo-fetcher) 
38ERROR: No files to upload                          
 39  Cleaning up project directory and file based variables  
  00:05
40Job succeeded