README.md 3.65 KB
Newer Older
Emmanuel Raviart's avatar
Emmanuel Raviart committed
1
2
3
4
5
6
7
8
9
10
11
12
# EZ-DDI

_Fetch, validate, convert & serve CESSDA-compliants DDI repositories._

## Installation

```bash
git clone https://git.nomics.world/progedo/ez-ddi.git
cd ez-ddi/
npm install
```

13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
### Database Creation

#### Using _Debian GNU/Linux_

As `root` user:

```bash
apt install postgresql
su - postgres
psql
```

#### Using _MacOS_

```bash
brew install postgresql
psql postgres
```

#### For everybody

```sql
CREATE USER ezddi WITH PASSWORD 'ezddi';
CREATE DATABASE ezddi WITH OWNER ezddi;
\connect ezddi
CREATE EXTENSION IF NOT EXISTS pg_trgm;
\q
logout # For Debian only
```

As normal user, create database tables:

```bash
npm run configure
```

Emmanuel Raviart's avatar
Emmanuel Raviart committed
49
50
## Usage

Emmanuel Raviart's avatar
Emmanuel Raviart committed
51
52
### Fetching Nesstar Servers

Emmanuel Raviart's avatar
Emmanuel Raviart committed
53
54
```bash
npx tsc
55
# ADISP
56
npx babel-node --extensions ".ts" src/scripts/retrieve_nesstar_ddis.js --url http://nesstar.progedo-adisp.fr/ ../public_data/adisp-ddi/
57
# CDSP Sciences Po
58
npx babel-node --extensions ".ts" src/scripts/retrieve_nesstar_ddis.js --url http://nesstar.sciences-po.fr/ ../public_data/cdsp-ddi/
59
# INED
60
npx babel-node --extensions ".ts" src/scripts/retrieve_nesstar_ddis.js --url http://nesstar.ined.fr/ ../public_data/ined-ddi/
61
# INED - Generations and Gender Survey
62
npx babel-node --extensions ".ts" src/scripts/retrieve_nesstar_ddis.js --url http://ggpsurvey.ined.fr/ ../public_data/ined-gpgsurvey-ddi/
63
# UK Data Service
64
npx babel-node --extensions ".ts" src/scripts/retrieve_nesstar_ddis.js --url http://nesstar.ukdataservice.ac.uk/ ../public_data/ukdataservice-ddi/
65
# Norwegian Centre for Research Data
66
npx babel-node --extensions ".ts" src/scripts/retrieve_nesstar_ddis.js --url http://nsddata.nsd.uib.no ../public_data/nsddata-ddi/
Emmanuel Raviart's avatar
Emmanuel Raviart committed
67
```
Emmanuel Raviart's avatar
Emmanuel Raviart committed
68
69
70
71

### Extracting TypeScript Raw Types from DDI files

```bash
72
npx babel-node --extensions ".ts" --max-old-space-size=10240 src/scripts/raw_types_from_ddi_files.ts ../public_data/adisp-ddi/ ../public_data/adisp-ddi/ ../public_data/ined-ddi/ --target=src/raw_types/code_books.ts
Emmanuel Raviart's avatar
Emmanuel Raviart committed
73
74
npx babel-node --extensions ".ts" --max-old-space-size=8192 src/scripts/raw_types_from_ddi_files.ts ../public_data/xml-ddi-adisp/ --target=src/raw_types/enquetes_adisp_fournies.ts
npx babel-node --extensions ".ts" --max-old-space-size=8192 src/scripts/raw_types_from_ddi_files.ts ../public_data/adisp-ddi/ --target=src/raw_types/enquetes_adisp.ts
75
76
npx babel-node --extensions ".ts" src/scripts/raw_types_from_ddi_files.ts ../public_data/cdsp-ddi/ --target=src/raw_types/enquetes_cdsp.ts
npx babel-node --extensions ".ts" src/scripts/raw_types_from_ddi_files.ts ../public_data/ined-ddi/ --target=src/raw_types/enquetes_ined.ts
Emmanuel Raviart's avatar
Emmanuel Raviart committed
77
78
79
80

# Prettify generated TypeScript files:
npm run prettier
```
81
82
83
84

### Indexing DDI files

```bash
85
86
87
88
89
90
npx babel-node --extensions ".ts" -- src/scripts/index_code_books.ts --path=adisp --title=\"Archives de données issues de la statistique publique \(ADISP\)\" ../public_data/adisp-ddi/
npx babel-node --extensions ".ts" -- src/scripts/index_code_books.ts --path=cdsp --title=\"SciencesPo Centre de données socio-politiques \(CDSP\)\" ../public_data/cdsp-ddi/
npx babel-node --extensions ".ts" -- src/scripts/index_code_books.ts --path=ined --title=\"Institut national d\'études démographiques \(INED\)\" ../public_data/ined-ddi/
npx babel-node --extensions ".ts" -- src/scripts/index_code_books.ts --path=ined/gpgsurvey --title=\"Enquête Générations et Genre du projet international Generations and Gender Programme \(GGP\)\" ../public_data/ined-gpgsurvey-ddi/
npx babel-node --extensions ".ts" -- src/scripts/index_code_books.ts --path=ukdataservice --title=\"UK Data Service\" ../public_data/ukdataservice-ddi/
npx babel-node --extensions ".ts" -- src/scripts/index_code_books.ts --path=nsddata --title=\"Norwegian Centre for Research Data \(NSD\)\" ../public_data/nsddata-ddi/
91
```