How to download and import data from zipped JSON files into PostgreSQL database - mohnanskygabriel/faersDb GitHub Wiki

1. Clone this GitHub-faersDb maven project into your IDE
2. Package or install the project
3. In target folder you will find faersDb-1.0-jar-with-dependencies.jar
4. In PostgeSQL database you have to create a new database e.g. faersdb and a schema with name faers otherwise data will be imported into public schema
5. It is recommended to create a database user e.g. faers with roles select, alter, update the created database and schema
6. Open command line in folder where is the .jar and run it with arguments

java -Xmx5500m -jar faersDb-1.0-jar-with-dependencies.jar ARGUMENTS

-Xmx5500m (Specify the maximum size, in bytes, of the memory allocation pool)

ARGUMENTS:

Download zip files:

-download https://api.fda.gov/download.json -directory D:/ZIP/

(download all drug/event files into D:/ZIP/ directory https://api.fda.gov/download.json is the location of the download.json file regularly by openFDA)

Extract zip files:

-extract D:/ZIP/ -extractDestination D:/JSON_EXTRACTED/

(extract all .zip files from D:/JSON/ directory and it's subdirectories into D:/JSON_EXTRACTED/ directory)

-import D:/JSON_EXTRACTED/ -db faersdb -ip 192.168.0.17 -port 5432 -user faers -pass brutepass

(import data from all .json files from directory D:/JSON_EXTRACTED/ including all subdirectories and import them into PostgreSQL database with name faersdb with ip 192.168.0.17 listening on port 5432 with user faers userpassword brutepass)

You can make downloading, extraction and importing at once by concatenation:

-download https://api.fda.gov/download.json -directory D:/ZIP/ -extract D:/ZIP/ -extractDestination D:/JSON_EXTRACTED/ -import D:/JSON_EXTRACTED/ -db faersdb -ip 192.168.0.17 -port 5432 -user faers -pass brutepass

So the complete commands for start will be like:

Only download:

java -Xmx5500m -jar faersDb-1.0-jar-with-dependencies.jar -download https://api.fda.gov/download.json -directory D:/ZIP/

Only extract:

java -Xmx5500m -jar faersDb-1.0-jar-with-dependencies.jar -extract D:/ZIP/ -extractDestination D:/JSON_EXTRACTED/

Only import:

java -Xmx5500m -jar faersDb-1.0-jar-with-dependencies.jar -import D:/JSON_EXTRACTED/ -db faersdb -ip 192.168.0.17 -port 5432 -user faers -pass brutepass

All:

java -Xmx5500m -jar faersDb-1.0-jar-with-dependencies.jar -download https://api.fda.gov/download.json -directory D:/ZIP/ -extract D:/ZIP/ -extractDestination D:/JSON_EXTRACTED/ -import D:/JSON_EXTRACTED/ -db faersdb -ip 192.168.0.17 -port 5432 -user faers -pass brutepass

Of course you can make download and extract first by command:

java -Xmx5500m -jar faersDb-1.0-jar-with-dependencies.jar -download https://api.fda.gov/download.json -directory D:/ZIP/ -extract D:/ZIP/ -extractDestination D:/JSON_EXTRACTED/