Configuration - leondutoit/data-centric-programming GitHub Wiki

A typical task is to get data from a remote source like a database. In such cases you typically need to specify a combination of the following properties in one of your functions: username, password, host name, database name and many more. It is common practice for such credentials to be kept in separate repositories with more strict access rules - for security purposes.

So how can you get access to these credentials in your program? One way is to put all your configuration in a YAML file and to read the properties of that file at runtime.

Using configuration files

To avoid having to copy files from other repositories and to avoid secrets from ending up in your source code you can create a symbolic link to the file and reference the symbolic link instead of the file in your script. For example

$ ln -s <path_to_file_with_credentials> <name_of_symbolic_file>

YAML configuration

user: leon
pw: mypassword
host: some-random-machine-name.whatever.com
db: database_name

Note: in R you need the final line of the YAML file to be a blank new line.

Python

import yaml

with open("my_config_file.yaml") as config:
    config = yaml.load(config)

# access and print configuration
print config["user"], config["pw"], config["host"], config["db"]

R

library(YAML)

config <- yaml.load_file("my_config_filename.yaml")

# access and print the properties
print(c(config$user, config$pw, config$host, config$db))