CartoDBInstallation - UCL-ShippingGroup/shipviz GitHub Wiki

See all related scripts here. To install and run an instance of CartoDB in a Ubuntu 14.04 vanilla machine, you can do the following:

sudo apt-get install -y git
git clone https://github.com/UCL-ShippingGroup/shipviz ~/shipviz
cd ~/shipviz/cartoDB/installation/
./cartoDB_install.sh
source ~/.bashrc
./cartoDB_run.sh

The default user created by this scripts is called development. CartoDB creates the new users with only 10GB space in the database and limitations on number of rows a table can have and how many things can be ingested at once. Those values can be changed using the order [user_name, import_file_size, table_row_count, concurrent_imports] in the following command:

rake cartodb:set_custom_limits_for_user[development,157286400000,10000000,10]

If the server connection is reset for whatever reason and CartoDB needs to be re-run, the best solution is to run the following:

#Start CartoDB SQL API
cd ~/CartoDB-SQL-API/
node app.js development &

#Start tiling API
cd ~/Windshaft-cartodb/
node app.js development &

#Start redis server that allows connection to CartoDB SQL API and Windshaft
sudo service redis-server start
#or, if already started:
sudo service redis-server restart

#Go to CartoDB repo folder because the following commands need to be run from the same place where the Gemfile is
cd ~/cartodb

#Start rails server - This is the HTTP server that serves the CartoDB website pages
RAILS_ENV=development bundle exec rails server

#Start resque - This is the task importer to load data, etc.
RAILS_ENV=development bundle exec ./script/resque

#Wait until everything is running (just a few seconds normally) and run
firefox http://development.localhost.lan:3000/login &

This will open the browser automatically. If the page doesn't load, it might be because the server is taking a little longer to load, so wait for the terminal to display a message like:

** [12:08:56 2015-07-17] 4905: Checking imports
** [12:08:56 2015-07-17] 4905: Checking users
** [12:08:56 2015-07-17] 4905: Sleeping for 5.0 seconds
** [12:08:56 2015-07-17] 4905: resque-1.25.2: Waiting for *

These are the detailed instructions:

Download CartoDB repo:

cd
git clone --recursive https://github.com/CartoDB/cartodb.git

Install PPAs:

sudo apt-get update
sudo apt-get install -y python-software-properties
sudo add-apt-repository -y ppa:cartodb/base
sudo add-apt-repository -y ppa:cartodb/gis
sudo add-apt-repository -y ppa:cartodb/mapnik
sudo add-apt-repository -y ppa:cartodb/nodejs
sudo add-apt-repository -y ppa:cartodb/redis
sudo add-apt-repository -y ppa:cartodb/postgresql-9.3
sudo add-apt-repository -y ppa:cartodb/nodejs-010
sudo add-apt-repository -y ppa:cartodb/varnish
sudo apt-get update

Install requirements:

#UTF-8
echo -e 'LANG=en_US.UTF-8\nLC_ALL=en_US.UTF-8' | sudo tee /etc/default/locale
source /etc/default/locale

#GIT
sudo apt-get -y install git

#VIM
sudo apt-get -y install vim

#CURL
sudo apt-get install -y curl

#MAKE
sudo apt-get install -y build-essential checkinstall 

#UNP
sudo apt-get install -y unp

#ZIP
sudo apt-get install -y zip

#GEOS
sudo apt-get install -y libgeos-c1 libgeos-dev

#GDAL
sudo apt-get install -y gdal-bin libgdal1-dev

#JSON-C
sudo apt-get install -y libjson0 python-simplejson libjson0-dev

#PROJ4
sudo apt-get install -y proj-bin proj-data libproj-dev

#NODEJS
sudo apt-get install -y nodejs-legacy
sudo apt-get install -y npm
sudo apt-get install -y libpango1.0-dev

#REDIS SERVER
sudo apt-get install -y redis-server

#PYTHON 
sudo apt-get install -y python2.7-dev
sudo apt-get install -y python-setuptools

#MapNik
sudo apt-get install -y libmapnik-dev python-mapnik mapnik-utils

#ImageMagick
sudo apt-get install -y imagemagick

#Varnish - no access to versions  < 3.x needed by CartoDB
#sudo apt-get install -y varnish=2.1.5.1-cdb1 #or any version <3.x

Install PostgreSQL:

sudo apt-get install -y postgresql-9.3 postgresql-client-9.3 postgresql-contrib-9.3 postgresql-server-dev-9.3
sudo apt-get install -y postgresql-plpython-9.3

#Configure all _md5_/_peer_ to _trust_: 
sudo sed -i -- 's/md5/trust/g' /etc/postgresql/9.3/main/pg_hba.conf 
sudo sed -i -- 's/peer/trust/g' /etc/postgresql/9.3/main/pg_hba.conf 

#Restart postgres: 
sudo /etc/init.d/postgresql restart

Install PostGIS:

cd /usr/local/src
sudo wget http://download.osgeo.org/postgis/source/postgis-2.1.7.tar.gz
sudo tar -xvzf postgis-2.1.7.tar.gz
cd postgis-2.1.7
sudo ./configure --with-raster --with-topology
sudo make
sudo make install

sudo su postgres <<'EOF'
POSTGIS_SQL_PATH=`pg_config --sharedir`/contrib/postgis-2.1
createdb -E UTF8 template_postgis
createlang -d template_postgis plpgsql
psql -d postgres -c "UPDATE pg_database SET datistemplate='true' WHERE datname='template_postgis'"
psql -d template_postgis -c "CREATE EXTENSION postgis"
psql -d template_postgis -c "CREATE EXTENSION postgis_topology"
psql -d template_postgis -c "GRANT ALL ON geometry_columns TO PUBLIC;"
psql -d template_postgis -c "GRANT ALL ON spatial_ref_sys TO PUBLIC;"
exit
EOF

cd /tmp
git clone https://github.com/CartoDB/pg_schema_triggers.git
cd pg_schema_triggers
sudo make all install PGUSER=postgres
sudo make installcheck PGUSER=postgres # to run tests

cd ..
git clone https://github.com/CartoDB/cartodb-postgresql.git
cd cartodb-postgresql
git checkout cdb
sudo make all install
echo "shared_preload_libraries = 'schema_triggers.so'" | sudo tee --append /etc/postgresql/9.3/main/postgresql.conf
sudo service postgresql restart # restart postgres
sudo PGUSER=postgres make installcheck # to run tests

Install Ruby:

#rbenv
git clone https://github.com/sstephenson/rbenv.git ~/.rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(rbenv init -)"' >> ~/.bashrc
export PATH="$HOME/.rbenv/bin:$PATH"
eval "$(rbenv init -)"

#ruby-build plugin for rbenv
git clone https://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build

#ruby 1.9.3-p551 needed by CartoDB
rbenv install 1.9.3-p551

#bundler
rbenv local 1.9.3-p551
gem install bundler

NVM for NodeJS (might need restarting the terminal):

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.25.4/install.sh | bash

#E.g.:
#nvm install v0.10
#nvm use 0.10

Install Python:

sudo su <<'EOF'
apt-get install python-setuptools
easy_install pip
export CPLUS_INCLUDE_PATH=/usr/include/gdal
export C_INCLUDE_PATH=/usr/include/gdal
pip install --no-use-wheel -r python_requirements.txt
exit
EOF

Configure raster import support:

sudo ln -s /usr/local/src/postgis-2.1.7/raster/loader/raster2pgsql /usr/bin/
sudo chown 501:staff /usr/local/src/postgis-2.1.7/raster/loader/.libs

Install CartoDB SQL API:

git clone git://github.com/CartoDB/CartoDB-SQL-API.git $HOME/CartoDB-SQL-API
cd $HOME/CartoDB-SQL-API
git checkout master
npm install
cp config/environments/development.js.example config/environments/development.js

Install Windshaft:

git clone git://github.com/CartoDB/Windshaft-cartodb.git $HOME/Windshaft-cartodb
cd $HOME/Windshaft-cartodb
git checkout master
npm install
cp config/environments/development.js.example config/environments/development.js

Run CartoDB!

#Run CartoDB SQL API
cd $HOME/CartoDB-SQL-API
node app.js development &

#Run Windshaft
cd $HOME/Windshaft-cartodb
node app.js development &

#Run Redis Server
redis-server &

#These need Redis-Server running
rbenv local 1.9.3-p551
bundle install

#Configure application constants
cp $HOME/cartodb/config/app_config.yml.sample $HOME/cartodb/config/app_config.yml

#Configure database details
cp $HOME/cartodb/config/database.yml.sample $HOME/cartodb/config/database.yml

#Add entries to /etc/hosts needed in development
echo "127.0.0.1 ${SUBDOMAIN}.localhost.lan" | sudo tee -a /etc/hosts

#Create a development user
sh  $HOME/cartodb/script/create_dev_user development development development [email protected]

#Start the resque daemon (needed for import jobs):
bundle exec  $HOME/cartodb/script/resque &

#Finally, start the CartoDB development server on port 3000:
bundle exec rails s -p 3000 &
                                       
#Open with Firefox 
firefox http://development.localhost.lan:3000/dashboard