GitLab Source install - HVboom/HowTo-DigitalOcean GitHub Wiki

Description

Version Control on your Server.

Manage Git repositories with fine grained access controls that keep your code secure

Perform code reviews and enhance collaboration with merge requests

Each project can also have an issue tracker and a wiki

Used by more than 100,000 organizations, GitLab is the most popular solution to manage Git repositories on-premises

Completely free and open source (MIT Expat license)

Powered by Ruby On Rails

💬 copied from the GitLab Community Edition page

Setup

Ideas taken from following resources:

Prerequisites

Installed software packages
- Postfix
- Apache and Phusion Passenger
- MySQL and phpMyAdmin
- Git
- RVM and Ruby installed for the git user
- Redis
Add git user to the redis group to be able to write to the redis socket /var/run/redis/redis.sock: sudo pw groupmod redis -m git
Add git user to the www group that Apache can read the application configurations: sudo pw groupmod www -m git
Create a MySQL user for the GitLab data base using phpMyAdmin interface
Change the collation for the GitLab data base to utf8mb4_unicode_ci

Login as git user

Clone GitLab branch 8-16-stable:

# Allow Apache to reach GitLab
cd $HOME
chmod o+rx $HOME

mkdir gitlab
chgrp www gitlab
cd $HOME/gitlab
git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b 8-16-stable .

Prepare necessary configuration files

GitLab - $HOME/gitlab/config/gitlab.yml - cp $HOME/gitlab/config/gitlab.yml.example $HOME/gitlab/config/gitlab.yml:

...

 ## GitLab settings
gitlab:
  ## Web server settings (note: host is the FQDN, do not include http://)
  host: gitlab.hvboom.org
  # port: 80 # Set to 443 if using HTTPS, see installation.md#using-https for additional HTTPS configuration details
  # https: false # Set to true if using HTTPS, see installation.md#using-https for additional HTTPS configuration details
  port: 443 # Set to 443 if using HTTPS, see installation.md#using-https for additional HTTPS configuration details
  https: true # Set to true if using HTTPS, see installation.md#using-https for additional HTTPS configuration details

...

  ## Email settings
  # Uncomment and set to false if you need to disable email sending from GitLab (default: true)
  # email_enabled: true
  # Email address used in the "From" field in mails sent by GitLab
  email_from: [email protected]
  email_display_name: GitLab
  email_reply_to: [email protected]

...

# ATTENTION: no symbolic linked directories allowed
satellites:
  path: /usr/home/git/gitlab-satellites/

repositories:
  storages: # You must have at least a `default` storage path.
    default: /usr/home/git/repositories/

...

# ATTENTION: no symbolic linked directories allowed
gitlab_shell:
  path: /usr/home/git/gitlab-shell/
  hooks_path: /usr/home/git/gitlab-shell/hooks/

...

## Git settings
# CAUTION!
# Use the default values unless you really know what you are doing
git:
  bin_path: /usr/local/bin/git

...

rack_attack:
  git_basic_auth:
    # Rack Attack IP banning enabled
    enabled: true
    #
    # Whitelist requests from 127.0.0.1 for web proxies (NGINX/Apache) with incorrect headers
    ip_whitelist: ["127.0.0.1"]
    #
    # Limit the number of Git HTTP authentication attempts per IP
    maxretry: 10
    #
    # Reset the auth attempt counter per IP after 60 seconds
    findtime: 60
    #
    # Ban an IP for one hour (3600s) after too many auth attempts
    bantime: 3600

...

a. Optional setup GitHub credentials to import repositories directly (see GitHub integration)

...

## OmniAuth settings
omniauth:
  # Allow login via Twitter, Google, etc. using OmniAuth providers
  enabled: true

...

  # CAUTION!
  # This allows users to login without having a user account first. Define the allowed providers
  # using an array, e.g. ["saml", "twitter"], or as true/false to allow all providers or none.
  # User accounts will be created automatically when authentication was successful.
  allow_single_sign_on: ["github"] 

...

  providers:
    # - { name: 'google_oauth2', 
    #     label: 'Google',
    #     app_id: 'YOUR_APP_ID', 
    #     app_secret: 'YOUR_APP_SECRET',
    #     args: { access_type: 'offline', approval_prompt: '' } }
    # - { name: 'twitter', 
    #     app_id: 'YOUR_APP_ID', 
    #     app_secret: 'YOUR_APP_SECRET' }
    - { name: 'github', 
        label: 'GitHub',
        app_id: '<Client ID>',
        app_secret: '<Client Secret>',
        args: { scope: 'user:email' } }
...

Rack attack - $HOME/gitlab/config/initializers/rack_attack.rb

Defaults in the example file are fine: cp $HOME/gitlab/config/initializers/rack_attack.rb.example $HOME/gitlab/config/initializers/rack_attack.rb
Redis - $HOME/gitlab/config/resque.yml

Defaults in the example file are fine: cp $HOME/gitlab/config/resque.yml.example $HOME/gitlab/config/resque.yml

MySQL - $HOME/gitlab/config/database.yml:

Use the provided example file as base: cp $HOME/gitlab/config/database.yml.mysql $HOME/gitlab/config/database.yml

#
# PRODUCTION
#
production:
  adapter: mysql2
  encoding: utf8mb4
  collation: utf8mb4_unicode_ci
  reconnect: false
  database: GitLab
  pool: 10
  username: GitLab
  password: <secure password>
  # host: localhost
  # socket: /tmp/mysql.sock

...

Miscellaneous

# Configure Git global settings for git user, used when editing via web editor
git config --global core.autocrlf input

Mail - $HOME/gitlab/config/initializers/smtp_settings.rb - cp $HOME/gitlab/config/initializers/smtp_settings.rb.sample $HOME/gitlab/config/initializers/smtp_settings.rb:

# To enable smtp email delivery for your GitLab instance do the following:
# 1. Rename this file to smtp_settings.rb
# 2. Edit settings inside this file
# 3. Restart GitLab instance
#
# For full list of options and their values see http://api.rubyonrails.org/classes/ActionMailer/Base.html
#
# If you change this file in a Merge Request, please also create a Merge Request on https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests

if Rails.env.production?
  Gitlab::Application.config.action_mailer.delivery_method = :smtp

  ActionMailer::Base.smtp_settings = {
    address: "localhost",
    port: 25,
    domain: "HVboom.org"
  }
end

Setup necessary run-time directories

cd $HOME
mkdir gitlab-satellites && chmod ug=rwx,o= gitlab-satellites
mkdir repositories && chmod ug=rwx,o= repositories

Bootstrap GitLab

Missing development packages

# necessary for gem: Rugged
sudo pkg install gmake
sudo pkg install cmake
sudo pkg install pkgconf

Install the GEMs

# set bundler options (http://bundler.io/v1.3/man/bundle-config.1.html) using
#    MySQL (not PostgreSQL) and Phusion Passenger (not Unicorn)
mkdir -p $HOME/gitlab/.bundle && touch $HOME/gitlab/.bundle/config
echo "---"                                                        >> $HOME/gitlab/.bundle/config
echo "BUNDLE_WITHOUT: development:test:kerberos:postgres:unicorn" >> $HOME/gitlab/.bundle/config
echo "BUNDLE_WITH: mysql"                                         >> $HOME/gitlab/.bundle/config
echo "BUNDLE_FROZEN: '1'"                                         >> $HOME/gitlab/.bundle/config
echo "BUNDLE_PATH: vendor/bundle"                                 >> $HOME/gitlab/.bundle/config
echo "BUNDLE_DISABLE_SHARED_GEMS: '1'"                            >> $HOME/gitlab/.bundle/config

cd $HOME/gitlab      # probably start ruby and bundler installation, if .rvmrc exists
gem install bundler  # if not installed automatically

bundle install   # --without development test kerberos postgres unicorn - these options are already set in .bundle/config

❗Excecuting rake tasks with bundle exec seems not to read the configuration file. Therefore the mysql group has to be activated manually in the Gemfile.

Install GitLab shell

cd $HOME/gitlab
bundle exec rake gitlab:shell:install REDIS_URL=unix:/var/run/redis/redis.sock RAILS_ENV=production
chgrp -R www $HOME/gitlab-shell

Check shell configuration: $HOME/gitlab-shell/config.yml

❗Attention

It is not allowed to have a symbolic link as the repos_path. Therefore instead of using $HOME/git/repositories/ you have to use /usr/home/git/repositories/

See GitLab Trouble-Shooting-Guide

---
user: git
gitlab_url: https://gitlab.hvboom.org/
http_settings:
  self_signed_cert: false
  ca_file: "/usr/local/etc/letsencrypt/live/hvboom.org/fullchain.pem"
  ca_path: "/usr/local/etc/letsencrypt/live/hvboom.org"
auth_file: "/usr/home/git/.ssh/authorized_keys"
redis:
  bin: "/usr/local/bin/redis-cli"
  namespace: resque:gitlab
  socket: "/var/run/redis/redis.sock"
log_level: INFO
audit_usernames: false

Adjust fixture for administrator user $HOME/gitlab/db/fixtures/production/001_admin.rb:

user_args = {
  email:    ENV['GITLAB_ROOT_EMAIL'].presence || '[email protected]',
  name:     'GitLab',
  username: 'root',
  admin:    true
}
...

Install GitLab

cd $HOME/gitlab
bundle exec rake gitlab:setup RAILS_ENV=production GITLAB_ROOT_PASSWORD=<secure password>
# Type 'yes' to create the database tables.
# When done you see 'Administrator account created:'

Compile assets

cd $HOME/gitlab
bundle exec rake assets:precompile RAILS_ENV=production

Adjust file and directory access rights

# Fix the access to the repositories
chmod -R ug+rwX,o= $HOME/repositories/
chmod -R ug-s $HOME/repositories/
find $HOME/repositories/ -type d -print0 | xargs -0 chmod g+s

# Fix the access to the satellites
chmod -R ug+rwX,o= $HOME/gitlab-satellites

# create upload directory
mkdir -m 700 $HOME/gitlab/public/uploads
mkdir -m 700 $HOME/gitlab/public/uploads/tmp

Configurate Sidekiq

Create a start script /usr/local/etc/rc.d/sidekiq:

#!/bin/sh
# 
# Sidekiq

# Based on https://gist.github.com/denvazh/5588936 written by Denis Vazhenin <[email protected]>
# Remove everything but the Sidekiq part

# PROVIDE: sidekiq
# REQUIRE: redis mysql
# KEYWORD: shutdown
#
# Requirements:
# 
# Before using this script, please install gitlab using guidelines from here:
# https://github.com/gitlabhq/gitlabhq/blob/master/doc/install/installation.md
#
# This file should be placed to: /usr/local/etc/rc.d/sidekiq
#
# Add the following lines to /etc/rc.conf to enable sidekiq:
#
# Required:
# sidekiq_enable="YES"
#
# Optional:
# sidekiq_chdir
# sidekiq_user
# sidekiq_env

. /etc/rc.subr

name="sidekiq"
rcvar="${name}_enable"

extra_commands="status restart"

###############################################################################
# Set default values (overrides should be placed in /etc/rc.conf )
###############################################################################

load_rc_config $name
: ${sidekiq_enable:=NO}
: ${sidekiq_user:=git}
: ${sidekiq_group:=git}
: ${sidekiq_chdir:=/home/${sidekiq_user}/gitlab}
: ${sidekiq_env:=production}
: ${sidekiq_log:=/var/log/${name}.log}

required_dirs="${sidekiq_chdir}"

_grep=`command -v grep >/dev/null 2>&1 && command -v grep`
_pgrep=`command -v pgrep >/dev/null 2>&1 && command -v pgrep`
_ps=`command -v ps >/dev/null 2>&1 && command -v ps`
_wc=`command -v wc >/dev/null 2>&1 && command -v wc`
_printf=`command -v printf >/dev/null 2>&1 && command -v printf`
_mkdir=`command -v mkdir >/dev/null 2>&1 && command -v mkdir`

# _bundle=/usr/local/bin/bundle
# _bundle=/home/git/.rvm/gems/ruby-2.2.1/bin/bundle
_bundle=`/usr/bin/su - git -c "cd ${sidekiq_chdir} >/dev/null 2>&1 && command -v bundle >/dev/null 2>&1 && command -v bundle"`

PID_PATH="${sidekiq_chdir}/tmp/pids"
pidfile="${PID_PATH}/${name}.pid"
STOP_SIDEKIQ="RAILS_ENV=${sidekiq_env} ${_bundle} exec rake ${name}:stop"
START_SIDEKIQ="RAILS_ENV=${sidekiq_env} ${_bundle} exec rake ${name}:start"

start_cmd="${name}_start"
stop_cmd="${name}_stop"
restart_cmd="${name}_restart"
status_cmd="${name}_status"

###############################################################################
# Helper functions
###############################################################################

__pid=0
__status=0


sidekiq_findbundler(){
  _bundle=sidekiq_execute "cd ${sidekiq_chdir} >/dev/null 2>&1 command -v bundle >/dev/null 2>&1 && command -v bundle"

  if [ x"${_bundle}" = x ]; then
    $_printf 'Unable to find %s command' 'bundle'
    exit 1
  fi
}

sidekiq_execute(){
  # ${_printf} '  Execute %s ...\n' "$1"
  # use - option to setup full environment
  /usr/bin/su - ${sidekiq_user} -c "$1"
}

sidekiq_check_if_running(){
  # checking if bundle gem is installed before doing anything
  #sidekiq_findbundler

  # check if sidekiq process is already running
  if [ -f $pidfile ]; then
    __pid=`${_pgrep} -F $pidfile 2>/dev/null`

    if [ x"${__pid}" != x ]; then
      __status=`${_ps} auxww | ${_grep} ${__pid} | ${_grep} -v "grep" | ${_wc} -l`
    else
      __pid=0
    fi
  else
    __pid=0
    __status=0
  fi 
}

sidekiq_check_if_root(){
  if [ `id -u` -ne 0 ]; then
    ${_printf} 'Must be a root user (current is %s)\n' `whoami`
    exit 1
  fi
}

###############################################################################
# Public functions
###############################################################################

sidekiq_prestart_check(){
  sidekiq_check_if_running

  # if process is already running, exit with code 1
  if [ ${__pid} -ne 0 ] && [ ${__status} -ne 0 ]; then
    ${_printf} '%s is already running...pid is %s\n' ${name} ${__pid}
    exit 1
  fi

  sidekiq_check_if_root
}

sidekiq_prestop_check(){
  sidekiq_check_if_running

  # if process is not running, exit with code 1
  if [ ${__pid} -eq 0 ] && [ ${__status} -eq 0 ]; then
    ${_printf} '%s not started\n' ${name}
    exit 1
  fi

  sidekiq_check_if_root
}

sidekiq_start(){
  sidekiq_prestart_check

  ${_printf} 'Start %s service...\n' ${name} | tee -a ${sidekiq_log}
  sidekiq_execute "cd ${sidekiq_chdir} >/dev/null 2>&1 && ${_mkdir} -p $PID_PATH && $START_SIDEKIQ &"

  sidekiq_status
}

sidekiq_stop(){
  sidekiq_prestop_check

  ${_printf} 'Stop %s service...\n' ${name} | tee -a ${sidekiq_log}
  sidekiq_execute "cd ${sidekiq_chdir} >/dev/null 2>&1 && ${_mkdir} -p $PID_PATH && $STOP_SIDEKIQ &"

  sidekiq_status
}

sidekiq_restart(){
  sidekiq_check_if_root

  cd ${sidekiq_chdir} >/dev/null 2>&1
  ${_printf} 'Restarting %s service...\n' ${name} | tee -a ${sidekiq_log}

  sidekiq_check_if_running

  # check if background task manager sidekiq is running, if not then don't kill it 
  if [ ${__pid} -ne 0 ] && [ ${__status} -ne 0 ]; then
    sidekiq_stop
  fi

  # start services normally
  sidekiq_start
  ${_printf} 'Service %s restarted.\n' ${name} | tee -a ${sidekiq_log}
}

sidekiq_status(){
  # wait to be sure services have enough time to start or stop
  echo -n "Get process status "
  sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .;
  sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .;
  sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .;
  sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .;
  sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .;
  sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo -n .; sleep 1; echo .;

  sidekiq_check_if_running

  # checking if gitlab sidekiq service is running
  if [ ${__pid} -ne 0 ] && [ ${__status} -ne 0 ]; then
    ${_printf} '%s service sidekiq is running with pid %s\n' ${name} ${__pid} | tee -a ${sidekiq_log}
  else
    ${_printf} '%s service sidekiq is not running\n' ${name} | tee -a ${sidekiq_log}
  fi
}

PATH="${PATH}:/usr/local/bin"

run_rc_command "$1"

Enable service on startup: sudo sysrc sidekiq_enable=yes
Start service: sudo service sidekiq restart

Configurate Apache
- Activating a virtual host to serve the GitLab application, by creating a dedicated configuration file. (No direct manipulation of the Apache configuration)
- Be sure to restart Apache sudo service apache24 restart
Check setup
- GitLab shell: $HOME/gitlab-shell/bin/check or cd $HOME/gitlab && bundle exec rake gitlab:gitlab_shell:check RAILS_ENV=production
- GitLab environment: cd $HOME/gitlab && bundle exec rake gitlab:env:info RAILS_ENV=production
- GitLab application: cd $HOME/gitlab && bundle exec rake gitlab:app:check RAILS_ENV=production
  - Error on init script can be ignored, because we use Phusion Passenger to run the application
- GitLab sidekiq: sudo service sidekiq status
Secure git user (see Git)

Troubleshooting

Wrong environment setup

Accessing web page gitlab.hvboom.org:

env: bash: No such file or directory

Cause: Phusion Passenger is not able to access all environment settings

Solution:

Check the PATH variable settings in $HOME/.profile:

if [ `echo "$PATH" | grep -c "/usr/local/bin"` == 0 ] 
then
  export PATH="$PATH:/usr/local/bin" # Add standard commands to PATH
fi

if [ `echo "$PATH" | grep -c "/usr/local/share/bin"` == 0 ] 
then
  export PATH="$PATH:/usr/local/share/bin" # Add shared commands to PATH
fi

if [ `echo "$PATH" | grep -c "$HOME/.rvm/bin"` == 0 ] 
then
  export PATH="$PATH:$HOME/.rvm/bin" # Add RVM to PATH for scripting
fi

Permission denied

Accessing web page gitlab.hvboom.net:

Forbidden

You don't have permission to access / on this server.
Apache log file /var/log/http-error.log:

AH01276: Cannot serve directory /home/git/gitlab/public/: No matching DirectoryIndex (none) found, and server-generated directory index forbidden by Options directive

Cause: Phusion Passenger is not able to read file $HOME/gitlab/config.ru and therefore do not switch the user to run the Ruby On Rails application as user git

Solution:

Change permission to allow user www to access $HOME/gitlab/config.ru:
```
chmod o+rX $HOME $HOME/gitlab $HOME/gitlab/config.ru
```
If you still get the permission denied error, you can test the accessibility by linking the file into the public directory
```
ln -s $HOME/gitlab/config.ru $HOME/gitlab/public/index.html
```
If you do see the content of the config.ru file in your browser, than the permissions are Ok.

Alternatively you could check with: sudo -u www /usr/local/share/bin/check-permissions /home/git/gitlab/config.ru

Push error

remote: GitLab: The project you were looking for could not be found. To gitlab.hvboom.org:Mario/RolfMeier.git ! [remote rejected] master -> master (pre-receive hook declined) error: failed to push some refs to

Cause: /home/git/gitlab/config/gitlab.yml contain symlink to repositories

Solution: ensure all path variables do not use any symlink components - at least for the repository definition

Seed DB error

❗Not an issue with Rails version 4.2.6

ActiveRecord::RecordInvalid: Validation failed: State is invalid

Cause: Gem state_machine does not work with Rails 4.2.4

Solution: Change fixture to set the state attribute - state: :active

TODO

Define logrotate for GitLab and Phusion Passenger log files

Upgrade

❗not working on FreeBSD do to issues with gem 'grpc'

Refer to the installation guides: GitLab-CE update

GitLab - run commands in directory /home/git/gitlab

Gem option for GPGME: bundle config build.gpgme --use-system-libraries
Backup: bundle exec rake gitlab:backup:create RAILS_ENV=production

Upgrade

git fetch --all
git checkout -- db/schema.rb
git checkout -- locale
git checkout 10-1-stable
bundle install
bundle clean
bundle exec rake db:migrate RAILS_ENV=production
bundle exec rake gettext:compile RAILS_ENV=production
bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production
bundle exec rake cache:clear RAILS_ENV=production

bundle exec rake "gitlab:workhorse:install[/home/git/gitlab-workhorse]" RAILS_ENV=production

bundle exec rake "gitlab:gitaly:install[/home/git/gitaly]" RAILS_ENV=production
# Restrict Gitaly socket access
chmod 0700 /home/git/gitlab/tmp/sockets/private
chown git /home/git/gitlab/tmp/sockets/private
# /home/git/gitlab/config/gitlab.yml
gitlab:
  repositories:
    storages:
      default:
        path: /home/git/repositories
        gitaly_address: unix:/home/git/gitlab/tmp/sockets/private/gitaly.socket
  gitaly:
    token: '<secret token>'
# /home/git/gitaly/config.toml - check token

Check configuration: git diff origin/8-16-stable:config/gitlab.yml.example origin/10-1-stable:config/gitlab.yml.example

GitLab Shell - run commands in directory /home/git/gitlab-shell

Upgrade

git fetch --all --tags
git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION)
bin/compile

GitLab Workhorse - run commands in directory /home/git/gitlab-workhorse

Upgrade

git fetch --all --tags
git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION)
make

Gitaly - run commands in directory /home/git/gitaly

Upgrade

git fetch --all --tags
git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION)
make
mv env env.old
cp config.toml.example config.toml

Check installation

/home/git/gitlab
bundle exec rake gitlab:env:info RAILS_ENV=production
bundle exec rake gitlab:check RAILS_ENV=production

GitLab Source install - HVboom/HowTo-DigitalOcean GitHub Wiki

Description

Setup

Prerequisites

Login as git user

Clone GitLab branch 8-16-stable:

Prepare necessary configuration files

Setup necessary run-time directories

Bootstrap GitLab

Troubleshooting

Wrong environment setup

Permission denied

Push error

Seed DB error

TODO

Upgrade

⚠️ **GitHub.com Fallback** ⚠️

⚠️ GitHub.com Fallback ⚠️