源码构建gpdb - 9dian/Index GitHub Wiki

OS: Centos6.7 root

预置条件

安装python2.7.12

安装python2.7.12，但同时兼容yum使用python2.6.6

相关依赖包

[root@dev /opt]$ yum groupinstall "Development Tools" -y
[root@dev /opt]$ yum install zlib* openssl openssl-devel sqlite-devel -y
[root@dev /opt]$ yum install -y boost-system-1.41.0-28.el6.x86_64 boost-thread-1.41.0-28.el6.x86_64 policycoreutils-python

Python2.7.2的安装源码包

[root@dev /opt]$ wget http://www.python.org/ftp/python/2.7.12/Python-2.7.12.tar.xz
[root@dev /opt]$ mv Python-2.7.12.tar.xz /usr/local
[root@dev local]$ cd /usr/local
[root@dev local]$ unxz Python-2.7.12.tar.xz
[root@dev local]$ tar -vxf Python-2.7.12.tar

配置编译安装python2.7.12

[root@dev /Python-2.7.12]$ ./configure --prefix=/usr/local/python --enable-shared --enable-loadable-sqlite-extensions --with-zlib
#修改Setup.dist文件
[root@dev /Python-2.7.12]$ vim ./Modules/Setup.dist
#找到#zlib zlibmodule.c -I$(prefix)/include -L$(exec_prefix)/lib -lz去掉注释并保存`
[root@dev /Python-2.7.12]$ make && make install

兼容python2.6.6的yum

[root@dev ~]$ mv /usr/bin/python /usr/bin/python2.6.6
[root@dev ~]$ ln -s /usr/local/python/bin/python2.7 /usr/bin/python
#修改yum文件，将第一行的#!/usr/bin/python修改成  #!/usr/bin/python2.6.6
[root@dev ~]$ vim /usr/bin/yum
#查看python版本，已经更新为2.7.12
[root@dev ~]$ python -V
# 可选
[root@dev ~]$ bash -s 'cat >> /etc/ld.so.conf << eof
/usr/local/lib
/usr/local/lib64
eof'
[root@dev ~]$ /sbin/ldconfig
[root@dev ~]$ /sbin/ldconfig -v

安装pip

[root@dev ~]$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
[root@dev ~]$ python get-pip.py
[root@dev ~]$ pip -V

https://github.com/greenplum-db/gpdb/tree/5X_STABLE

https://github.com/greenplum-db/gpdb/tree/5.9.0

安装依赖的python模块

[root@dev ~]$ pip install conan

安装cmake3(可选)

https://centos.pkgs.org/6/epel-x86_64/cmake3-3.6.1-3.el6.x86_64.rpm.html

# Download the latest epel-release rpm from
[root@dev ~]$ wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
#Install epel-release rpm:
[root@dev ~]$ rpm -Uvh epel-release*rpm
[root@dev ~]$ yum install cmake3
[root@dev ~]$ ln -s /usr/bin/cmake3 /usr/bin/cmake

ref mast 安装依赖，pre install orca

#README.CentOS.bash (https://github.com/greenplum-db/gpdb/blob/master/README.CentOS.bash)

   [root@dev ~]$  sudo yum install -y epel-release
   [root@dev ~]$  sudo yum install -y \
    apr-devel \
    bison \
    bzip2-devel \
    cmake3 \
    flex \
    gcc \
    gcc-c++ \
    krb5-devel \
    libcurl-devel \
    libevent-devel \
    libkadm5 \
    libyaml-devel \
    libxml2-devel \
    openssl-devel \
    perl-ExtUtils-Embed \
    readline-devel \
    xerces-c-devel \
    zlib-devel

#python-dependencies.txt (https://github.com/greenplum-db/gpdb/blob/master/python-dependencies.txt)

   [root@dev ~]$  pip install argparse==1.2.1 \
   behave==1.2.4 \
   ecdsa==0.13 \
   epydoc==3.0.1 \
   lockfile==0.9.1 \
   logilab-astng==0.20.1 \
   logilab-common==0.50.1 \
   MarkupSafe==1.0 \
   mock==1.0.1 \
   paramiko==1.18.1 \
   parse==1.8.2 \
   psutil==4.0.0 \
   pycrypto==2.6.1 \
   pylint==0.21.0 \
   setuptools==36.6.0 \
   unittest2==0.5.1

===================================================================

Orca can now be installed on the local system using "make install" and be used as any normal system library

If you'd rather compile GPDB using ORCA in it's current location and then install ORCA into the gpdb installation location then first run the top level configure as follows: LD_LIBRARY_PATH=/opt/gpdb-5.9.0/depends/build/lib ./configure
--with-libraries=/opt/gpdb-5.9.0/depends/build/lib
--with-includes=/opt/gpdb-5.9.0/depends/build/include

Then run "make". Then run "LD_LIBRARY_PATH=/opt/gpdb-5.9.0/depends/build/lib make install"

These steps should work on both MacOS and Linux

安装devtoolset-6 with support for C++11 language features

sudo yum install -y centos-release-scl
sudo yum install -y devtoolset-6-toolchain
#echo 'source scl_source enable devtoolset-6' >> ~/.bashrc
scl enable devtoolset-6 bash

安装GPDB

构建数据库

# Configure build environment to install at /usr/local/gpdb
./configure --with-perl --with-python --with-libxml --with-gssapi --prefix=/usr/local/gpdb

# Compile and install
make -j8
make -j8 install

# Bring in greenplum environment into your running shell
source /usr/local/gpdb/greenplum_path.sh

Demo Cluster

[gpadmin@dev ~]$ cd gpAux/gpdemo
[gpadmin@dev gpdemo]$ . /usr/local/greenplum-db/greenplum_path.sh
#export DATADIRS=/home/gpadmin/gp_cluster_demo
[gpadmin@dev gpdemo]$ DATADIRS=/tmp/gpdb-cluster MASTER_PORT=15432 PORT_BASE=25432 make cluster
[gpadmin@dev gpdemo]$ make cluster
[gpadmin@dev gpdemo]$ psql -p 15432 postgres
[gpadmin@dev gpdemo]$ make clean

相关技术

在没有网络的机器编译，安装。

迁移python

迁移scl rpms包

rpmrebuild http://rpmrebuild.sourceforge.net/

yum install rpmrebuild

使用rpmrebuild在已经安装scl环境的机器上重新生成rpm包

rpmrebuild -d .  centos-release-scl-7-3.el6.centos.noarch
rpmrebuild -d . centos-release-scl-rh-2-3.el6.centos.noarch
rpmrebuild -d . devtoolset-6-6.1-1.el6.x86_64
rpmrebuild -d . devtoolset-6-binutils-2.27-10.el6.1.x86_64
rpmrebuild -d . devtoolset-6-dwz-0.12-1.el6.x86_64
rpmrebuild -d . devtoolset-6-dyninst-9.2.0-4.el6.x86_64
rpmrebuild -d . devtoolset-6-elfutils-0.168-3.el6.x86_64
rpmrebuild -d . devtoolset-6-elfutils-libelf-0.168-3.el6.x86_64
rpmrebuild -d . devtoolset-6-elfutils-libs-0.168-3.el6.x86_64
rpmrebuild -d . devtoolset-6-gcc-6.3.1-3.1.el6.x86_64
rpmrebuild -d . devtoolset-6-gcc-c++-6.3.1-3.1.el6.x86_64
rpmrebuild -d . devtoolset-6-gcc-gfortran-6.3.1-3.1.el6.x86_64
rpmrebuild -d . devtoolset-6-gdb-7.12.1-48.el6.x86_64
rpmrebuild -d . devtoolset-6-libquadmath-devel-6.3.1-3.1.el6.x86_64
rpmrebuild -d . devtoolset-6-libstdc++-devel-6.3.1-3.1.el6.x86_64
rpmrebuild -d . devtoolset-6-ltrace-0.7.91-13.el6.x86_64
rpmrebuild -d . devtoolset-6-make-4.1-3.el6.x86_64
rpmrebuild -d . devtoolset-6-memstomp-0.1.5-5.el6.x86_64
rpmrebuild -d . devtoolset-6-oprofile-1.1.0-4.el6.x86_64
rpmrebuild -d . devtoolset-6-perftools-6.1-1.el6.x86_64
rpmrebuild -d . devtoolset-6-runtime-6.1-1.el6.x86_64
rpmrebuild -d . devtoolset-6-strace-4.12-3.el6.x86_64
rpmrebuild -d . devtoolset-6-systemtap-3.0-8s.el6.x86_64
rpmrebuild -d . devtoolset-6-systemtap-client-3.0-8s.el6.x86_64
rpmrebuild -d . devtoolset-6-systemtap-devel-3.0-8s.el6.x86_64
rpmrebuild -d . devtoolset-6-systemtap-runtime-3.0-8s.el6.x86_64
rpmrebuild -d . devtoolset-6-toolchain-6.1-1.el6.x86_64
rpmrebuild -d . devtoolset-6-valgrind-3.12.0-1.el6.x86_64

下载scl-utils-20120927-27.el6_6.x86_64.rpm

使用rpmrebuild生成的scl-utils-201209XXX.rpm会有摘要不匹配问题。

[root@11_2_smdw rpms]# rpm -Uvh scl-utils-20120927-29.el6_9.x86_64.rpm 
Preparing...                ########################################### [100%]
   1:scl-utils              ########################################### [100%]
error: unpacking of archive failed on file /usr/bin/scl;5b3f0feb: cpio: Digest mismatch

下载scl-utils-20120927-27.el6_6.x86_64.rpm： http://vault.centos.org/6.6/updates/x86_64/Packages/?C=S;O=A

⚠️ **GitHub.com Fallback** ⚠️