lua, torch install & troubleshooting - sogang-nlp-paper/WNGT-2019-DGT-NLG-Track GitHub Wiki

Requirements

  1. Download Lua from the official website and build.
curl -R -O http://www.lua.org/ftp/lua-5.3.5.tar.gz
tar zxf lua-5.3.5.tar.gz
cd lua-5.3.5
make linux test
sudo make install
  1. Download torch from the official website and install.
git clone https://github.com/torch/distro.git ~/torch --recursive
cd ~/torch; bash install-deps;
./install.sh

Trouble shooting

Torch ์„ค์น˜

./install.sh๋กœ torch ์„ค์น˜์‹œ ์•„๋ž˜์™€ ๊ฐ™์€ ์—๋Ÿฌ๊ฐ€ ๋‚ฌ๋‹ค.

CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
CUDA_cublas_device_LIBRARY (ADVANCED)
linked by target "THC" in directory /home/hwijeen/Downloads/distro/extra/cutorch/lib/THC

CuBLAS๋Š” basic linear algebra๋ฅผ ๋‹ด๋‹นํ•˜๋Š” ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ์ธ๋ฐ, CUDA 10.1๋ถ€ํ„ฐ๋Š” ๋น ์ ธ์žˆ๋‹ค. ์„œ๋ฒ„์— CUDA 10.0์„ ์„ค์น˜ํ•˜๊ณ  PATH, LD_LIBRARY_PATH๋ฅผ ์žก์•„์คฌ๋‹ค. ์ดํ›„์— ./install.sh์„ ๋‹ค์‹œ ํ–ˆ๋Š”๋ฐ ๊ฐ™์€ ์—๋Ÿฌ๊ฐ€ ๋‚ฌ๊ณ , ์—๋Ÿฌ ๋ฉ”์‹œ์ง€์— -- got cuda version 10.1๊ฐ€ ์žˆ์—ˆ๋‹ค. ์–ด๋Š ๊ฒฝ๋กœ๋กœ CUDA 10.1์ด ์žกํžˆ๋Š” ๊ฑด์ง€ ๋ชจ๋ฅด๊ฒ ์–ด์„œ ์„œ๋ฒ„์—์„œ ์•„์˜ˆ CUDA 10.1์„ ์ง€์›Œ๋ฒ„๋ ธ๋‹ค. apt๋ฅผ ์ด์šฉํ•œ ์ œ๊ฑฐ ํ›„ /usr/local์— ์žˆ๋Š” ๊ฑฐ๊นŒ์ง€ ์ง์ ‘ ์ง€์› ๋‹ค.

apt --installed list | grep cuda # ํ™•์ธ
sudo apt-get --purge remove cuda-10.1
sudo apt-get --purge remove cuda-toolkit-10-1
suro rm -rf /usr/local/cuda-10.1 

(์—ฌ๊ธฐ์„œ๋ถ€ํ„ฐ ๋กœ์ปฌ์€ ํฌ๊ธฐํ•˜๊ณ  Docker๋กœ ๋„˜์–ด์˜ด) ์—๋Ÿฌ๋Š” ์•„๋ž˜์™€ ๊ฐ™์•˜๋Š”๋ฐ, gcc ๋ฒ„์ „ ๋ฌธ์ œ์˜€๋‹ค. Github issue ์กฐ์–ธ์„ ๋”ฐ๋ผ gcc version์„ 4.9๋กœ ๋ฐ”๊ฟ”์คฌ๋‹ค.

jopts=$(getconf _NPROCESSORS_CONF) # ์˜ค๋ฅ˜ ๋ฉ”์‹œ์ง€ ์ค‘ ์ผ๋ถ€

sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install gcc-4.9 g++-4.9
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.9 60 --slave /usr/bin/g++ g++ /usr/bin/g++-4.9

๊ทธ๋Ÿฌ๋‚˜ gcc4.9๋ฒ„์ „์„ ๋ฐ”๊ฟ”์ฃผ๋‹ˆ ์—๋Ÿฌ. ๊ฒ€์ƒ‰ํ•ด๋ณด๋‹ˆ gfortran๊ณผ ๋ฒ„์ „์ด ๋‹ฌ๋ผ์ ธ์„œ ์˜ค๋ฅ˜๊ฐ€ ๋‚œ๋‹จ๋‹ค

/usr/bin/ld: cannot find -lgfortran
collect2: error: ld returned 1 exit status
Makefile:157: recipe for target '../libopenblas_haswellp-r0.3.7.dev.so' failed
make[1]: Leaving directory '/tmp/tmp.KmVvu8PjfI/OpenBLAS/exports'
make[1]: *** [../libopenblas_haswellp-r0.3.7.dev.so] Error 1
make: *** [shared] Error 2
Makefile:101: recipe for target 'shared' failed
Error. OpenBLAS could not be compiled

๊ทธ๋ž˜์„œ gfortran-4.9 ์„ค์น˜ํ•ด์ฃผ๊ณ  uptate-alternatives ์‚ฌ์šฉํ•ด์„œ ๊ธฐ๋ณธ gfortran์œผ๋กœ ์žก์•„์ฃผ๊ธฐ

sudo apt-get update
sudo apt-get install gfortran-4.9
sudo update-alternatives --install /usr/bin/gfortran gfortran /usr/bin/gfortran-4.9

์—ฌ๊ธฐ์— ๋‚˜์˜จ cutorh ๊ด€๋ จ ์—๋Ÿฌ๋„ ๊ผญ ์žก์•„์ฃผ์ž

export TORCH_NVCC_FLAGS="-D__CUDA_NO_HALF_OPERATORS__"

cutorch๋งŒ ๋”ฐ๋กœ ์„ค์น˜์‹œ๋„

luarocks๋ฅผ ํ†ตํ•ด cutorch ํŒจํ‚ค์ง€๋ฅผ ์„ค์น˜ํ•˜๋ ค๋‹ˆ ์•„๋ž˜์™€ ๊ฐ™์€ ์˜ค๋ฅ˜๊ฐ€ ๋‚ฌ๋‹ค

luarocks install cutorch # ์„ค์น˜
CMake Error at /home/hwijeen/Downloads/distro/install/share/cmake/torch/FindCUDA.cmake:643 (message):
  Specify CUDA_TOOLKIT_ROOT_DIR

Stackoverflow๋‹ต๋ณ€์„ ๋”ฐ๋ผ cmake CUDA_TOOLKIT_ROOT_DIR์„ ์ง์ ‘ ์žก์•„์คฌ๋‹ค.

cmake -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda # torch๊ฐ€ ์„ค์น˜๋œ ๋””๋ ‰ํ† ๋ฆฌ์—์„œ