How to run GrimoireLab - X-lab2017/grimoirelab GitHub Wiki
建议：4GB 以上内存。
GrimoireLab 在 GNU/Linux 平台下开发、测试通过。在 Windows 下（包括WSL）可能会出现问题。以下内容在 ubuntu18.04 测试通过。
- 安装 Docker：https://docs.docker.com/engine/install/ubuntu/
- 安装 Docker-Compose：https://docs.docker.com/compose/install/
- 设置系统 vm 参数：https://github.com/chaoss/grimoirelab-sirmordred/blob/master/Getting-Started.md#low-virtual-memory-
# Linux
$ sudo sysctl -w vm.max_map_count=262144
# Mac
$ $ screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty
sysctl -w vm.max_map_count=262144$ git clone https://github.com/chaoss/grimoirelab
$ cd grimoirelab/docker-compose该目录下包含两个 docker-compose.yml，其中 docker-compose-secured.yml 是需要账号/密码认证的版本，建议使用。
默认加载的是 default-grimoirelab-settings 中的配置文件，如需修改配置，直接修改这些配置文件即可。
# 启动时指定.yml文件
$ sudo docker-compose -f docker-compose-secured.yml up
# 如果要后台运行
$ sudo docker-compose -f docker-compose-secured.yml up -d启动后即可查看 http://localhost:5601/ 。
# 如果要重启
$ sudo docker-compose -f docker-compose-secured.yml restart
# 如果要关掉
$ sudo docker-compose -f docker-compose-secured.yml down参考资料:https://github.com/chaoss/grimoirelab/blob/master/docker-compose/README.md
- 安装 Docker：https://docs.docker.com/engine/install/ubuntu/
- 设置系统 vm 参数：https://github.com/chaoss/grimoirelab-sirmordred/blob/master/Getting-Started.md#low-virtual-memory-
$ git clone https://github.com/chaoss/grimoirelab
$ cd grimoirelab
$ sudo docker run -p 127.0.0.1:5601:5601 \
-v $(pwd)/default-grimoirelab-settings/projects.json:/projects.json \
-v $(pwd)/default-grimoirelab-settings/setup.cfg:/setup.cfg \
-t grimoirelab/full同样，可以按需修改配置文件，并挂载进去。
- https://github.com/chaoss/grimoirelab/blob/master/docker/README.md
-
https://chaoss.github.io/grimoirelab-tutorial/sirmordred/container.html
- 安装 Docker：https://docs.docker.com/engine/install/ubuntu/
- 安装 Docker-Compose：https://docs.docker.com/compose/install/
- 设置系统 vm 参数：https://github.com/chaoss/grimoirelab-sirmordred/blob/master/Getting-Started.md#low-virtual-memory-
- Python 与 Pip3
# python 3.5/3.6/3.7
$ python3 --version
Python 3.6.9
# 安装 python3-venv
$ sudo apt update
$ sudo apt -y install python3-venv
# pip3
$ pip3 --version
pip 20.1.1 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)
# 如果未安装 pip:
$ sudo apt install -y python3-pip
# 升级
$ sudo pip3 install --upgrade pip使用 docker-compose 启动，docker-compose.yml 文件内容可参考：https://github.com/X-lab2017/grimoirelab/blob/master/docker-compose/docker-compose-infra-secured.yml
# 新建 docker-compose.yml 并启动
$ sudo docker-compose up -d编译安装工具位于 grimoirelab/utils/ ，build_grimoirelab 各参数的意思可以通过 build_grimoirelab --help查看，或者参考：https://github.com/chaoss/grimoirelab/tree/master/utils
$ git clone https://github.com/chaoss/grimoirelab.git
$ cd grimoirelab/utils
$ ./build_grimoirelab -l debug --logfile /tmp/log --build --install --check \
--relfile ../releases/latest --reposdir /tmp/repos --venv /tmp/venv --install_venv /tmp/ivenv以上参数意思为：
- -l 指定日志等级
- --logfile 指定日志文件
- --relfile 指定 release 版本文件
- --reposdir 指定下载好的repo存放位置
- --venv 指定用户 build 各模块的 venv
- --install_venv 指定安装各模块的venv
如上，安装好的 venv 位于 /tmp/ivenv：
$ source /tmp/ivenv/bin/activate可使用 pip3 list 查看已安装的包：
$ pip3 list运行 sirmodred，默认参数配置文件可参考 default-grimoirelab-settings 目录，可根据需要修改，建议的 setup.cfg 文件内容如下：
[general]
short_name = GrimoireLab
update = true
min_update_delay = 60
debug = true
logs_dir = /tmp/logs
# Number of items per bulk request to Elasticsearch
bulk_size = 500
# Number of items to get from Elasticsearch when scrolling
scroll_size = 500
menu_file = ./menu.yaml
aliases_file = ./aliases.json
[projects]
projects_file = ./projects.json
[es_collection]
url = https://admin:admin@localhost:9200
[es_enrichment]
url = https://admin:admin@localhost:9200
[sortinghat]
host = 127.0.0.1
user = root
password =
database = demo_sh
load_orgs = true
orgs_file = ./organizations.json
autoprofile = [github, pipermail, git]
matching = [email]
sleep_for = 100
unaffiliated_group = Unknown
affiliate = true
strict_mapping = false
reset_on_load = false
identities_file = [./identities.yml]
identities_format = grimoirelab
[panels]
kibiter_time_from = now-5y
kibiter_default_index = git
kibiter_url = http://admin:admin@localhost:5601
[phases]
collection = true
identities = true
enrichment = true
panels = true
[git]
raw_index = git_demo_raw
enriched_index = git_demo_enriched
latest-items = true
studies = [enrich_demography:git, enrich_areas_of_code:git, enrich_onion:git]
[enrich_demography:git]
[enrich_areas_of_code:git]
in_index = git_demo_raw
out_index = git-aoc_demo_enriched
[enrich_onion:git]
in_index = git
out_index = git-onion_demo_enriched
contribs_field = hash
$ cd ../default-grimoirelab-settings/
# 若将上述配置拷贝至 /default-grimoirelab-settings/setup-secured-source-code.cfg
$ sirmordred -c setup-secured-source-code.cfg大功告成！打开 http://localhost:5601/ 即可看到效果。
- Q：在源码编译安装的时候能不能只指定
--install，不指定--build参数？
A：如不指定 --build 参数，将会从pip源下载包，但pip源上的包不是最新版。指定 --build 参数可以确保安装的是最新版本。
- Q：从 GitHub 拉取仓库速度慢，怎么办？
A：可指定仓库地址文件 reposfile ，切换到从 Gitee 上拉取镜像仓库：
$ ./build_grimoirelab -l debug --logfile /tmp/log --build --install --check \
--relfile ../releases/latest --reposdir /tmp/repos --reposfile repos-from-gitee.json \
--venv /tmp/venv --install_venv /tmp/ivenv其中 repos-from-gitee.json 文件内容见 附录1.
输出结果见 附录2.
- 在默认配置中，指定日志文件为 /tmp/logs/all.log，因此需要确保 /tmp/logs/ 目录存在：
$ mkdir /tmp/logs- Error feeding raw from git (https://github.com/chaoss/grimoirelab-perceval): <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1076)>
- macOS下的常见错误，原因是Python 2.7.9 之后引入了一个新特性，当你使用urllib.urlopen一个 https 的时候会验证一次 SSL证书。当目标使用的是自签名的证书时就会报urllib.error.URLError错误——解决方法
repos-from-gitee.json 文件内容如下：
{
"grimoirelab": [
{
"name": "grimoirelab",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab",
"version_file": "grimoirelab/_version.py"
}
],
"grimoirelab-toolkit": [
{
"name": "grimoirelab-toolkit",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab-toolkit",
"version_file": "grimoirelab-toolkit/_version.py"
}
],
"perceval": [
{
"name": "perceval",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab-perceval",
"version_file": "perceval/_version.py",
"check_run": "perceval --help"
}
],
"perceval-mozilla": [
{
"name": "perceval-mozilla",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab-perceval-mozilla",
"version_file": "setup.py",
"check_run": "perceval kitsune --help"
}
],
"perceval-opnfv": [
{
"name": "perceval-opnfv",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab-perceval-opnfv",
"version_file": "setup.py",
"check_run": "perceval functest --help"
}
],
"perceval-puppet": [
{
"name": "perceval-puppet",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab-perceval-puppet",
"version_file": "setup.py",
"check_run": "perceval puppetforge --help"
}
],
"perceval-finos": [
{
"name": "perceval-finos",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab-perceval-finos",
"version_file": "setup.py",
"check_run": "perceval finosmeetings --help"
}
],
"kingarthur": [
{
"name": "kingarthur",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab-kingarthur",
"version_file": "arthur/_version.py",
"check_run": "arthurd --help"
}
],
"grimoireelk": [
{
"name": "grimoire-elk",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab-elk",
"version_file": "grimoire_elk/_version.py",
"check_run": "p2o.py --help"
}
],
"sortinghat": [
{
"name": "sortinghat",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab-sortinghat",
"version_file": "sortinghat/_version.py",
"check_run": "sortinghat --help"
}
],
"kidash": [
{
"name": "kidash",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab-kidash",
"version_file": "kidash/_version.py",
"check_run": "kidash --help"
}
],
"sigils": [
{
"name": "grimoirelab-panels",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab-sigils",
"version_file": "panels/_version.py"
}
],
"manuscripts": [
{
"name": "manuscripts",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab-manuscripts",
"version_file": "manuscripts/_version.py",
"check_run": "manuscripts --help"
}
],
"sirmordred": [
{
"name": "sirmordred",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab-sirmordred",
"version_file": "sirmordred/_version.py",
"check_run": "sirmordred --help"
}
],
"cereslib": [
{
"name": "cereslib",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab-cereslib",
"version_file": "cereslib/_version.py"
}
],
"graal": [
{
"name": "graal",
"dir": "",
"repo_url": "https://gitee.com/heming6666/grimoirelab-graal",
"version_file": "graal/_version.py",
"check_run": "graal --help"
}
]
}$ git clone https://github.com/chaoss/grimoirelab.git
$ cd grimoirelab/utils
$ ./build_grimoirelab -l debug --logfile /tmp/log --build --install --check \
--relfile ../releases/latest --reposdir /tmp/repos --venv /tmp/venv --install_venv /tmp/ivenv以上命令输出为：
GrimoireLab release: 0.2.43
Building...
Built module grimoirelab: OK
Built module grimoirelab-toolkit: OK
Built module perceval: OK
Built module perceval-mozilla: OK
Built module perceval-opnfv: OK
Built module perceval-puppet: OK
Built module perceval-finos: OK
Built module kingarthur: OK
Built module grimoire-elk: OK
Built module sortinghat: OK
Built module kidash: OK
Built module grimoirelab-panels: OK
Built module manuscripts: OK
Built module sirmordred: OK
Built module cereslib: OK
Built module graal: OK
Installing...
Installed modules: grimoirelab,grimoirelab-toolkit,perceval,perceval-mozilla,perceval-opnfv,perceval-puppet,perceval-finos,kingarthur,grimoire-elk,sortinghat,kidash,grimoirelab-panels,manuscripts,sirmordred,cereslib,graal
Checking...
Checking versions...
Checked version for grimoirelab: OK
Checked version for grimoirelab-toolkit: Fail
Checked version for perceval: OK
Checked version for perceval-mozilla: OK
Checked version for perceval-opnfv: OK
Checked version for perceval-puppet: OK
Checked version for perceval-finos: OK
Checked version for kingarthur: OK
Checked version for grimoire-elk: OK
Checked version for sortinghat: OK
Checked version for kidash: OK
Checked version for grimoirelab-panels: OK
Checked version for manuscripts: OK
Checked version for sirmordred: OK
Checked version for cereslib: OK
Checked version for graal: OK
Checking installable (all modules at once)...OK
Checking installable (grimoirelab)...OK
Checking installable (grimoirelab-toolkit)...OK
Checking installable (perceval)...OK
Checking run (perceval)...OK
Checking installable (perceval-mozilla)...OK
Checking run (perceval-mozilla)...OK
Checking installable (perceval-opnfv)...OK
Checking run (perceval-opnfv)...OK
Checking installable (perceval-puppet)...OK
Checking run (perceval-puppet)...OK
Checking installable (perceval-finos)...OK
Checking run (perceval-finos)...OK
Checking installable (kingarthur)...OK
Checking run (kingarthur)...OK
Checking installable (grimoire-elk)...OK
Checking run (grimoire-elk)...OK
Checking installable (sortinghat)...OK
Checking run (sortinghat)...OK
Checking installable (kidash)...OK
Checking run (kidash)...OK
Checking installable (grimoirelab-panels)...OK
Checking installable (manuscripts)...OK
Checking run (manuscripts)...Fail
Checking installable (sirmordred)...OK
Checking run (sirmordred)...OK
Checking installable (cereslib)...OK
Checking installable (graal)...OK
Checking run (graal)...OK
Installed modules (all):
Venv for building in /tmp/venv
Repos for source code in /tmp/repos
Distribution packages in /tmp/tmpbkwdsjq9
Installed modules in /tmp/ivenv