Conan - qtwebkit/qtwebkit GitHub Wiki

Conan (https://conan.io/) is a package manager that we are using on Windows to install 3rd party libraries in Coin.

Maintenance dashboard

Package Coin version Bintray version Latest version
icu 65.1 Bintray Repology
libxml2 2.9.10 Bintray Repology
libxslt 1.1.34 Bintray Repology
libjpeg-turbo 2.0.5 Bintray Repology
zlib 1.2.11 Bintray Repology
libgpg-error - Bintray Repology
libgcrypt - Bintray Repology
libtasn1 - Bintray Repology
brotli - Bintray Repology
woff2 - Bintray Repology

Overview

  1. For each dependency we have a separate git repository with conan recipe
  2. Conan builds binary packages from this recipe. We need to build separate package for every compiler version which is used in Coin to build Qt modules
  3. Conan uploads all binary packages to Bintray repository
  4. Coin, when doing provisioning for each Windows VM image, executes Conan and downloads appropriate packages, required by particular conanfile. Conan Manifests (checksum files) are used to ensure integrity of downloaded data
  5. Now builds of QtWebKit can use these libraries

Conan recipes

Use links in dasboard above.

Note that libxslt recipe is original, and libxml2 configuration is heavily customized.

Compilers and versions

We need to match https://wiki.qt.io/Qt_5.13_Tools_and_Versions

For Qt 5.13:

  • MSVC2015 (x86_64)
  • MSVC2017 (x86, x86_64)
  • MinGW 7.3 (i686-7.3.0-release-posix-dwarf-rt_v5-rev0, x86_64-7.3.0-release-posix-seh-rt_v5-rev0)

Bintray Repository

https://bintray.com/qtproject/conan/

Conan in Coin

All files mentioned below are extracted to https://github.com/qtwebkit/coin-provisioning. AppVeyor is used to test changes before submitting them into qt5, and to generate manifest updates (produced as build artifact).

Installation and setup of Conan

https://github.com/qt/qt5/blob/5.13/coin/provisioning/common/windows/03-conan.ps1#L3-L8

Conanfiles

These determine which Conan packages are installed into Coin's VMs

Where these conanfiles are used

https://github.com/qt/qt5/blob/5.13/coin/provisioning/common/windows/03-conan.ps1#L43-L45

Configurations:

Manifests

Manifests are used to compare checksums of all files in a package with reference.

https://github.com/qt/qt5/tree/5.13/coin/provisioning/common/windows/conan_manifests

When Conan builds binary package, it produces manifests, which should be copied to qt5 repository. To update package in coin we should change required package versions in all conan files, and also update all mainfests.

Updating packages

  • Make necessary changes in conan-* repositories
  • Add new tag in each repository (otherwise CI doesn't upload resulting packages)
  • If necessary create new branch in coin-provisioning:
    • E.g. from 5.14 branch git checkout -b 5.15
    • Remove "provisioning" patches which were upstreamed to qt5
    • Run sync_code.sh and sync_conanfiles.sh
    • cherry-pick MODIFICATION patch
  • Change package versions in all qtwebkit.txt files
  • Update generate_conan_wiki.pl and regenerate table for this wiki page
  • Push changes to coin-provisioning
  • If everything is ok, make tag and push it
  • Download new manifests from Appveyor and replace old manifests in qt5 repo with new ones

Troubleshoot: provisioning errors in Coin

If provisioning error happens, it's likely caused by one of two reasons:

  • Package in Bintray was overwritten and checksums in manifests don't match, or
  • there is an error when installing Conan from pip

If package was written, quick fix is to download matching archive from https://bintray.com/qtproject/coin-provisioning, extract it's contents to ~/.conan/data and ... (describe steps, or better make a script which installs conan to virtualenv and reuploads)