Development:Upstreams - cu-uis/cu-starterkit-project GitHub Wiki

Each site managed by UIS with this upstream that requires additional modules, custom theming or configuration will need a site specific project defined. This can be done with just the Pantheon development instance, but most system projects use GitLab with a Pipeline to deploy commits and merges to the master branch of the Gitlab repository which is then deployed to the master branch of the Pantheon development instance. The Pipeline configuration for online.cu.edu is a variation on what Pantheon's Andrew Taylor documented at https://about.gitlab.com/2019/03/26/connecting-gitlab-and-pantheon-streamline-wordpress-drupal-workflows.

In a production environment, Upstreams can only be added by Organization Admins, but Pantheon allows anyone to create a free, EDU Organization for testing purposes. Once your Organization has been created, follow the steps for adding the CU Starter Kit as an option for new sandbox sites in this Organization.

A local development environment and site specific repository may not be a requirement for new sites using the CU Starter Kit upstream in the future, but the current upstream is just a starting point that saves developers time reconfiguring the same features for every site. The sites managed by UIS that use the CU Starter Kit all extend it using DDEV and GitLab Pipelines, but the same functionality can be found in Lando and GitHub or the Acquia Cloud IDE and Cloud Pipelines.

Not all changes made to the Upstream will require running composer install or generating a local site directly from the Upstream using DDEV, but if you do need to do that you need to make changes to your local git configuration so that specific files are not modified in the upstream.

NEVER commit a composer.lock to the Upstream. NEVER add the composer.lock to the .gitignore.

Pantheon updated https://pantheon.io/docs/integrated-composer in December to make it clear that composer.lock should NEVER be part of the upstream. Because a composer.lock will be created by each site, including in the upstream will result in constant merge conflicts. Unfortunately the composer.lock can not be added to the .gitignore either as that causes a different type of error during the build.

Instead, add the composer.lock to .git/info/exclude.

1. Open .git/info/exclude
2. Add

# Ignore composer.lock to avoid site level conflicts #
composer.lock

3. Save the .git/info/exclude

4. If composer.lock is tracked by git before excluding it, use the git command:git update-index --assume-unchanged composer.lock

This has to be done for each project as it isn't committed to the repo like changes to the .gitignore.

The same local git repository can be used to synchronize code with GitLab, Pantheon and GitHub by adding multiple remotes. In most cases the origin remote will be Pantheon, but when using a more complicated workflow the origin could be GitLab. In that use case, adding remotes for GitHub and Pantheon will allow you to push changes to the Upstream or a Multidev instance.

git remote add pantheon ssh://codeserver.dev.afba1bef-c07f-4e0b-ace7-496917014603@codeserver.dev.afba1bef-c07f-4e0b-ace7-496917014603.drush.in:2222/~/repository.git

git remote add github [email protected]:cu-uis/cu-starterkit-project.git

Testing a Change on a Multidev WITHOUT committing it to the Upstream or project specific repo first.

1. Create a Multidev instance cloning it from dev, test or live.
2. Run git fetch --all. Depending on the number of new branches in the remotes, you could see more information, but it will include something like...

Fetching origin
Fetching pantheon
remote: Counting objects: 1, done.
remote: Total 1 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (1/1), done.
From ssh://codeserver.dev.afba1bef-c07f-4e0b-ace7-496917014603.drush.in:2222/~/repository
 * [new branch]      938-lpv2            -> pantheon/938-lpv2
 * [new tag]         pantheon_938-lpv2_1 -> pantheon_938-lpv2_1

3. git checkout -t pantheon/938-lpv2
4. MAKE CHANGES to root level, project composer.json and rerun composer install or composer update --lock
5. git add composer.* (to add composer.json and composer.lock)
6. git commit -m "unpinning drupal/core to allow update to 9.3.8"
7. git push pantheon 938-lpv2

Ensure the Pantheon Autopilot updates have been applied, through the Pantheon UI for Dev.

1. git checkout master
2. git fetch
3. git pull
4. git checkout -b 123-m
5. git pull pantheon master
6. git push --set-upstream origin 123-m
7. Create Merge Request in Gitlab and double check it has all the Autopilot commits.
8. Merge the request.
9. Check pantheon Dev to ensure the merge was successful and there are no more Autopilot updates.

In most cases, the recommended/supported version of Drupal core will automatically be applied to any sites using the upstream. There are cases where you don't want sites moving to that version like when not all contrib dependencies are compatible with a major core version update like 9.3.0. In that case, we want to keep the sites using 9.2.10.

Pinning Core to a Specific Version:

1. in the upstream project cd upstream-configuration
2. Use composer update drupal/core "drupal/core-*" --with-all-dependencies
3. Commit the updates to the /upstream-configuration/composer.json
4. Push the changes to master (or create a branch and merge to master)
5. Apply the upstream changes to each site using the Pantheon Dashboard or terminus upstream:updates:apply --updatedb --accept-upstream -- <site>.<env>

Overriding the Pinned version of Core at the Site Level:

In https://pantheon.io/docs/integrated-composer#how-to-add-dependencies-to-your-upstream, Pantheon recommends using composer require drupal/pkg-name --no-update to add dependencies to the upstream-configuration/composer.json. There are a number of issues with this approach.

1. Minimum Stability - This is a root-only level item in the Composer schema. If left undefined at the upstream-configuration level, Composer will default to stable. Composer will NOT look back to the composer.json in the project root. If the root and upstream do not match, it will result in build errors when composer install is run by the Pantheon service user when building the dev instance.
2. Repositories - [This is another root-only level item in the Composer schema][https://getcomposer.org/doc/04-schema.md#repositories]. This limits the use of Composer when declaring drupal-library items. Pantheon does not support the use of wikimedia/composer-merge-plugin, which is the recommended approach for managing Webform dependencies. For Webform, the result of including the drupal/webform dependencies in the upstream-configuration/composer.json without the recommended drupal-library dependencies is that they are loaded from a CDN. In other cases like cweagans/composer-patches, the repository must declared at the root project for patches to be applied in the upstream.
3. Patches - cweagans/composer-patches will only process patches included in the root. The work around for this is to declare (an external patch file in the root that exists in the upstream-configuration)[https://github.com/cweagans/composer-patches#using-an-external-patch-file].
4. Composer Does NOT Recognize Changes in upstream-configuration/composer.json - Pantheon recommends changing the version number of force an update.

When adding or updating a dependency in the CU Starter Kit upstream using composer require drupal/pkg-name --no-update, our recommended process is to immediately test the build by running composer clearcache, deleting the /composer.lock at the project root and re-running composer install. This will force Composer to attempt to resolve any changes to custom repositories, flag any dependency conflicts and attempt to re-apply any patches to the current dependency version.

1. Problem: Composer still doesn’t acknowledge that changes have been made to the that to the upstream-configuration/composer.json after running composer clearcache. Solution: Delete the composer.lock and rerun composer install
2. Problem: Composer exhausts available memory trying to resolve dependencies. This happens less often with Composer 2, but when Composer is trying to evaluate a large project with dozens of dependencies... and dependencies or dependencies... in multiple composer.json files, it is more likely to happen Solution: Run COMPOSER_MEMORY_LIMIT=-1 composer install

Normally Composer dependencies are added as the current version or later. If the current recommended release of Webform is 6.1.3, Drupal.org will recommend adding it with composer require 'drupal/webform:^6.1. This will add the supported version of Webform as long as it is greater than 6.1.0. It will also allow adding version 6.2.0 when that becomes the supported version because 6.2.0 is greater than 6.1.

Because the composer dependencies are split between the root, project level composer.json and the upstream-configuration/composer.json, the versions in the these files must be compatible. If the upstream is pinned to a specific version or only allows a version range like >=1.0 <2.0 and the project level composer.json tries to add 2.1.0, Composer will fail on the install because a compatible version of the dependencies that meets both requirements cannot be found.

In general, we should try to only use greater than dependencies in the upstream. Pinning to specific version can be necessary when dealing with a bug or requiring a patch, but we should try to unpin those as quickly as possible.