Client side AB Testing - guardian/support-frontend GitHub Wiki

In this section we will go through the steps and considerations that you must have when you want to set up a new test.

Psst! you can also set up server-side AB tests by following this guide.

AB Test Definitions

The abtestDefinitions module contains definitions for client side AB tests. These definitions include information such as targeting (eg. should the test be targeting specific URLs or Regions), amount of cohorts included, audience size of test and whether a test is enabled. Aside: do not prefix your test name with ab as this is stripped during the data pipeline and causes problems with analytics later.

AB Test Participation Allocation

The logic to allocate users to AB Test Participations sits in the abtest.ts module, this module exports the following methods:

init()

This function is called on page initialisation and returns active test participations for a user which are then saved in Redux (common.abParticipations) and logged in Ophan / Quantum Metric. AB participations are allocated by executing the following steps:

1. Read the MVT (multi variant testing) id from the users's cookie. If the user has no mvt in his or her browser, it generates a new MVTid and saves it in the cookie.

2. From the MVT generates a participations object. The steps to build that object are to combine test participations from tests defined in abtestDefinitions, tests from the URL and tests from server-side:

• Read tests from abtestDefinitions
  • Iterate over the tests object from abtestDefinitions and check whether each test is runnable.
    • A test can be marked as referrerControlled in it's abtestDefinition. If a test has referrerControlled set to true we will only assign a user to the test if the acquisitionData contains a reference to a corresponding test. Matching is done on test name.
  • If a test is runnable assign the user to a variant based on the value of their mvtId.
• Read tests from URL
  • A user can force themselves into a test / variant via a url fragment using the form: #ab-[name_of_test]=[variant_name].
• Read tests from server-side
  • Read window.guardian.serversideTests allocations.

getAmountsTestVariant()

Like init() this function is also called on page initialisation on pages where Amounts tests are applicable (eg. the Support checkout) and returns active Amounts test participations for a user which are then saved in Redux (common.abParticipations) and logged in Ophan / Quantum Metric.

window.guardian.settings.amounts contains configuration data related to RRCP configured “Amounts” tests. Amounts tests define what choice cards are rendered on the Support checkout.

Amounts test participations are allocated by executing the following steps:

• Check URL for amounts test participation
  • Check if the acquisitionData contains a reference to an amounts test participation, and the corresponding test and cohort is available in the window.guardian.settings.amounts, if a match is found select that test / cohort
• Query window.guardian.settings.amounts for a matching targeted test
  • Amounts tests can be targeted by Country OR Region, search for a Country match first, if a match is found select that test / cohort otherwise search fora Region targeted test.
• Fallback to default amounts
  • If we couldn't match an Amounts test on Country OR Region we fallback to the default amounts, which are defined in code and not in the RRCP.

Reading AB Test Participations

AB Test participations are stored in Redux state once a user has been allocated to a test / cohort. You can read these participations as so:

const { abParticipations } = useContributionsSelector(
    (state) => state.common,
);

const isUserInSupporterPlusOnlyVariant = abParticipations?.supporterPlusOnly === 'variant';