Destroy facilitates third-party add-ons to itself which explore areas of chemistry that Destroy does not - for example, Destroy does not currently include extensive alkyne chemistry, but if you wanted to make that in an add-on then the way to do that would be through Groups and Generic Reactions. Before making an add-on consult the currently planned features to avoid a conflict.

Groups classes should extend the abstract class Group with themselves as the generic type parameter, (1) include all relevant Atoms as public final attributes, and (2) the only way of setting being in the initiation. Additionally, you must (3) override the method getGroupType(), to return a Group Type. you will define later.

public class CarboxlyicAcidGroup extends Group<CarboxlyicAcidGroup> {
    //(1) all relevant Atoms as public final attributes:
    public final Atom carbon;
    public final Atom carbonylOxygen;
    public final Atom alcoholOxygen;
    public final Atom proton;

    public CarboxlyicAcidGroup(Atom carbon, Atom carbonylOxygen, Atom alcoholOxygen, Atom proton) {
        //(2) attributes are only set in the initiation:
        this.carbon = carbon;
        this.carbonylOxygen = carbonylOxygen;
        this.alcoholOxygen = alcoholOxygen;
        this.proton = proton;
    };

    // (3) Group Type getter
    @Override
    public String getGroupType() {
        return MyModsGroupTypes.CARBOXYLIC_ACID;
    };
};

You will also want to include any methods you might need for informing Reactions - for example, if this was the alcohol group, it would be good to have a method like getOrder() to indicate whether it was primary, secondary or tertiary.

Separately to the group class, you must define Group Types.

public class MyModsGroupTypes {
    
    public static final GroupType<CarboxylicAcidGroup> CARBOXLYIC_ACID = new GroupType<>(() -> MyModsMolecule.EXAMPLE_CARBOXLYIC_ACID);
};

You can see the constructor takes a supplier of a Molecule. This Molecule is used, for example, in JEI for generating the exemplar Generic Reactions applicable to your Group. Obviously the example Molecule should contain the Group (or else an exception will be raised).

The system must be able to identify what constitutes your Group. This is done with a Group Finder. Group Finder objects have a method findGroups() which is called every time a new Molecule is created, or a Molecule is deserialized. The method findGroups() is passed the 'structure' of the Molecule (a Map mapping all Atoms in the Molecule to the list of Bonds each Atom has), and should return a list of Group objects.

You only need one Group Finder, as a single one can handle all your Groups, and can be used to prevent conflicts (for example, two different finders assigning a carbon as both being in a Carboxylic Acid Group and a Carbonyl Group). However you may wish to add more than one Group Finder for cleanliness. Group Finders should extend the abstract class Group Finder. Ensure the super is called in the constructor.

public class MyNewGroupFinder extends GroupFinder {

    public MyNewGroupFinder() {
        super();
    };

    @Override
    public List<? extends Group> findGroups(Map<Atom, Bond> structure) {
        List<Group> groups = new ArrayList<>();
        for (Atom atom : structure.keySet()) {
            //do the logic that determines if an Atom is part of a Group
            groups.add(new CarboxylicAcidGroup(atom, otherAtom1, otherAtom2, otherAtom3);
        };
        return groups;
    };
    
};

To alert Destroy to the fact your Group Finder exists (so it can then use the Group Finder in the circumstances described above), simply instantiate the class (for example, by calling the following register method from your main mod class). Ensure this is done before you register your Molecules.

public static void register() {
    new MyNewGroupFinder();
};

Destroy will handle the rest automatically. Now you've got to grips with custom Groups, you can add custom Generic Reactions.