Different from user-defined decompilers, custom decompilers are Java classes that implement the ICustomDecompiler interface.

Steps

First

Create a Java class, then implements ICustomDecompiler. The custom decompiler is executed by the decompile method. Write your decompile logic in the decompile method.
The name method returns a unique name to let MinecraftDecompiler know your custom decompiler name.
The getSourceType method defines the input type your customized decompiler can accept.

ILibRecommendedDecompiler

IExternalResourcesDecompiler

Second

Build your class files and put them in a jar.
Put a file named cn.maxpixel.mcdecompiler.decompiler.ICustomizedDecompiler in the location META-INF/services of the jar. Then write the FQCN of your custom decompiler class that implements this interface in it.

Third

Run MinecraftDecompiler with required options and "-d --customDecompiler <your custom decompiler name returned by name method> --customDecompilerJars <your jar location>" option