glassfish memory allocation - AdvancedPhotonSource/ComponentDB GitHub Wiki
Edit the setting in file [support-dir]/netbeans/payara/glassfish/domains/domain1/config/domain.xml
"<jvm-options>-Xmx4096m</jvm-options>". The default value seems to be 512m, change it to something like 4096m.