OutOfMemoryError on startup when there are a lot of very large FragmentEntryLinks

Issue

• Liferay takes a very long time to startup, and throws the following Heap Space error:

ERROR [Framework Event Dispatcher: Equinox Container: d97119a4-f960-4e1c-839b-88c78fbc6cc6][Framework:93] FrameworkEvent ERROR 
java.lang.OutOfMemoryError: Java heap space
at java.base/java.util.Arrays.copyOf(Arrays.java:3745)
at java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:172)
at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:686)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:232)
at org.hibernate.type.StringClobType.nullSafeGet(StringClobType.java:69)
at org.hibernate.type.CustomType.nullSafeGet(CustomType.java:109)
at org.hibernate.type.AbstractType.hydrate(AbstractType.java:104)
at org.hibernate.persister.entity.AbstractEntityPersister.hydrate(AbstractEntityPersister.java:2283)
at org.hibernate.loader.Loader.loadFromResultSet(Loader.java:1527)
at org.hibernate.loader.Loader.instanceNotYetLoaded(Loader.java:1455)
at org.hibernate.loader.Loader.getRow(Loader.java:1355)
at org.hibernate.loader.Loader.getRowFromResultSet(Loader.java:611)
at org.hibernate.loader.Loader.doQuery(Loader.java:829)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:274)
at org.hibernate.loader.Loader.doList(Loader.java:2542)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2276)
at org.hibernate.loader.Loader.list(Loader.java:2271)
at org.hibernate.hql.classic.QueryTranslatorImpl.list(QueryTranslatorImpl.java:940)
at org.hibernate.engine.query.HQLQueryPlan.performList(HQLQueryPlan.java:196)
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1268)
at org.hibernate.impl.QueryImpl.list(QueryImpl.java:102)
at com.liferay.portal.dao.orm.hibernate.QueryImpl.list(QueryImpl.java:112)
at com.liferay.portal.dao.orm.hibernate.QueryImpl.list(QueryImpl.java:104)
at com.liferay.portal.kernel.dao.orm.QueryUtil.list(QueryUtil.java:61)
at com.liferay.portal.kernel.dao.orm.QueryUtil.list(QueryUtil.java:53)
at com.liferay.fragment.service.persistence.impl.FragmentEntryLinkPersistenceImpl.findByRendererKey(FragmentEntryLinkPersistenceImpl.java:2670)
at com.liferay.fragment.service.persistence.impl.FragmentEntryLinkPersistenceImpl.findByRendererKey(FragmentEntryLinkPersistenceImpl.java:2560)
at com.liferay.fragment.service.persistence.impl.FragmentEntryLinkPersistenceImpl.findByRendererKey(FragmentEntryLinkPersistenceImpl.java:2519)
at com.liferay.fragment.service.impl.FragmentEntryLinkLocalServiceImpl.getFragmentEntryLinks(FragmentEntryLinkLocalServiceImpl.java:385)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

• There are tens of thousands of large fragments on content pages across Liferay.

Environment

• DXP 7.2
• DXP 7.3
• DXP 7.4

Resolution

• This issue is resolved by LPS-177286.
• Open a help center ticket to request a hotfix at your patch level containing  LPS-168582, LPS-177987, LPS-178527
• Once the hotfix is installed, create a file in the osgi/configs folder named com.liferay.fragment.configuration.FragmentServiceConfiguration.config with the following content:

  propagateContributedFragmentChanges=B"false" 

Additional Information

• There is a procedure during startup to update all the fragment entry links by fetching all the fragment entry links belonging to a given fragment entry into memory and then iterating over them and running the update logic on each one.
• However, the logic doesn't scale well for large portal instances that have many detailed content pages, since it fetches every fragment link into memory at once.
• LPS-177286 allows an admin to enable/disable the propagation of changes for the Contributed Fragment on the startup