Patch Management Best Practices

The principles below are adopted from general industry standard patch management best practices. Following these will help mitigate the risk of change on production servers.

Awareness

• Liferay communicates the availability of maintenance updates for Liferay products through Announcements. If you did not do so already during the onboarding call, stay up to date with the latest by using this subscribe link.
• Keep an updated list of Liferay features that your organization uses so that you can cross-reference this list with the documentation for the maintenance updates. This will allow you to determine when you should evaluate updates for promotion to your production environment.

Mentality

• Maintain stability by treating patches as being as risky as any other change you may make to a production system, whether it’s customizations developed by your own company, new products, or configuration updates.
• Confirm the authenticity of your maintenance updates by checking the MD5 of the file you’ve downloaded against the MD5 listed on the Liferay website. Fix packs and hotfixes are signed by Liferay; their validity is checked by the Patching Tool.
• Apply changes to your production systems only after analyzing whether the risk of introducing a change outweighs the risk of not introducing the change. Along with your organization’s general risk assessment templates, here are a few sample questions that are more specific to Liferay patches:
  • Are any other Liferay changes being deployed alongside the patch?
  • Are any systems (network/infrastructure, databases, integrated services) planning a maintenance update at the same time as the Liferay update?
  • Are the appropriate staff working in the time period of deployment to handle any unexpected effects of the deployment?
  • With DXP, some patches are ineligible for rolling restart updates. In these cases, you must apply the update to the entire cluster at the same time; rolling restart updates will not be supported. Do your uptime requirements allow you to create a maintenance window, or will you need to establish and validate a process for a zero downtime deployment? For information on which patches are eligible, see here.

Testing

• Create a test environment that mirrors your production deployment of Liferay (customizations, data, configurations, third party technologies, and etc should be the same). Create a schedule to sync the test environment to the production environment.
• Confirm that patches are successfully applied in this test environment.
• Create automated and manual QA tests for the test environment that reflect the use cases and load intended on the Liferay deployment.
• Use the automated tests to test maintenance updates before deploying the update to production. Make sure the update doesn't interfere with your customizations of Liferay features.

Backup

• Develop a disaster recovery plan that is understood by all relevant stakeholders and ready to be executed in the event of unexpected events.
• Incorporate Backing up a Liferay DXP Installation into your overall disaster recovery plan.
• Create a disaster recovery environment that mirrors the production deployment of Liferay (customizations, data, configurations, third party technologies, and etc should be the same) to periodically validate the plan.
• Test the disaster recovery plan frequently and at least once prior to going live. Doing so will ensure that the disaster recovery plan adapts to the changing nature of your production environment and help you identify common anomalies in the backup process such as:
  • Inaccurate backup plan steps (knowledge base documentation that has moved, other invalid links)
  • Unusually out of date backups (backups from a week ago are missing most of the data from the past month)
  • Data or steps that are missing (missing document library, search index, or data pulled from third party systems)
• After deploying a change to production, monitor the system (and any other systems that Liferay integrates with or pulls data from) closely for potential unexpected responses to the change.

Rollback

• Uninstall a maintenance update if the risk of keeping it on production outweighs the risk of uninstalling it. Make sure to test rollback / uninstalling of a patch.
• Before deploying a patch to production, make a backup of production systems in the event you need to restore from backup when uninstalling the maintenance update.