Issue
- The sitemap.xml needs to be disabled as user does not want search engines to crawl the non-prod instances
Environment
- Liferay DXP 7.0+
Resolution
-
Please navigate to
Site Builder --> Pages --> Configuration --> Public Page configuration --> Advanced - Remove the sitemap url from the Robots tab.
- Furthermore, user can add a * in Disallow section to not allow any search engine to crawl the website at all.