Changing the total number of fields in the index for Elasticsearch through DXP

This article documents how to increase the maximum number of fields in the Elasticsearch index through the Control Panel in DXP 7.0, 7.1 and 7.2.  The default settings are set to 7500 fields and there is normally no need to exceed 7500 fields.

If you notice in your logs that you are exceeding this upper limit, you should analyze the root cause - whether it is from a custom development or Liferay DXP. At the same time, upgrade to the latest DXP fix pack which can contain the latest fixes to address any performance issues before increasing the maximum number of fields. 

  • If using DXP 7.0, the default connector is version 2.x. For those using Elasticsearch 6.x, you have to install the Elasticsearch 6 connector first.
  • If using DXP 7.2, the default connector is version 6.x. For those using Elasticsearch 7.x, you have to install the Elasticsearch 7 connector first.

Information about the mapping and the appropriate number of fields in the index is be found here: Settings to prevent mapping explosion.

The current maximum number of fields is 7500. This is specified in the property index.mapping.total_fields.limit=7500.

Environment

  • Liferay DXP 7.0, 7.1 and 7.2
  • Elasticsearch 6.x through 7.x

Resolution

To increase the number of fields, this requires two steps: 1) change the settings in the Systems Settings and 2) reindex.

Part 1: Modifying the System Settings.

  1. Navigate to the Control Panel → Configuration → System Settings.
  2. Elasticsearch / Elasticsearch 6 / Elasticsearch 7 (depending on your connector version)
  3. In the Additional Index Configurations, enter the following: index.mapping.total_fields.limit: 10000. (Note: this field is found near the bottom of the page.)
  4. Click the Update button.

At this point, the platform has been configured but a new index is required.

Part 2: Rebuilding the Index

  1. Navigate to the Control Panel → Configuration → Server Administration or Control Panel → Configuration → Search in case of DXP 7.1 and 7.2).
  2. Under the Index Action section, click Execute next to Reindex all search indexes.

Wait for the index to finish rebuilding. During the reindexing process, the console displays something to this:

2018-12-12 18:52:45.267 INFO [elasticsearch[Q][clusterService#updateTask][T#1]][metadata:317] [Q] [liferay-77336] update_mapping [LiferayDocumentType]

2018-12-12 18:52:45.493 INFO [elasticsearch[Q][clusterService#updateTask][T#1]][metadata:317] [Q] [liferay-77336] update_mapping [LiferayDocumentType]

2018-12-12 18:52:45.569 TRACE [elasticsearch[Q][bulk][T#4]][index:158] [Q][liferay-0][0] took[591.7ms], took_millis[591], type[LiferayDocumentType], id[com_liferay_configuration_admin_web_portlet_SystemSettingsPortlet_PORTLET_com.liferay.portal.template.freemarker.configuration.FreeMarkerEngineConfiguration], routing[] , source[{"uid":"com_liferay_configuration_admin_web_portlet_SystemSettingsPortlet_PORTLET_com.liferay.portal.template.freemarker.configuration.FreeMarkerEngineConfiguration","companyId":"0","configurationModelId":"com.liferay.portal.template.freemarker.configuration.FreeMarkerEngineConfiguration","entryClassName":"com.liferay.configuration.admin.web.internal.model.ConfigurationModel","configurationModelFactoryPid":"com.liferay.portal.template.freemarker.configuration.FreeMarkerEngineConfiguration","title_zh_CN":"FreeMarker引擎","title_es_ES":"Motor FreeMarker","title_ja_JP":"FreeMarker エンジン","title_iw_IL":"מנוע FreeMarker","title_nl_NL":"FreeMarker Engine","title_fi_FI":"FreeMarker -moottori","title_ca_ES":"Motor de Freemarker","title_hu_HU":"FreeMarker motor","title_fr_FR":"Moteur FreeMarker","title":"FreeMarker Engine","title_en_US":"FreeMarker Engine","title_pt_BR":"Mecanismo FreeMarker","title_de_DE":"FreeMarker Engine","configurationModelAttributeName":["localized-lookup","resource-modifica]

2018-12-12 18:52:45.612 INFO [elasticsearch[Q][clusterService#updateTask][T#1]][metadata:317] [Q] [liferay-77336] update_mapping [LiferayDocumentType]

2018-12-12 18:52:46.409 INFO [elasticsearch[Q][clusterService#updateTask][T#1]][metadata:317] [Q] [liferay-20116] update_mapping [LiferayDocumentType]

Once the reindex is completed successfully, the maximum number of fields in Elasticsearch has been increased.

Alternatively, to avoid doing a full reindex, you can update this setting on existing indexes through the Settings API of Elasticsearch like this:

PUT liferay-20155/_settings
{
"index.mapping.total_fields.limit": 2000
}

(where liferay-20155 refers to the company ID of your Virtual Instance. If you have multiple indexes, you need to update this setting for each.)

If you are using curl, the command will look something like this (depending on your actual host and port numbers):

curl -X PUT "localhost:9200/liferay-20155/_settings" -H 'Content-Type: application/json' -d
'{
"index.mapping.total_fields.limit": 2000
}'

If your Elasticsearch Stack is using encrypted communication (TLS/SSL) and user authentication, the curl command will be slightly different:

curl -u "elastic:liferay" -k -X PUT "https://localhost:9200/liferay-20155/_settings" -H "Content-Type: application/json" -d
'{
"index.mapping.total_fields.limit": 2000
}'

(where "elastic:liferay" should be the username and password of a user in Elasticsearch X-Pack Security with administrative privileges, like the built-in elastic user.)

Additional Information

这篇文章有帮助吗?
0 人中有 0 人觉得有帮助