Configuring the Liferay Elasticsearch Connector

This document has been updated and ported to Liferay Learn and is no longer maintained here.

For detailed Elasticsearch configuration information, refer to the Elasticsearch documentation.

The name of your Elasticsearch cluster is important. When you’re running Elasticsearch in remote mode, the cluster name is used by Liferay DXP to recognize the Elasticsearch cluster. To learn about setting the Elasticsearch cluster name on the Liferay DXP side, refer below to the section called Configuring the Liferay Elasticsearch Connector.

Note: The http.enabled setting in Elasticsearch corresponds to the httpEnabled setting in the Liferay Connector to Elasticsearch 6 application. As this setting was deprecated in Elasticsearch 6.3, the connector’s corresponding setting is now also deprecated. This setting was only used for configuring the embedded Elasticsearch server, so its deprecation should have minimal impact to production deployments.

Elasticsearch’s configuration files are written in YAML and kept in the [Elasticsearch Home]/config folder. The main configuration file is elasticsearch.yml, used for configuring Elasticsearch modules.

To set the name of the Elasticsearch cluster, open [Elasticsearch Home]/config/elasticsearch.yml and specify

cluster.name: LiferayElasticsearchCluster

Since LiferayElasticsearchCluster is the default name given to the cluster in the Liferay DXP Elasticsearch connector, this works just fine. Of course, you can name your cluster whatever you want (we humbly submit the recommendation clustery_mcclusterface).¹ Configure your node name using the same syntax (setting the node.name property). There’s no client setting for this, it exists only in each Elasticsearch node’s elasticsearch.yml file.

If you’d rather work from the command line than in the configuration file, navigate to Elasticsearch Home and enter

./bin/elasticsearch --cluster.name clustery_mcclusterface --node.name nody_mcnodeface

Feel free to change the node name or the cluster name. Once you configure Elasticsearch to your liking, start it up.

Starting Elasticsearch

Start Elasticsearch by navigating to Elasticsearch Home and typing

./bin/elasticsearch

if you run Linux, or

\bin\elasticsearch.bat

if you run Windows.

To run as a daemon in the background, add the -d switch to either command:

./bin/elasticsearch -d

Once both Elasticsearch and Liferay DXP are installed and running, introduce them to each other.

Configuring the Liferay Elasticsearch Connector

The Elasticsearch connector provides integration between Elasticsearch and the portal. Before configuring the connector, make sure Elasticsearch is running.

There are two ways to configure the connector:

1. Use the System Settings application in the Control Panel.

2. Manually create an OSGi configuration file.

It’s convenient to configure the Elasticsearch connector from System Settings, but this is often only possible during development and testing. If you’re not familiar with System Settings, read about it here. Remember that you can generate configuration files for deployment to other systems by configuring System Settings, and then exporting the .config file with your configuration.

Configuring the Connector in the Control Panel

To configure the Elasticsearch connector from the System Settings application,

1. Start Liferay DXP.

2. Navigate to Control Panel → Configuration → System Settings → Platform.

3. Find the Elasticsearch entry (scroll down and browse to it or use the search box) and click the Actions icon (), then Edit.

   

   Figure 1: Use the System Settings application in Liferay DXP's Control Panel to configure the Elasticsearch connector.

4. Make any edits to the configuration and click Save.

   

   Figure 2: Configure the Elasticsearch connector's settings. Make sure you set the Operation Mode to *Remote*.

Note: If you switch operation modes (EMBEDDED → REMOTE), you must trigger a re-index. Navigate to Control Panel → Configuration → Search, and click Execute next to Reindex all search indexes.

Configuring the Connector with an OSGi .config File

When preparing a system for production deployment, you want to use a repeatable deployment process. Therefore, it’s best to use the OSGi configuration file, where your configuration is maintained in a controlled source.

Follow these steps to configure the Elasticsearch connector using a configuration file:

1. Create the following file:

    [Liferay_Home]/osgi/configs/com.liferay.portal.search.elasticsearch7.configuration.ElasticsearchConfiguration.config
   

   Elasticsearch 6: The name of the .config file for the Elasticsearch 6 connector is com.liferay.portal.search.elasticsearch6.configuration.ElasticsearchConfiguration.config

2. Add configurations to the file, in the format propertyName="Value". For example,

    operationMode="REMOTE"
    # If running Elasticsearch from a different computer:
    #transportAddresses="ip.of.elasticsearch.node:9300"
    # Highly recommended for all non-production usage (e.g., practice, tests, diagnostics):
    #logExceptionsOnly="false"
   

3. Start Liferay DXP or re-index if already running.

As you can see from the System Settings entry for Elasticsearch, there are a lot more configuration options available that help you tune your system for optimal performance.

What follows here are some known good configurations for clustering Elasticsearch. These, however, can’t replace the manual process of tuning, testing under load, and tuning again, so we encourage you to examine the Elasticsearch documentation and go through that process once you have a working configuration.

Configuring a Remote Elasticsearch Host

In production systems Elasticsearch and Liferay DXP are installed on different servers. To make Liferay DXP aware of the Elasticsearch cluster, set

transportAddresses=[IP address of Elasticsearch Node]:9300

Here’s an example that sets the IP address of two nodes in the Elasticsearch cluster:

transportAddresses=["192.168.1.1:9300","192.168.1.2:9300"]

Set this in the Elasticsearch connector’s OSGi configuration file. List as many or as few Elasticsearch nodes in this property as you want. This tells Liferay DXP the IP address or host name where search requests should be sent. If using System Settings, set the value in the Transport Addresses property.

Note: In an Elasticsearch cluster you can list the transport addresses for multiple Elasticsearch nodes as a comma-separated list in the transportAddresses property. If you set only one transport address, Liferay DXP loses contact with Elasticsearch if that node goes down.

On the Elasticsearch side, set the network.host property in your elaticsearch.yml file. This property simultaneously sets both the bind host (the host where Elasticsearch listens for requests) and the publish host (the host name or IP address Elasticsearch uses to communicate with other nodes). See here for more information.

Clustering Elasticsearch in Remote Operation Mode

To cluster Elasticsearch, first set node.max_local_storage_nodes to be something greater than 1. When you run the Elasticsearch start script, a new local storage node is added to the cluster. If you want four nodes running locally, for example, run ./bin/elasticsearch four times. See here for more information.

Configure the number of shards and replicas in the Elasticsearch 6 connector, using the indexNumberOfShards and indexNumberOfReplicas properties to specify the number of primary shards and number of replica shards, respectively. Elasticsearch’s default configuration works for a cluster of up to ten nodes, since the default number of shards is 5 and the default number of replica shards is 1.

Note: Elasticsearch uses the Zen Discovery Module by default, which provides unicast discovery. Additionally, nodes in the cluster communicate using the Transport Module, through TCP. See the Elasticsearch documentation for the available properties (to be set in the elasticsearch.yml file), and the Liferay DXP Elasticsearch connector’s settings for the connector’s available settings.

At a minimum, provide the list of hosts (as host:port) to act as gossip routers during unicast discovery in the elasticsearch.yml:

discovery.zen.ping.unicast.hosts: ["node1.ip.address", "node2.ip.address"]

For example,

discovery.zen.ping.unicast.hosts: ["10.10.10.5", "10.10.10,.5:9305"]

For more information on configuring an Elasticsearch cluster, see the documentation on Elasticsearch Index Settings.

Elasticsearch Connector System Settings, By Operation Mode

Some of the settings available for the Elasticsearch connector are applicable for only one operation mode (REMOTE or EMBEDDED). Refer to the table below:

Connector Setting/Operation Mode	EMBEDDED	REMOTE
clusterName	x	x
operationMode	x	x
indexNamePrefix	x	x
indexNumberOfReplicas*	x	x
indexNumberOfShards*	x	x
bootstrapMlockAll	x	-
logExceptionsOnly	x	x
retryOnConflict	x	x
discoveryZenPingUnicastHostsPort	x	-
networkHost	x	-
networkBindHost	x	-
networkPublishHost	x	-
transportTcpPort	x	-
transportAddresses	-	x
clientTransportSniff	-	x
clientTransportIgnoreClusterName	-	x
clientTransportPingTimeout*	-	x
clientTransportNodesSamplerInterval	-	x
httpEnabled	x	-
httpCORSEnabled	x	-
httpCORSAllowOrigin	x	-
httpCORSConfigurations	x	-
additionalConfigurations	x	x
additionalIndexConfigurations	x	x
additionalTypeMappings	x	x
overrideTypeMappings	x	x

1 This is, of course, a nod to all those fans of Boaty Mcboatface.