Installing X-Pack Monitoring for Elasticsearch 6.1

Elasticsearch 6.1 is EOL as of June 13, 2019. If possible, install Elastic stack 6.5 products.

To monitor Elasticsearch, use X-Pack Monitoring. First install X-Pack onto Elasticsearch and configure security if you’re using X-Pack’s security features. Then come back here for instructions on installing and configuring Kibana (the monitoring server) with X-Pack so that Elasticsearch, Kibana, and Liferay DXP can communicate effortlessly and securely (if you’re using X-Pack Security). A Liferay Enterprise Search subscription is necessary for this integration. Contact Liferay’s Sales department for more information.

1. Download and install Kibana.

2. Install X-Pack onto Kibana and configure Kibana

   If using X-Pack’s security, this includes proper security configuration.

3. Download and install the Liferay Enterprise Search Monitoring [Elastic Stack 6.x].

4. Configure the connector to communicate with Elasticsearch.

The exact steps differ if you’re enabling security and monitoring or just monitoring. Differences in the process are noted as appropriate.

For the X-Pack installation procedure, refer to the X-Pack security article.

This guide starts with the installation of Kibana.

Install Kibana

Make sure to install the correct version of Kibana. Check the Liferay Enterprise Search compatibility matrix for details.

1. Download Kibana and extract it. The root folder is referred to as Kibana Home.

2. Install X-Pack into Kibana:

   ./bin/kibana-plugin install x-pack
   

3. Tell Kibana where to send monitoring data by setting Elasticsearch’s URL in kibana.yml:

   elasticsearch.url: "http://localhost:9200"
   

   If SSL is enabled on Elasticsearch, this is an https URL.

4. If not using X-Pack security, start Kibana by entering

    ./bin/kibana
   

   from Kibana Home.

If you’re using X-Pack’s security features, there’s additional configuration required that you can find below and is related to Securing Elasticsearch 6 with X-Pack.

Configure Kibana with Authentication

If X-Pack requires authentication to access the Elasticsearch cluster, follow these steps or refer to Elastic’s documentation.

1. Set the password for the built-in kibana user in Kibana Home/config/kibana.yml:

   elasticsearch.username: "kibana"
   elasticsearch.password: "liferay"
   

   The password is whatever you set it to when initially setting up X-Pack. Once Kibana is installed, you can change the built-in user passwords from the Management user interface.

2. If you’re not encrypting communication with the Elasticsearch cluster, start Kibana with

   ./bin/kibana
   

   and navigate to localhost:5601. Log in with a user who has the kibana_user role.

Configuring Kibana with Encryption

Follow these steps to configure Kibana if X-Pack encrypts communication with the Elasticsearch cluster. Consult Elastic’s guide for more information.

Add these settings to kibana.yml:

xpack.security.encryptionKey: "xsomethingxatxleastx32xcharactersx"
xpack.security.sessionTimeout: 600000

elasticsearch.ssl.verificationMode: certificate
elasticsearch.url: "https://localhost:9200"
elasticsearch.ssl.certificateAuthorities: [ "/path/to/ca.crt" ]

server.ssl.enabled: true
server.ssl.certificate: /path/to/[Elasticsearch Home]/config/localhost.crt
server.ssl.key: /path/to/[Elasticsearch Home]/config/localhost.key

For more information about monitoring and security best practices in a clustered environment, refer to Elastic’s documentation.

After this step you can access Kibana at https://localhost:5601 and sign in with a Kibana user. The last step is to hook Kibana up with Liferay DXP.

Configuring the Liferay Enterprise Search Monitoring app

If you have a Liferay Enterprise Search subscription, download the Liferay Enterprise Search Monitoring. Install the LPKG file by copying it into the Liferay Home/deploy folder. That’s all there is to it.

1. Once the connector is installed and Kibana and Elasticsearch are securely configured, create a configuration file named

   com.liferay.portal.search.elasticsearch6.xpack.monitoring.web.internal.configuration.XPackMonitoringConfiguration.config
   

2. Place these settings in the .config file:

   kibanaPassword="liferay"
   kibanaUserName="elastic"
   kibanaURL="http://localhost:5601"
   

   Alternatively, configure the monitoring adapter from the Control Panel. Navigate to Configuration → System Settings and find the X-Pack Monitoring entry in the Foundation category. All the configuration options for the monitoring connector appear there.

   The values differ depending on your Kibana configuration. For example, kibanaURL="https://localhost:5601" if using X-Pack Security features.

3. Deploy this configuration file to Liferay Home/osgi/configs, and the settings are picked up by your running instance. There’s no need to restart the server.

4. There’s one more setting to add to Kibana itself. It sets Kibana’s base path to let the Monitoring Portlet act as a proxy for Kibana’s monitoring UI. Add this to kibana.yml:

   server.basePath: "/o/portal-search-elasticsearch-xpack-monitoring/xpack-monitoring-proxy"
   

   Note that once you set the server.basePath, you cannot access the Kibana UI through Kibana’s URL (for example, https://localhost:5601). All access to the Kibana UI is via the monitoring portlet, which is only accessible to logged in Liferay DXP users. Navigate directly to the portlet using this URL: http://localhost:8080/o/portal-search-elasticsearch-xpack-monitoring/xpack-monitoring-proxy/app/monitoring

5. Because you’re using the Monitoring portlet in Liferay DXP as a proxy to Kibana’s UI, if you are using X-Pack Security, you must configure the application server’s startup JVM parameters to recognize a valid truststore and password.

   First, navigate to Elasticsearch Home and generate a PKSC#12 certificate from the CA you created when setting up X-Pack security:

   ./bin/x-pack/certutil cert --ca-cert /path/to/ca.crt --ca-key /path/to/ca.key --ip 127.0.0.1 --dns localhost --name localhost --out /path/to/Elasticsearch_Home/config/localhost.p12
   

   Next use the keytool command to generate a truststore:

   keytool -importkeystore -deststorepass liferay -destkeystore /path/to/truststore.jks -srckeystore /path/to/Elasticsearch_Home/config/localhost.p12 -srcstoretype PKCS12 -srcstorepass liferay
   

   Add the trustore path and password to your application server’s startup JVM parameters. For a Tomcat server, append this to your existing CATALINA_OPTS:

   -Djavax.net.ssl.trustStore=/path/to/truststore.jks -Djavax.net.ssl.trustStorePassword=liferay
   

Restart Liferay DXP and Kibana.

---

header-id: monitoring-in-product

Monitoring in Liferay DXP

Once Kibana and X-Pack are successfully installed and configured and all the servers are up and running, add the X-Pack Monitoring portlet to a page:

1. Open the Add menu on a page and choose Applications

2. Search for monitoring and drag the X-Pack Monitoring application from the Search category onto the page.

See the Elastic documentation for information on monitoring Elasticsearch and monitoring production systems.