Elasticsearch nodes are not discoverable

Issue

• Getting the below errors in the elastic search server, and hence the search is not working:
  

  [WARN ][o.e.d.PeerFinder ] [node1] address [10.199.131.143:9302], node [null], requesting [false] connection failed: [][10.199.131.143:9302] connect_timeout[30s]

  [WARN ][o.e.c.c.ClusterFormationFailureHelper] [node1] master not discovered or elected yet, an election requires at least 2 nodes with ids from [h9blXiZTSMGmYC7SUFE5oQ, PKd2J1kxQXy139oAnoj2VA, 9H0oHpXXTZyEGWVxG1zig], have only discovered non-quorum [{node1}{h9blXiZTSMGmYC7SUFE5oQ}{jochDPpQT8asIOSuPzdfOw}{10.199.131.141}{10.199.131.141:9300}{cdfhilmrstw}]; discovery will continue using [10.199.131.142:9301, 10.199.131.143:9302] from hosts providers and [{node1}{h9blXiZTSMGmYC7SUFE5oQ}{jochDPpQT8asIOSuPzdfOw}{10.199.131.141}{10.199.131.141:9300}{cdfhilmrstw}, {node3}{9H0oHpXXTZyEGWVxG1zig}{02G6uCQHSniTqnIsmAkoA}{10.199.131.143}{10.199.131.143:9302}{cdfhilmrstw}, {node2}{PKd2J1kxQXy139oAnoj2VA}{oDTp-5ZrT1Wn6ikn9JV4tQ}{10.199.131.142}{10.199.131.142:9301}{cdfhilmrstw}] from last-known cluster state; node term 11, last-accepted version 9612 in term 11 [2023-04-20T05:23:03,726][WARN ][r.suppressed ] [node1] path: /cluster/health, params: {pretty=}
  org.elasticsearch.discovery.MasterNotDiscoveredException: null
  at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$2.onTimeout(TransportMasterNodeAction.java:297) [elasticsearch-7.17.1.jar:7.17.1]
  at org.elasticsearch.cluster.ClusterStateObserver$ContextPreservingListener.onTimeout(ClusterStateObserver.java:345) [elasticsearch-7.17.1.jar:7.17.1]
  at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:263) [elasticsearch-7.17.1.jar:7.17.1]
  at org.elasticsearch.cluster.service.ClusterApplierService$NotifyTimeout.run(ClusterApplierService.java:660) [elasticsearch-7.17.1.jar:7.17.1]
  at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:718) [elasticsearch-7.17.1.jar:7.17.1]
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
  at java.lang.Thread.run(Thread.java:833) [?:?]

Environment

• Liferay DXP 7.3
• Liferay DXP 7.4

Resolution

• These errors indicate that elastic search nodes are unable to communicate with each other.
• To make sure that each elastic search node is able to communicate with each other on the specified port. The user can confirm the connection by pinging or telneting the other nodes on the server.

Additional Information

• Troubleshooting the Elasticsearch Connection