Creating a logstash keystore

I want us to explore how to install and configure Elasticsearch and Kibana, to get a cluster ready for ingesting logs for any server. I will be using Rocky Linux (Redhat family) for this setup, and installing Elasticsearch and Kibana as RPM packages.

Elasticsearch is an open-source search and analytics engine. It is a free tool to store and quickly search data, manage and view it with Kibana, with a lot of features such as: indexing several types of data from different appliances, Data Analytics and Observability, Machine Learning, SIEM, Endpoint Security and many more. A cluster consists of a number of nodes, but for the purpose of this setup, I will be configuring a one-node cluster in addition to one Kibana instance.

Installing Elasticsearch

The installation on this article is based on RHEL version 8, you can use yum instead of dnf.

Switch to superuser

sudo su

Update applications installed on a system

dnf update

Install vim editor - the colored editor may indicate errors in the config files

dnf -y install vim

Install java. You can specify the version and release you desire.

dnf -y install java

Import the elasticsearch GnuPG key

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Create a file called elasticsearch.repo in the /etc/yum.repos.d/ directory with the command

vim /etc/yum.repos.d/elasticsearch.repo

and paste the following then save the file:

[elasticsearch]
name=Elasticsearch repository for  8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=0
autorefresh=1
type=rpm-md

Install Elasticsearch

dnf install --enablerepo=elasticsearch elasticsearch
systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl start elasticsearch.service

Change the “elastic” user password:

/usr/share/elasticsearch/bin/elasticsearch-reset-password -i -u elastic

Use -i for interactive, which allows you to type in your own password

To check if the node is running, run the following command:

curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic https://localhost:9200

Configuring Elasticsearch

The configuration file is in /etc/elasticsearch/

vim /etc/elasticsearch/elasticsearch.yml

Use the following settings to set up the node:

node.name: node1
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
# IP of the Elasticsearch node:
network.host: 192.168.25.100
http.port: 9200

Configure the firewall on this machine, allow communication to port 9200/tcp:

firewall-cmd --add-port=9200/tcp --permanent
firewall-cmd --reload

Installing Kibana

Kibana is the open-source user interface tool used to view the stored information in Elasticsearch and manage the cluster. It has many types of visualizations that help you analyze the data and make sense of what the it looks like. In this setup, Kibana will let us look at the Syslogs coming from the firewall by connecting to the Elasticsearch node and sending queries to it in the background.

On the Kibana VM, run the following commands:

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Create the following file and copy the content below to it:

vim /etc/yum.repos.d/kibana.repo

[kibana-8.x]
name=Kibana repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

Run the following command to start installing Kibana:

sudo dnf install kibana

Enable and start the Kibana service:

/bin/systemctl daemon-reload
/bin/systemctl enable kibana.service
systemctl start kibana.service

Configuring Kibana

The configuration file is in /etc/kibana/

vim /etc/kibana/kibana.yml

server.port: 5601
# IP for the Kibana server
server.host: “192.168.25.120”
# remove a warning at the login page:
server.publicBaseUrl: “http://192.168.25.120:5601”
server.name: “kibana”
# use https
elasticsearch.hosts: [“https://192.168.25.100:9200”]
elasticsearch.username: “kibana_system”
elasticsearch.password: “<password>”
# Copy http_ca.crt from the Elasticsearch node:
elasticsearch.ssl.certificateAuthorities: [ “/etc/kibana/certs/http_ca.crt” ]

From the Elasticsearch node, copy the CA certificate to Kibana:

scp /etc/elasticsearch/certs/http_ca.crt root@192.168.25.120:/etc/kibana/certs

Also, run the following command to change the password for the “kibana_system” built-in user:

/usr/share/elasticsearch/bin/elasticsearch-reset-password -i -u kibana_system

Use the -i option for interactive, which lets you choose your own password

Finally, allow connections to port 5601 in Kibana:

firewall-cmd --add-port=5601/tcp --permanent
firewall-cmd --reload

At this point, the Elasticsearch node and the Kibana instance are ready. After changes to the .yml files, you should restart the service using “systemctl restart elasticsearch.service”, then checking the status to see if there are any errors or issues.

You can browse to the Kibana IP address and port combination to log in to the Kibana user interface

Use the “elastic” built-in user and the password you set after the Elasticsearch installation to log in.

In this blog post, we talked about how to install and configure Elasticsearch and Kibana.