Here in this section we are setting the elasticsearch host and port to which we will be sending the log data captured by filebeat for indexing. Here we are setting two new fields named bu and env which will be root level fields added globally to all the data captured. ‘exclude_files’ will ignore all the files ending with. Also, you can see that i am setting a field named log_type which will be the root level fields added additionally to the log data captured. You can edit this value as per your logs size. ‘harvester_buffer_size’ is the size in bytes of the buffer that each harvester uses when fetching a file. Filebeat starts a harvester for each file that it finds under the specified paths. We are enabling the filebeat inputs section. Here are the details about the configuration file which i am using. Once the backup is completed, lets update the filebeat.yml with the below content. ~]$ cp /etc/filebeat/filebeat.yml /etc/filebeat/filebeat.yml_original Take backup of the existing filebeat.yml file before making any modification to the configuration. ~]$ sudo systemctl enable ~]$ sudo systemctl start rvice Step5: Configure filebeat to capture logs from nginx service Once the kibana service is installed, you can enable and start the service as shown below. We will be using kibana to visualize the indexed nginx log data. Here we will now install kibana service from the fedora repositories. ~]$ sudo systemctl enable ~]$ sudo systemctl start rvice Step4: Install Kibana Once the elasticsearch service is installed, you can enable and start the service as shown below. We will be using elasticsearch to index the nginx log data. Here we will now install elasticsearch service from the fedora repositories. The standard installation of filebeat has its default configuration file at location ‘/etc/filebeat/filebeat.yml’ and the logging location for this service is available at ‘/var/log/filebeat/filebeat’. ~]$ sudo systemctl enable ~]$ sudo systemctl start rvice Once the filebeat service is installed, you can enable and start the service as shown below. Here i am installing the latest available version of filebeat i.e v7.14.1. Now that we have our source of data i.e nginx service up and running, in this step we will be setting up the filebeat service by installing it from the fedora repositories. Once the package is installed, you can enable and start the nginx service and validate it by requesting the default nginx static content page by hitting the URL as below. The standard installation of nginx comes with a default ‘/etc/nginx/nf’ which serves static content from ‘/usr/share/nginx/html’ and writes the access and error logs at the following location ‘/var/log/nginx/access.log’ and ‘/var/log/nginx/error.log’. Here in this step we will be installing the nginx service from the fedora repository. Procedure Step1: Install and Configure Nginx to serve static content Here is the YouTube video on the same step by step procedure shown below. If you are interested in watching the video. We will be carrying out this activity using filebeat inputs approach available in the tool. Here in this article we will will try to capture the access logs from nginx service using the filebeat service and send it to elasticsearch service for indexing the data. ‘libbeat’ is the library which can be used to write custom beat. Metricsbeat, Filebeat and Packetbeat are some of the beats available. THey reads data, parses it and ships it to either elasticsearch or logstash. These are lightweight and are installed as agents. Also we use kibana to visualise and generates graphs and charts for the data that is indexed. Its used to reads/query data from elasticsearch indices using its API’s. This tool is based on the Apache Lucene search engine. It provides tools to query, access and aggregate the data using the API’s. Step8: Validate the indexed data in kibanaįedora 32 installed What is ElasticsearchĮlasticsearch helps in indexing the data read from Logstash.Step7: Restart the elasticsearch service.Step5: Configure filebeat to capture logs from nginx service.Step1: Install and Configure Nginx to serve static content.ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |