elk收集tomcat日志
部署tomcat
安装jdk环境
- 安装yum自带的openjdk1.8版本
- 配置JAVA_HOME和PATH
安装tomcat
上传tomcat二进制包并解压；做软链接
[root@es1 ~]# mkdir /apps
[root@es1 apps]# ll
total 10140
-rw-r--r-- 1 root root 10379806 Sep 2 14:36 apache-tomcat-8.5.57.tar.gz
[root@es1 apps]# tar -xf apache-tomcat-8.5.57.tar.gz
[root@es1 apps]# ln -sv apache-tomcat-8.5.57 tomcat
‘tomcat’ -> ‘apache-tomcat-8.5.57’
修改server.xml,修改webapp所在的数据目录
<Host name="localhost" appBase="/data/tomcat/"
unpackWARs="true" autoDeploy="true">
创建单独的数据目录，存放webapp，并定义一个myapp的webapp
[root@es1 apps]# mkdir /data/tomcat/myapp
[root@es1 apps]# echo myapp-v1 > /data/tomcat/myapp/index.html
启动，并访问
[root@es1 apps]# /apps/tomcat/bin/startup.sh
修改tomcat日志格式为json
1、修改日志格式
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="tomcat_access_log" suffix=".log"
pattern="{"client":"%h", "client user":"%l", "authenticated":"%u", "access time":"%t", "method":"%r", "status":"%s", "send bytes":"%b", "Query?string":"%q", "partner":"%{Referer}i", "Agent version":"%{User-Agent}i"}"/>
在pattern部分，定义了json格式的日志
改了日志文件的前缀和后缀
2、重启tomcat，查看日志
[root@es1 apps]# ./tomcat/bin/startup.sh
[root@es1 apps]# tailf /apps/tomcat/logs/tomcat_access_log.2020-12-12.log
{"client":"192.168.80.1", "client user":"-", "authenticated":"-", "access time":"[12/Dec/2020:14:24:07 +0800]", "method":"GET /myapp/ HTTP/1.1", "status":"304", "send bytes":"-", "Query?string":"", "partner":"-", "Agent version":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"}
{"client":"192.168.80.1", "client user":"-", "authenticated":"-", "access time":"[12/Dec/2020:14:24:07 +0800]", "method":"GET /favicon.ico HTTP/1.1", "status":"404", "send bytes":"648", "Query?string":"", "partner":"http://192.168.80.106:8080/myapp/", "Agent version":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"}
配置logstash收集tomcat日志
-
先给tomcat日志加上logstash用户可以读的权限
[root@logstash ~]# chmod a+r /apps/tomcat/logs/tomcat_access_log.2020-12-12.log -
配置logstash的配置文件，输入为tomcat日志，输出为es的地址
[root@logstash ~]# cat /etc/logstash/conf.d/tomcat.conf input { file { path => "/apps/tomcat/logs/tomcat_access_log.*.log" start_position => "beginning" stat_interval => 3 type => "tomcat-log" codec => "json" } file { path => "/var/log/logstash/logstash-plain.log" start_position => "beginning" stat_interval => 3 type => "logstash-log" } } output { if [type] == "tomcat-log" { elasticsearch { hosts => ["192.168.80.107:9200"] index => "tomcat-log-%{+YYYY.MM.dd}" } } if [type] == "logstash-log" { elasticsearch { hosts => ["192.168.80.107:9200"] index => "logstash-log-%{+YYYY.MM.dd}" } } } -
语法检查，启动logstash
[root@logstash apps]# logstash -f /etc/logstash/conf.d/tomcat.conf -t WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console [WARN ] 2020-12-12 15:23:41.717 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified Configuration OK [INFO ] 2020-12-12 15:23:48.507 [LogStash::Runner] runner - Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash [root@logstash apps]# logstash -f /etc/logstash/conf.d/tomcat.conf -
kibana界面创建tomcat的索引
-
kibana界面discover查看上步创建的索引日志
-
另外一个数据流logstash日志的同理操作即可