原创

ELK部署-elasticsearch安装调试（三）

一、简介

Elasticsearch 是一个分布式、高扩展、高实时的搜索与数据分析引擎。它能很方便的使大量数据具有搜索、分析和探索的能力。充分利用Elasticsearch的水平伸缩性，能使数据在生产环境变得更有价值。Elasticsearch 的实现原理主要分为以下几个步骤，首先用户将数据提交到Elasticsearch 数据库中，再通过分词控制器去将对应的语句分词，将其权重和分词结果一并存入数据，当用户搜索数据时候，再根据权重将结果排名，打分，再将返回结果呈现给用户。
Elasticsearch是与名为Logstash的数据收集和日志解析引擎以及名为Kibana的分析和可视化平台一起开发的。这三个产品被设计成一个集成解决方案，称为“Elastic Stack”（以前称为“ELK stack”）。
Elasticsearch可以用于搜索各种文档。它提供可扩展的搜索，具有接近实时的搜索，并支持多租户。Elasticsearch是分布式的，这意味着索引可以被分成分片，每个分片可以有0个或多个副本。每个节点托管一个或多个分片，并充当协调器将操作委托给正确的分片。再平衡和路由是自动完成的。相关数据通常存储在同一个索引中，该索引由一个或多个主分片和零个或多个复制分片组成。一旦创建了索引，就不能更改主分片的数量。

二、安装

下载安装包

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.16.2-linux-x86_64.tar.gz

解压文件到安装目录

[root@bogon src]# tar xf elasticsearch-7.16.2-linux-x86_64.tar.gz -C /data/env/

创建一个运行es的用户（不能直接使用root用户运行）

[root@bogon src]# id www >/dev/null 2>&1 || useradd www -u 1001 -s /bin/bash

切换到安装目录，修改配置文件

[root@bogon]# cd /data/env/elasticsearch-7.16.2/config
[root@bogon config]# vim elasticsearch.yml

修改network.host、http.port、discovery.seed_hosts参数

修改的目的是让其它机器可以访问，方便验证

node.name: node-1
network.host: 0.0.0.0
http.port: 9200
cluster.initial_master_nodes: ["node-1"]

http.cors.enabled: true
http.cors.allow-origin: "*"

discovery.zen.fd.ping_timeout: 1000s
discovery.zen.fd.ping_retries: 10

# 开启用户认证
http.cors.allow-headers: Authorization 
xpack.security.enabled: true           
xpack.security.transport.ssl.enabled: true

配置密码

./bin/elasticsearch-setup-passwords interactive
# 需要配置多个账号密码

network.host 是访问IP，通过这个IP访问，“0.0.0.0”就可以

http.port 是配置http的访问端口，为9200

三、启动测试

1、启动前配置

服务目录授权

[root@bogon config]# chown -R www:www /data/env/elasticsearch-7.16.2

切换至www用户

[root@bogon config]# su - www

切换至服务目录

-bash-4.2$ cd /data/env/elasticsearch-7.16.2

2、运行测试

-bash-4.2$ bin/elasticsearch

启动报错

ERROR: [1] bootstrap checks failed. You must address the points described in the following [1] lines before starting Elasticsearch.
bootstrap check failure [1] of [1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
ERROR: Elasticsearch did not exit normally - check the logs at /data/env/elasticsearch-7.16.2/logs/elasticsearch.log
[2022-01-07T18:41:08,624][INFO ][o.e.n.Node               ] [bogon] stopping ...
[2022-01-07T18:41:08,638][INFO ][o.e.n.Node               ] [bogon] stopped
[2022-01-07T18:41:08,639][INFO ][o.e.n.Node               ] [bogon] closing ...
[2022-01-07T18:41:08,654][INFO ][o.e.n.Node               ] [bogon] closed

解决方法

# 使用root用户修改
[root@bogon config]# vim /etc/security/limits.conf
# 最下面追加两行
* hard nofile 65536
* soft nofile 65536

再次切换到www用户测试运行，无报错启动

[2022-01-07T18:43:34,602][INFO ][o.e.h.AbstractHttpServerTransport] [bogon] publish_address {192.168.30.132:9200}, bound_addresses {[::]:9200}
[2022-01-07T18:43:34,603][INFO ][o.e.n.Node               ] [bogon] started
[2022-01-07T18:43:34,926][INFO ][o.e.l.LicenseService     ] [bogon] license [31e8d57f-a54f-4cb3-be0e-b6411d96d7f5] mode [basic] - valid
[2022-01-07T18:43:34,927][INFO ][o.e.x.s.s.SecurityStatusChangeListener] [bogon] Active license is now [BASIC]; Security is disabled
[2022-01-07T18:43:34,927][WARN ][o.e.x.s.s.SecurityStatusChangeListener] [bogon] Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security.
[2022-01-07T18:43:34,937][INFO ][o.e.g.GatewayService     ] [bogon] recovered [3] indices into cluster_state
[2022-01-07T18:43:35,405][INFO ][o.e.c.r.a.AllocationService] [bogon] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[.ds-.logs-deprecation.elasticsearch-default-2022.01.06-000001][0], [.ds-ilm-history-5-2022.01.06-000001][0], [.geoip_databases][0]]]).
[2022-01-07T18:43:35,406][INFO ][o.e.i.g.GeoIpDownloader  ] [bogon] updating geoip databases
[2022-01-07T18:43:35,407][INFO ][o.e.i.g.GeoIpDownloader  ] [bogon] fetching geoip databases overview from [https://geoip.elastic.co/v1/database?elastic_geoip_service_tos=agree]
[2022-01-07T18:43:35,458][INFO ][o.e.i.g.DatabaseNodeService] [bogon] downloading geoip database [GeoLite2-Country.mmdb] to [/tmp/elasticsearch-12158226475284072661/geoip-databases/jG0ZHmUdTpmyHTRIqmBb7A/GeoLite2-Country.mmdb.tmp.gz]
[2022-01-07T18:43:35,460][INFO ][o.e.i.g.DatabaseNodeService] [bogon] downloading geoip database [GeoLite2-ASN.mmdb] to [/tmp/elasticsearch-12158226475284072661/geoip-databases/jG0ZHmUdTpmyHTRIqmBb7A/GeoLite2-ASN.mmdb.tmp.gz]
[2022-01-07T18:43:35,461][INFO ][o.e.i.g.DatabaseNodeService] [bogon] downloading geoip database [GeoLite2-City.mmdb] to [/tmp/elasticsearch-12158226475284072661/geoip-databases/jG0ZHmUdTpmyHTRIqmBb7A/GeoLite2-City.mmdb.tmp.gz]
[2022-01-07T18:43:35,879][INFO ][o.e.i.g.DatabaseNodeService] [bogon] successfully reloaded changed geoip database file [/tmp/elasticsearch-12158226475284072661/geoip-databases/jG0ZHmUdTpmyHTRIqmBb7A/GeoLite2-Country.mmdb]
[2022-01-07T18:43:35,933][INFO ][o.e.i.g.DatabaseNodeService] [bogon] successfully reloaded changed geoip database file [/tmp/elasticsearch-12158226475284072661/geoip-databases/jG0ZHmUdTpmyHTRIqmBb7A/GeoLite2-ASN.mmdb]
[2022-01-07T18:43:36,666][INFO ][o.e.i.g.GeoIpDownloader  ] [bogon] geoip database [GeoLite2-ASN.mmdb] is up to date, updated timestamp
[2022-01-07T18:43:36,823][INFO ][o.e.i.g.GeoIpDownloader  ] [bogon] geoip database [GeoLite2-City.mmdb] is up to date, updated timestamp
[2022-01-07T18:43:36,982][INFO ][o.e.i.g.GeoIpDownloader  ] [bogon] geoip database [GeoLite2-Country.mmdb] is up to date, updated timestamp
[2022-01-07T18:43:37,275][INFO ][o.e.i.g.DatabaseNodeService] [bogon] successfully reloaded changed geoip database file [/tmp/elasticsearch-12158226475284072661/geoip-databases/jG0ZHmUdTpmyHTRIqmBb7A/GeoLite2-City.mmdb]

测试访问，服务器IP+9200端口，访问返回结果如下：

{
  "name" : "bogon",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "GpaLnsc8Ti2aat7pQRg7ug",
  "version" : {
    "number" : "7.16.2",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "2b937c44140b6559905130a8650c64dbd0879cfb",
    "build_date" : "2021-12-18T19:42:46.604893745Z",
    "build_snapshot" : false,
    "lucene_version" : "8.10.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

3、配置启动文件

回到root用户

编辑文件

vim /lib/systemd/system/elasticsearch.service

添加内容

[Unit]
Description=Elasticsearch
Documentation=http://www.elastic.co
Wants=network-online.target
After=network-online.target
[Service]
Environment=ES_HOME=/data/soft/elasticsearch
Environment=CONF_DIR=/data/soft/elasticsearch/config
Environment=DATA_DIR=/data/soft/elasticsearch/data
Environment=LOG_DIR=/data/soft/elasticsearch/logs
Environment=PID_DIR=/data/soft/elasticsearch
EnvironmentFile=-/etc/sysconfig/elasticsearch

WorkingDirectory=/data/soft/elasticsearch

User=www
Group=www

#ExecStartPre=/data/soft/elasticsearch/bin/elasticsearch-systemd-pre-exec

ExecStart=/data/soft/elasticsearch/bin/elasticsearch \
                                                -p /data/soft/elasticsearch/elasticsearch.pid \
                                                --quiet 


# StandardOutput is configured to redirect to journalctl since
# some error messages may be logged in standard output before
# elasticsearch logging system is initialized. Elasticsearch
# stores its logs in /var/log/elasticsearch and does not use
# journalctl by default. If you also want to enable journalctl
# logging, you can simply remove the "quiet" option from ExecStart.
StandardOutput=journal
StandardError=inherit

# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65536

# Specifies the maximum number of processes
LimitNPROC=65536

# Specifies the maximum number of bytes of memory that may be locked into RAM
# Set to "infinity" if you use the 'bootstrap.memory_lock: true' option
# in elasticsearch.yml and 'MAX_LOCKED_MEMORY=unlimited' in /etc/sysconfig/elasticsearch
#LimitMEMLOCK=infinity

# Disable timeout logic and wait until process is stopped
TimeoutStopSec=0

# SIGTERM signal is used to stop the Java process
KillSignal=SIGTERM

# Send the signal only to the JVM rather than its control group
KillMode=process

# Java process is never killed
SendSIGKILL=no

# When a JVM receives a SIGTERM signal it exits with code 143
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

通过启动文件启动服务

[root@bogon config]# systemctl start elasticsearch

查看启动状态

[root@bogon config]# systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
   Active: active (running) since 五 2022-01-07 18:51:09 CST; 11s ago
     Docs: http://www.elastic.co
 Main PID: 30541 (java)
   CGroup: /system.slice/elasticsearch.service
           ├─30541 /data/soft/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThr...           └─30740 /data/soft/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

1月 07 18:51:09 bogon systemd[1]: Started Elasticsearch.
1月 07 18:51:09 bogon systemd[1]: Starting Elasticsearch...