Day05- ES安装使用

本章小结：

同步

idea的alt+insert, 自动添加测试代码、getset方法很哇塞！
idea在方法参数上用ctrl+p,效果很赞

[TOC]

No.01-ES、Kibana、IK安装使用

1、初识ES

ES是什么
- 来源lucene，因lucene开发维护难度大，不支持水平扩展、局限java、接口不优化等问题而在2010年问世
ES的组成部分
- 指的是Elastic Stack，主要包括可视化kibana，存储搜索引擎ElasticSearch，数据抓取Logstash、Beats
什么是倒排索引
- 相比正向索引，对行数据（文档）进行分词、排序形成的索引表，使用时先查倒排索引再查一般索引而得到json
ES与Mysql的关系
- 小量数据，高效索引建议直接用mysql
- 强事务增删改必须用mysql！数据及时同步到es即可。
- es的优势在于面向海量数据、水平扩展、复杂查询全表遍历的情况下效果显著！

2、ElasticSearch安装

部署单点es

因为我们还需要部署kibana容器，因此需要让es和kibana容器互联。这里先创建一个网络：

docker network create es-net

运行docker命令，部署单点es：

docker run -d \
    --name es \
    -e "ES_JAVA_OPTS=-Xms512m -Xmx512m" \
    -e "discovery.type=single-node" \
    -v es-data:/usr/share/elasticsearch/data \
    -v es-plugins:/usr/share/elasticsearch/plugins \
    --privileged \
    --network es-net \
    -p 9200:9200 \
    -p 9300:9300 \
elasticsearch:7.14.2

命令解释：

-e "cluster.name=es-docker-cluster"：设置集群名称
-e "http.host=0.0.0.0"：监听的地址，可以外网访问
-e "ES_JAVA_OPTS=-Xms512m -Xmx512m"：内存大小
-e "discovery.type=single-node"：非集群模式
-v es-data:/usr/share/elasticsearch/data：挂载逻辑卷，绑定es的数据目录
-v es-logs:/usr/share/elasticsearch/logs：挂载逻辑卷，绑定es的日志目录
-v es-plugins:/usr/share/elasticsearch/plugins：挂载逻辑卷，绑定es的插件目录
--privileged：授予逻辑卷访问权
--network es-net ：加入一个名为es-net的网络中
-p 9200:9200：端口映射配置

在浏览器中输入：http://192.168.20.165:9200 即可看到elasticsearch的响应结果：

3、Kibana安装

运行docker命令，部署kibana

docker run -d \
--name kibana \
-e ELASTICSEARCH_HOSTS=http://es:9200 \
--network=es-net \
-p 5601:5601  \
kibana:7.14.2

--network es-net ：加入一个名为es-net的网络中，与elasticsearch在同一个网络中
-e ELASTICSEARCH_HOSTS=http://es:9200"：设置elasticsearch的地址，因为kibana已经与elasticsearch在一个网络，因此可以用容器名直接访问elasticsearch
-p 5601:5601：端口映射配置

kibana启动一般比较慢，需要多等待一会，可以通过命令：

docker logs -f kibana

查看运行日志，当查看到==日志出现5601端口号==，说明成功：

此时，在浏览器输入地址访问：http://192.168.20.165:5601，即可看到结果

4、Kibana DevTools基本操作（含分词器调用）

入口 http://192.168.20.165:5601，左侧按钮点击，然后滚动到最底下dev tools

一般操作

```json
GET _search
{
“query”: {
“match_all”: {}
}
}

同kibana首页

GET /

对比分词器,standard, ik_smart, ik_max_word

POST /_analyze
{
“analyzer”: “ik_smart”,
“text”: “传智播客的Java就业超过90%,奥力给！！”
}


![image-20211019171515356](http://image.iyyxx.com/i/2022/08/11/62f459a56f7d2.png)



### 5、IK安装、分词器插件升级

- IK安装

```sh
# 卷来自于es安装时指定的plugins目录， 操作只是找到目录，创建一个ik文件夹，然后上传压缩包、解压，重启es即可

[root@docker volumes]# docker volume inspect es-plugins
[
    {
        "CreatedAt": "2021-10-19T15:09:25+08:00",
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/volumes/es-plugins/_data",
        "Name": "es-plugins",
        "Options": null,
        "Scope": "local"
    }
]
[root@docker volumes]# cd /var/lib/docker/volumes/es-plugins/_data
[root@docker _data]# mkdir ik
[root@docker _data]# cd ik
[root@docker ik]# rz
rz waiting to receive.
Starting zmodem transfer.  Press Ctrl+C to cancel.
Transferring elasticsearch-analysis-ik-7.14.2.zip...
  100%    4398 KB    4398 KB/sec    00:00:01       0 Errors  

[root@docker ik]# unzip elasticsearch-analysis-ik-7.14.2.zip 
[root@docker ik]# mv elasticsearch-analysis-ik-7.14.2.zip /tmp/
[root@docker ik]# docker restart es

分词器增加扩展分词、停用分词

修改配置，添加扩展

[root@docker ik]# cd /var/lib/docker/volumes/es-plugins/_data/ik/config/
[root@docker config]# vim IKAnalyzer.cfg.xml

配置文件

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
        <comment>IK Analyzer 扩展配置</comment>
        <!--用户可以在这里配置自己的扩展字典 -->
        <entry key="ext_dict">ext.dic</entry>
         <!--用户可以在这里配置自己的扩展停止词字典-->
        <entry key="ext_stopwords">stopwords.dic</entry>
        <!--用户可以在这里配置远程扩展字典 -->
        <!-- <entry key="remote_ext_dict">words_location</entry> -->
        <!--用户可以在这里配置远程扩展停止词字典-->
        <!-- <entry key="remote_ext_stopwords">words_location</entry> -->
</properties>

添加词库到扩展、停用

[root@docker config]# touch ext.dic
[root@docker config]# echo "传智播客" >> ext.dic
[root@docker config]# echo "奥利给" >> ext.dic
[root@docker config]# echo "的" >> stopwords.dic
[root@docker ik]# docker restart es

使用

```json

对比分词器,standard, ik_smart, ik_max_word

POST /_analyze
{
“analyzer”: “ik_smart”,
“text”: “传智播客的Java就业超过90%,奥力给！！”
}




`效果`

![image-20211019171515356](http://image.iyyxx.com/i/2022/08/11/62f459a56f7d2.png)

## No.02-Kibana DevTools操作

### 1、操作索引库(CRUD)

`修改部分只允许新增字段`

```json
# 创建索引库
PUT /iyyxx
{
"mappings": {
  "properties": {
    "test": {
      "type": "text",
      "analyzer": "ik_smart"
    },
    "age": {
      "type": "text",
      "index": false
    },
    "name": {
      "properties": {
        "firstName": {
          "type": "keyword"
        },
        "lastName": {
          "type": "keyword"
        }
      }
    }
  }
}
}

# 查看索引库
GET /iyyxx

# 删除索引库
DELETE /iyyxx

# 修改索引库(添加字段,不允许修改)
PUT /iyyxx/_mapping
{
"properties": {
  "newField": {
    "type": "keyword"
  }
}
}

2、操作文档

# 新增文档(存在就是先删除)
POST /iyyxx/_doc/1
{
  "test":"测试",
  "age":"15",
  "name":{
    "lastName":"三",
    "firstName":"张"
  },
  "newField":"ttt"
}

# 查询文档
GET /iyyxx/_doc/1

# 全量替换
PUT /iyyxx/_doc/1
{
  "test":"测试"
}

# 部分修改
POST /iyyxx/_update/1
{
  "doc": {
    "test":"测试22"
  }
}

# 删除文档
DELETE /iyyxx/_doc/1

No.03-RestClient操作

使用培训机构提供的demo工程和数据库脚本

pom依赖

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.3.10.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

    <groupId>org.example</groupId>
    <artifactId>hotel-demo</artifactId>
    <version>1.0-SNAPSHOT</version>
    <name>hotel-demo</name>
    <description>xxxxxxxxxxxxxxxxx</description>

    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
    </properties>


    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>mybatis-plus-boot-starter</artifactId>
            <version>3.4.2</version>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.junit.vintage</groupId>
                    <artifactId>junit-vintage-engine</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <!--FastJson-->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.71</version>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <excludes>
                        <exclude>
                            <groupId>org.projectlombok</groupId>
                            <artifactId>lombok</artifactId>
                        </exclude>
                    </excludes>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

pojo/Hotel

package com.iyyxx.hotel.pojo;

import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data;

@Data
@TableName("tb_hotel")
public class Hotel {
    @TableId(type = IdType.INPUT)
    private Long id;
    private String name;
    private String address;
    private Integer price;
    private Integer score;
    private String brand;
    private String city;
    private String starName;
    private String business;
    private String longitude;
    private String latitude;
    private String pic;
}

pojo/HotelDoc

package com.iyyxx.hotel.pojo;

import lombok.Data;
import lombok.NoArgsConstructor;

@Data
@NoArgsConstructor
public class HotelDoc {
    private Long id;
    private String name;
    private String address;
    private Integer price;
    private Integer score;
    private String brand;
    private String city;
    private String starName;
    private String business;
    private String location;
    private String pic;

    public HotelDoc(Hotel hotel) {
        this.id = hotel.getId();
        this.name = hotel.getName();
        this.address = hotel.getAddress();
        this.price = hotel.getPrice();
        this.score = hotel.getScore();
        this.brand = hotel.getBrand();
        this.city = hotel.getCity();
        this.starName = hotel.getStarName();
        this.business = hotel.getBusiness();
        this.location = hotel.getLatitude() + ", " + hotel.getLongitude();
        this.pic = hotel.getPic();
    }
}

mapper/HotelMapper

package com.iyyxx.hotel.mapper;

import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.iyyxx.hotel.pojo.Hotel;

public interface HotelMapper extends BaseMapper<Hotel> {
}

service/IHotelService

package com.iyyxx.hotel.service;

import com.baomidou.mybatisplus.extension.service.IService;
import com.iyyxx.hotel.pojo.Hotel;

public interface IHotelService extends IService<Hotel> {
}

service/impl/HotelService

package com.iyyxx.hotel.service.impl;

import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import com.iyyxx.hotel.mapper.HotelMapper;
import com.iyyxx.hotel.pojo.Hotel;
import com.iyyxx.hotel.service.IHotelService;
import org.springframework.stereotype.Service;

@Service
public class HotelService extends ServiceImpl<HotelMapper, Hotel> implements IHotelService {
}

入口程序HotelDemoApplication

package com.iyyxx.hotel;

import com.baomidou.mybatisplus.annotation.DbType;
import com.baomidou.mybatisplus.extension.plugins.MybatisPlusInterceptor;
import com.baomidou.mybatisplus.extension.plugins.inner.PaginationInnerInterceptor;
import org.mybatis.spring.annotation.MapperScan;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;

@MapperScan("com.iyyxx.hotel.mapper")
@SpringBootApplication
public class HotelDemoApplication {

  public static void main(String[] args) {
    SpringApplication.run(HotelDemoApplication.class, args);
  }
}

1、操作索引库

pom依赖增加es，并强制覆盖父工程对es的版本限制

<properties>
    <elasticsearch.version>7.14.2</elasticsearch.version>
</properties>

<!-- elasticSearch依赖 -->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
</dependency>

添加测试程序

package com.iyyxx.hotel;

import org.apache.http.HttpHost;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.indices.CreateIndexRequest;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.GetIndexRequest;
import org.elasticsearch.common.xcontent.XContentType;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

import java.io.IOException;

import static com.iyyxx.hotel.constants.HotelConstants.MAPPING_TEMPLATE;

public class HotelIndexTest {
  private RestHighLevelClient restHighLevelClient;

  @BeforeEach
  void setUp() {
    restHighLevelClient =
        new RestHighLevelClient(RestClient.builder(HttpHost.create("http://192.168.20.165:9200")));
  }

  @AfterEach
  void tearDown() throws IOException {
    this.restHighLevelClient.close();
  }

  @Test
  void testCreateHotelIndex() throws IOException {
    // 创建对象
    CreateIndexRequest request = new CreateIndexRequest("hotel");
    // 配置请求参数
    request.source(MAPPING_TEMPLATE, XContentType.JSON);
    // 发起请求
    restHighLevelClient.indices().create(request, RequestOptions.DEFAULT);
  }

  @Test
  void testDeleteHotelIndex() throws IOException {

    DeleteIndexRequest request = new DeleteIndexRequest("hotel");

    restHighLevelClient.indices().delete(request, RequestOptions.DEFAULT);
  }

  @Test
  void testExistsHotelIndex() throws IOException {

    GetIndexRequest request = new GetIndexRequest("hotel");
    boolean exists = restHighLevelClient.indices().exists(request, RequestOptions.DEFAULT);
    System.out.println(exists);
  }


}

2、操作文档

文档操作类似索引操作，多了数据增删改、批量导入！

package com.iyyxx.hotel;

import com.alibaba.fastjson.JSON;
import com.iyyxx.hotel.pojo.Hotel;
import com.iyyxx.hotel.pojo.HotelDoc;
import com.iyyxx.hotel.service.IHotelService;
import org.apache.http.HttpHost;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.get.GetRequest;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.xcontent.XContentType;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;

import java.io.IOException;
import java.util.List;

import static com.iyyxx.hotel.constants.HotelConstants.MAPPING_TEMPLATE;

@SpringBootTest
public class HotelDocumentTest {

  @Autowired private IHotelService service;

  private RestHighLevelClient restHighLevelClient;

  @BeforeEach
  void setUp() {
    restHighLevelClient =
        new RestHighLevelClient(RestClient.builder(HttpHost.create("http://192.168.20.165:9200")));
  }

  @AfterEach
  void tearDown() throws IOException {
    this.restHighLevelClient.close();
  }

  @Test
  void test() {
    System.out.println(restHighLevelClient);
  }

  @Test
  void testIndexDocument() throws IOException {
    Hotel hotel = service.getById(38812L);
    HotelDoc doc = new HotelDoc(hotel);
    IndexRequest request = new IndexRequest("hotel").id(hotel.getId().toString());
    request.source(JSON.toJSONString(doc), XContentType.JSON);
    restHighLevelClient.index(request, RequestOptions.DEFAULT);
  }

  @Test
  void testGetDocument() throws IOException {
    GetRequest request = new GetRequest("hotel", "38812");
    GetResponse response = restHighLevelClient.get(request, RequestOptions.DEFAULT);
    String json = response.getSourceAsString();
    HotelDoc doc = JSON.parseObject(json, HotelDoc.class);
    System.err.println(doc);
  }

  @Test
  void testUpdateDocument() throws IOException {
    UpdateRequest request = new UpdateRequest("hotel", "38812");
    request.doc("score", 55, "starName", "三三钻");
    restHighLevelClient.update(request, RequestOptions.DEFAULT);
  }

  @Test
  void testDeleteDocument() throws IOException {
    DeleteRequest request = new DeleteRequest("hotel", "38812");
    restHighLevelClient.delete(request, RequestOptions.DEFAULT);
  }

  @Test
  void testBulkOperateDocument() throws IOException {
    BulkRequest request = new BulkRequest();
    List<Hotel> list = service.list();
    IndexRequest index;
    for (Hotel hotel : list) {
      HotelDoc doc = new HotelDoc(hotel);
      index = new IndexRequest("hotel").id(hotel.getId().toString());
      index.source(JSON.toJSONString(doc), XContentType.JSON);
      request.add(index);
    }
    restHighLevelClient.bulk(request, RequestOptions.DEFAULT);
  }
}

分享技术，品味人生

微服务-ES使用

Day05- ES安装使用

No.01-ES、Kibana、IK安装使用

1、初识ES

2、ElasticSearch安装

3、Kibana安装

4、Kibana DevTools基本操作（含分词器调用）

同kibana首页

对比分词器,standard, ik_smart, ik_max_word

对比分词器,standard, ik_smart, ik_max_word

2、操作文档

No.03-RestClient操作

1、操作索引库

2、操作文档