logo头像

分享技术，品味人生

微服务-ES高级查询和案例

2022/08/24

Day06- ES高级查询和案例

本章小结：

  1. 同步

  • i

[TOC]

No.01-ES高级查询

1、全部查询

# 全部查询（默认前十条）
GET /hotel/_search
{
  "query": {
    "match_all": {}
  }
}

2、全文搜索

# 单字段全文搜索（分词倒排索引，但是用到了多字段聚合）
GET /hotel/_search
{
  "query": {
    "match": {
      "all": "外滩如家"
    }
  }
}

# 多字段全文检索（性能低下，可以的话提前做聚合)
GET /hotel/_search
{
  "query": {
    "multi_match": {
      "query": "外滩如家",
      "fields": ["brand", "business", "name"]
    }
  }
}

3、精确匹配

# 精确匹配
GET /hotel/_search
{
  "query": {
    "term": {
      "city": {
        "value": "北京"
      }
    }
  }
}

4、范围查询（数字、日期？）

# range范围查询
GET /hotel/_search
{
  "query": {
    "range": {
      "price": {
        "gte": 3000,
        "lte": 5000
      }
    }
  }
}

5、地理查询（矩形、半径）

# 地理坐标矩形查询 geo_bounding_box查询
GET /hotel/_search
{
  "query": {
    "geo_bounding_box": {
      "location": {
        "top_left": { 
          "lat": 31.1, 
          "lon": 121.5
        },
        "bottom_right": { 
          "lat": 30.9,
          "lon": 121.7
        }
      }
    }
  }
}


# 地理半径查询 geo_distance 查询
GET /hotel/_search
{
  "query": {
    "geo_distance": {
      "distance": "15km", 
      "location": "31.21,121.5" 
    }
  }
}

6、算法函数查询

# 算法函数查询
GET /hotel/_search
{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "all": "外滩如家"
        }
      },
      "functions": [
        {
          "filter": {
            "term": {
              "brand": "君悦"
            }
          },
          "weight": 2
        }
      ],
      "boost_mode": "multiply"
    }
  }
}

7、组合查询

# 布尔组合查询，must_not，filter不算法性能高，推荐
GET /hotel/_search
{
  "query": {
    "bool": {
      "must": [
        {"term": {"city": "上海" }},
        {"match": {"starName": "五钻"}}
      ],
      "should": [
        {"term": {"brand": "皇冠假日" }},
        {"term": {"brand": "华美达" }}
      ],
      "must_not": [
        { "range": { "price": { "lte": 500 } }}
      ],
      "filter": [
        { "range": {"score": { "lte": 45 } }}
      ]
    }
  }
}

No.02-ES搜索结果处理

1、一般排序

# 一般排序
GET /hotel/_search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "score": "desc"  
    },
    {
      "price": "asc"  
    }
  ]
}

2、地理排序

# 经纬度排序
GET /hotel/_search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "_geo_distance" : {
          "location" : "30.9, 121.7", 
          "order" : "asc", 
          "unit" : "km" 
      }
    }
  ]
}

3、分页

分页默认1万条，超出要么改配置，要么用search after后翻页、scroll快照，但都有缺陷和性能浪费，一般从业务层面控制

百度、京东也是限制一千条、百页左右的而已

# 分页
GET /hotel/_search
{
  "query": {
    "match_all": {}
  },
  "from": 0, 
  "size": 10, 
  "sort": [
    {"price": "asc"}
  ]
}

4、高亮

# 高亮
GET /hotel/_search
{
  "query": {
    "match": {
      "name": "如家"
    }
  },
  "highlight": {
    "fields": { 
      "name": {
        "pre_tags": "<em>",  
        "post_tags": "</em>",
        "require_field_match": "false"
      }
    }
  }
}

No.03-RestClient编程

地理位置查询的实现采用后过滤的方案，==算法函数查询没有实现==

@Test
void testQueryFunctionScore() {}

官方文档-searchAPI

官方文档☞地理位置查询

1、查询

package com.iyyxx.hotel;

import com.alibaba.fastjson.JSON;
import com.iyyxx.hotel.pojo.HotelDoc;
import com.iyyxx.hotel.service.IHotelService;
import org.apache.http.HttpHost;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightField;
import org.elasticsearch.search.sort.SortOrder;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.util.CollectionUtils;

import java.io.IOException;
import java.util.Map;

@SpringBootTest
public class HotelSearchTest {

  @Autowired private IHotelService service;

  private RestHighLevelClient client;

  @BeforeEach
  void setUp() {
    client =
        new RestHighLevelClient(RestClient.builder(HttpHost.create("http://192.168.20.165:9200")));
  }

  @AfterEach
  void tearDown() throws IOException {
    this.client.close();
  }


  private void handleResponse(SearchResponse response) {
    SearchHits searchHits = response.getHits();
    long total = searchHits.getTotalHits().value;
    System.out.println("查询到总条数："+total);

    System.out.println("实际条数："+searchHits.getHits().length);
    SearchHit[] hits = searchHits.getHits();
    for (SearchHit hit : hits) {
      String json = hit.getSourceAsString();
      HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
      System.out.println(hotelDoc);
    }
  }

  @Test
  void testQueryMatchALl() throws IOException {
    SearchRequest request = new SearchRequest("hotel");

    // 全表扫描，但默认只范围前十条
    request.source().query(QueryBuilders.matchAllQuery());

    SearchResponse response = client.search(request, RequestOptions.DEFAULT);

    handleResponse(response);
  }


  @Test
  void testQueryMatch() throws IOException {
    SearchRequest request = new SearchRequest("hotel");

    // 单字段匹配，但是all其实有4,5个字段的提前聚合，具体可以 GET /hotel 查看
    request.source().query(QueryBuilders.matchQuery("all", "外滩如家"));

    SearchResponse response = client.search(request, RequestOptions.DEFAULT);

    handleResponse(response);
  }

  @Test
  void testQueryMulthiMatch()  throws IOException {
    SearchRequest request = new SearchRequest("hotel");

    // 多字段匹配“外滩如家”，按品牌、商业圈、名称
    request.source().query(QueryBuilders.multiMatchQuery("外滩如家", "brand", "business", "name"));

    SearchResponse response = client.search(request, RequestOptions.DEFAULT);

    handleResponse(response);
  }

  @Test
  void testQueryTerm()  throws IOException {
    SearchRequest request = new SearchRequest("hotel");

    // 精确查询位于北京城的酒店
    request.source().query(QueryBuilders.termQuery("city", "北京"));

    SearchResponse response = client.search(request, RequestOptions.DEFAULT);

    handleResponse(response);
  }

  @Test
  void testQueryRang()  throws IOException {
    SearchRequest request = new SearchRequest("hotel");

    // 查询价格大于300，小于500的酒店
    request.source().query(QueryBuilders.rangeQuery("price").gte(300).lte(500));

    SearchResponse response = client.search(request, RequestOptions.DEFAULT);

    handleResponse(response);
  }


  @Test
  void testQueryBool() throws IOException {
    SearchRequest request = new SearchRequest("hotel");

    // 上海市、五钻、品牌（皇冠假日、华美达）、价格不低于500，评分大于4.5
    BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
    boolQuery.must(QueryBuilders.termQuery("city","上海"));
    boolQuery.must(QueryBuilders.termQuery("starName","五钻"));
    boolQuery.should(QueryBuilders.termQuery("brand","皇冠假日"));
    boolQuery.should(QueryBuilders.termQuery("brand","华美达"));
    boolQuery.mustNot(QueryBuilders.rangeQuery("price").lt(500));
    boolQuery.filter(QueryBuilders.rangeQuery("score").lt(45));

    request.source().query(boolQuery);
    SearchResponse response = client.search(request, RequestOptions.DEFAULT);
    handleResponse(response);
  }

    
  @Test
  void testQueryGeoBox() throws IOException {
    SearchRequest request = new SearchRequest("hotel");
    request.source().query(QueryBuilders.matchAllQuery());

    // Using pre-indexed shapes
    request
        .source()
        .postFilter(
            QueryBuilders.geoBoundingBoxQuery("location").setCorners(31.1, 121.5, 30.9, 121.7));

    SearchResponse response = client.search(request, RequestOptions.DEFAULT);
    handleResponse(response);
  }

  @Test
  void testQueryGeoDistance() throws IOException {

    SearchRequest request = new SearchRequest("hotel");
    request.source().query(QueryBuilders.matchAllQuery());

    request
        .source()
        .postFilter(
            QueryBuilders.geoDistanceQuery("location")
                .point(31.21, 121.5)
                .distance(15, DistanceUnit.KILOMETERS));

    SearchResponse response = client.search(request, RequestOptions.DEFAULT);

    handleResponse(response);
  }
}

2、搜索结果处理（排序、分页、过滤、高亮等）

高亮部分的数据是另外组织，需要使用时得单独拿出来，然后做替换！

package com.iyyxx.hotel;

import com.alibaba.fastjson.JSON;
import com.iyyxx.hotel.pojo.HotelDoc;
import com.iyyxx.hotel.service.IHotelService;
import org.apache.http.HttpHost;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder;
import org.elasticsearch.search.fetch.subphase.highlight.HighlightField;
import org.elasticsearch.search.sort.SortOrder;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.util.CollectionUtils;

import java.io.IOException;
import java.util.Map;

@SpringBootTest
public class HotelSearchTest {

  @Autowired private IHotelService service;

  private RestHighLevelClient client;

  @BeforeEach
  void setUp() {
    client =
        new RestHighLevelClient(RestClient.builder(HttpHost.create("http://192.168.20.165:9200")));
  }

  @AfterEach
  void tearDown() throws IOException {
    this.client.close();
  }


  private void handleResponse(SearchResponse response) {
    SearchHits searchHits = response.getHits();
    long total = searchHits.getTotalHits().value;
    System.out.println("查询到总条数："+total);

    System.out.println("实际条数："+searchHits.getHits().length);
    SearchHit[] hits = searchHits.getHits();
    for (SearchHit hit : hits) {
      String json = hit.getSourceAsString();
      HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);

      // 高亮处理，取出name再塞回去。。。
      Map<String, HighlightField> highlightFieldMap = hit.getHi
          ghlightFields();
      if(!CollectionUtils.isEmpty(highlightFieldMap)){
        HighlightField highlightField = highlightFieldMap.get("name");
        if(highlightField!=null){
          String name = highlightField.getFragments()[0].string();
          hotelDoc.setName(name);
        }
      }

      System.out.println(hotelDoc);
    }
  }

  @Test
  void testPageAndSor() throws IOException {
    SearchRequest request = new SearchRequest("hotel");

    // 全表扫描
    request.source().query(QueryBuilders.matchAllQuery());

    // 排序，按价格从高到底
    request.source().sort("score", SortOrder.DESC);
    request.source().sort("price", SortOrder.ASC);

    // 分页，从0开始，50条一页
    request.source().from(0).size(50);

    SearchResponse response = client.search(request, RequestOptions.DEFAULT);

    handleResponse(response);
  }
    
  @Test
  void testHighLight() throws IOException {
    SearchRequest request = new SearchRequest("hotel");

    request.source().query(QueryBuilders.matchQuery("all", "如家"));

    // 全表扫描，对名称字段里的关键字进行高亮，并从10条开始取11条，突破默认0-10
    request.source().highlighter(new HighlightBuilder().field("name").requireFieldMatch(false));

    request.source().from(11).size(11);
    SearchResponse response = client.search(request, RequestOptions.DEFAULT);
    handleResponse(response);
  }
}

No.04-旅游商城案例

1、全部查询

转载声明： 商业转载请联系作者获得授权,非商业转载请注明出处 © Eric

评论系统未开启，无法评论！

访问量:   |   访客数:
Copyright © 2022 闽ICP备2022013260号-2 | Powered by Hexo | Theme by Snippet