elasticsearch实现根据某一个字段去重后并求总数

版本说明:

elasticsearch:elasticsearch-7.12.1

kibana:kibana-7.12.1-windows-x86_64

原始数据:

_index

_type

_id

_score

address

age

bir

id

name

jason_doc_10

_doc

L2aqNoABfXKo_Zq7BPkO

1

中国深圳

69

1650184553376

daZhaoLiu

大赵六

jason_doc_10

_doc

MGaqNoABfXKo_Zq7mfma

1

中国上海市宝山区

20

1649862303342

user001

大张三

jason_doc_10

_doc

MWaqNoABfXKo_Zq7mfma

1

中国上海市浦东区

50

1649862303342

user00100

小张三

jason_doc_10

_doc

MmaqNoABfXKo_Zq7mfma

1

中国上海市黄埔

34

1649862303342

00user1

李四

jason_doc_10

_doc

M2aqNoABfXKo_Zq7mfma

1

上海普陀区

45

1649862303342

user0011

大李四

jason_doc_10

_doc

NGaqNoABfXKo_Zq7mfma

1

中国北京市朝阳区

62

1649862303342

user2201

小李四

jason_doc_10

_doc

NWaqNoABfXKo_Zq7mfma

1

中国北京市东城

42

1649862303342

78user01

王五

jason_doc_10

_doc

NmaqNoABfXKo_Zq7mfma

1

中国北京市西城区

12

1649862303342

22

大王五

jason_doc_10

_doc

N2aqNoABfXKo_Zq7mfma

1

北京市海淀区

18

1649862303342

user01

小王五

jason_doc_10

_doc

OGasNoABfXKo_Zq77Pkx

1

10

test_save_01

测试save用户

jason_doc_10

_doc

OWasNoABfXKo_Zq77Pmu

1

20

test_save_02

测试save用户_02

jason_doc_10

_doc

OmasNoABfXKo_Zq77Pmu

1

30

test_save_03

测试save用户_03

jason_doc_10

_doc

Q2aeO4ABfXKo_Zq7TvlP

1

广东省广州市天河区

55

1650267672018

daZhaoLiu

王海

 因为数据的原因,我这里只有年龄age字段有重复的,所以根据age去重。

DSL请求:

GET /jason_doc_10/_search
{
  "size": 0,
  "aggs": {
    "distic_age": {
      "cardinality": {
        "field": "age"
      }
    } 
  }
}

响应结果:

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 13,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "distic_age" : {
      "value" : 12
    }
  }
}

java中的实现:

首先pom依赖:

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
            <version>2.6.6</version>
        </dependency>
<!--  注意:这里使用的ES版本是7.12.1,所以我本地安装的ES和kibana也是7.12.1 -->

 配置的客户端:HighLevelClient

@SpringBootConfiguration
public class RestHingLevelClient {

    @Bean
    public RestClientBuilder restClientBuilder() {
        return RestClient.builder(createHttpHost("127.0.0.1:9200"));
    }

    @Bean
    public RestHighLevelClient restHighLevelClient(@Autowired RestClientBuilder restClientBuilder) {
        return new RestHighLevelClient(restClientBuilder);
    }

    private HttpHost createHttpHost(String ipPort) {
        String[] split = ipPort.split(":");
        String ip = split[0];
        int port = Integer.parseInt(split[1]);
        return new HttpHost(ip, port, "http");
    }
}

测试类:

@SpringBootTest
public class TestQueryDocument {

    @Autowired
    @Qualifier(value = "restHighLevelClient")
    RestHighLevelClient client;

    @Test
    public void testDistinctByAge() throws IOException {
        // 创建查询请求
        SearchRequest request = new SearchRequest();
        request.indices("jason_doc_10");
        // 创建查询条件
        SearchSourceBuilder builder = new SearchSourceBuilder();
        // 根据年龄去重,并起别名为:u_age
        builder.aggregation(AggregationBuilders.cardinality("distic_age").field("age"));
        request.source(builder);
        // 查询
        SearchResponse response = client.search(request, RequestOptions.DEFAULT);
        Aggregations aggregations = response.getAggregations();
        ParsedCardinality cardinality = (ParsedCardinality) aggregations.asList().get(0);
        long count = cardinality.getValue();
        System.out.println("去重后的总数:" + count);
    }

    原文作者:九零大叔芭蕉
    原文地址: https://blog.csdn.net/whxjason/article/details/124261929
    本文转自网络文章,转载此文章仅为分享知识,如有侵权,请联系博主进行删除。
点赞