概述
在使用graylog时，默认分页查询存在限制，真实使用不能满足，需要我们手动处理。当查询超过执行长度时，会出现一下错误提示
While retrieving data for this widget, the following error(s) occurred:
Unable to perform search query: Elasticsearch exception [type=illegal_argument_exception, reason=Result window is too large, from + size must be less than or equal to: [10000] but was [3382050]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.].
Elasticsearch检索问题
Elasticsearch的max_result_window限制
问题描述
查询超过10000页，Elasticsearch出现异常
Elasticsearch exception [type=illegal_argument_exception, reason=Result window is too large, from + size must be less than or equal to: [10000] but was [7135950]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.].
解决方案
方案一：修改配置文件，重启Elasticsearch服务【Elasticsearch5.x版本以后不支持】
修改Elasticsearch 集群中的 配置文件 config/elasticsearch.yml 在配置文件最后增加一行，如下：
index.max_result_window: 1000000
注意： 日志文件路径：/var/log/elasticsearch/graylog.log
方案二：通过接口修改具体的index
具体操作命令，如下（比如，设置可查询 1000000 条数据，其中 alarm 是index名称）： 推荐使用全局修改方式。
# 修改个别索引
PUT alarm/_settings
{
"max_result_window" : 1000000
}
# 修改全局 100W
PUT _settings
{
"index": {
"max_result_window": "1000000"
}
}
CURL方式
curl -H "Content-Type: application/json" -XPUT http://127.0.0.1:9200/_all/_settings -d '{ "index" : { "max_result_window" : 1000000}}'
注意：
上述修改方式，对于新建的索引不会生效。如果需要让新建的索引也生效，必须重新覆盖_template
方案三：修改template【推荐】
curl -H "Content-Type: application/json" -XPUT http://127.0.0.1:9200/_template/graylog-gdmp-mapping -d '{
"order": 1,
"index_patterns": [
"gdmp_*"
],
"settings": {
"index": {
"analysis": {
"analyzer": {
"analyzer_keyword": {
"filter": "lowercase",
"tokenizer": "keyword"
}
}
},
"max_result_window": 1000000
}
},
"mappings": {
"_source": {
"enabled": true
},
"dynamic_templates": [
{
"internal_fields": {
"mapping": {
"type": "keyword"
},
"match_mapping_type": "string",
"match": "gl2_*"
}
},
{
"store_generic": {
"mapping": {
"type": "keyword"
},
"match_mapping_type": "string"
}
}
],
"properties": {
"streams": {
"type": "keyword"
},
"message": {
"fielddata": false,
"analyzer": "standard",
"type": "text"
},
"timestamp": {
"format": "uuuu-MM-dd HH:mm:ss.SSS",
"type": "date"
}
}
}
}'
# 查看索引映射 /索引名/_mapping
GET /gdmp_f08985deb3064a02ab46eeaff55fe001_0/_mapping
# 查看索引配置 /索引名/_settings
GET /gdmp_da7eb85c302f4224b10eeed5314c2cae_1/_settings
参考资料
使用elasticsearch分页时报max_result_window is too large的错误解决方案 | 宝贝云计算知识分享京东面试题：ElasticSearch深度分页解决方案_Java_小小怪下士_InfoQ写作社区https://www.cnblogs.com/rongfengliang/p/16845628.htmlhttps://blog.csdn.net/weixin_44692700/article/details/122160837
参考链接
您阅读本篇文章共花了：
发表评论