elasticsearch笔记-001-基础操作

置顶 tech elasticsearch

发布时间 : 2020-10-22 22:53

字数:3.2k 阅读 :

1. _cat查看信息

1.1. _cat/health 查看集群健康状况

1602493310 09:01:50 elasticsearch yellow 1 1 10 10 0 0 5 0 - 66.7%

1.2. _cat/master 查看主节点信息

bQY8I8hFSjmCHO_1-MKfZg 127.0.0.1 127.0.0.1 83c34a872f4c

1.3 _cat/indices 查看索引信息

green open .security-7 M61Q-ekUQmGZP1FtS8ukfQ 1 0 42 0 74kb 74kb
yellow open niewj UuJpWw6UQcKBH8-TWquNng 5 1 0 0 1.3kb 1.3kb
green open .kibana_task_manager_1 1ebsopUOSN-P8ToMX-mklA 1 0 2 0 22.5kb 22.5kb
green open .apm-agent-configuration Le9Ig6qiSdaxMRLvmg43Lg 1 0 0 0 283b 283b
green open kibana_sample_data_logs jo7AsHgeTniG509NZWUi-A 1 0 14074 0 11.7mb 11.7mb
green open .kibana_1 o1ntzX2YRFGfU__BzFPYhg 1 0 47 0 109.1kb 109.1kb

1.4 其他 _cat 命令

/_cat/allocation
/_cat/shards
/_cat/shards/{index}
/_cat/master
/_cat/nodes
/_cat/tasks
/_cat/indices
/_cat/indices/{index}
/_cat/segments
/_cat/segments/{index}
/_cat/count
/_cat/count/{index}
/_cat/recovery
/_cat/recovery/{index}
/_cat/health
/_cat/pending_tasks
/_cat/aliases
/_cat/aliases/{alias}
/_cat/thread_pool
/_cat/thread_pool/{thread_pools}
/_cat/plugins
/_cat/fielddata
/_cat/fielddata/{fields}
/_cat/nodeattrs
/_cat/repositories
/_cat/snapshots/{repository}
/_cat/templates

2. put/post 新增和修改二合一

注意下划线开始的属性是元数据

2.1 带ID的PUT和POST

2.1.1 带ID的put

PUT customer/_doc/1
{
  "name": "niewj",
  "age": 20,
  "gender": "male"
}

连续两次请求结果:

{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 21,
  "result" : "updated",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 24,
  "_primary_term" : 1
}
---
{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 22,
  "result" : "updated",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 25,
  "_primary_term" : 1
}

可见: _seq_no和version都出现了新增, 每次都是 updated;

2.1.2 带ID的post跟put一样

2.2 不带ID的put=报错405

put customer/_doc/
{
  "name": "niewj",
  "age": 20,
  "gender": "male"
}

报错405:

{
  "error": "Incorrect HTTP method for uri [/customer/_doc/?pretty] and method [PUT], allowed: [POST]",
  "status": 405
}

2.3 不带ID的post=每次生成随机新ID

POST customer/_doc/
{
  "name": "niewj",
  "age": 20,
  "gender": "male"
}

连续两次请求结果: 每次都可以成功

{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "BvNkUHUB7InhghdEZD0I",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 26,
  "_primary_term" : 1
}
---
{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "B_NkUHUB7InhghdEqD3G",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 27,
  "_primary_term" : 1
}

但是每次都生成不同的”_id”, 也就是说: 每次都是新的数据!

3. POST+_update的更新: 检查更新

#POST customer/_doc/1/_update
POST customer/_update/1
{
  "doc":{
    "name": "niewj",
    "age": 20,
    "gender": "male"
  }
}

注意:

#POST customer/_doc/1/_update 这种方式不再推荐(因为type机制移除的原因), 如果使用上面的

POST customer/_doc/1/_update: 会提示:
#! Deprecation: [types removal] Specifying types in document update requests is deprecated, use the endpoint /{index}/_update/{id} instead.

POST customer/_update/1 连续多次请求的结果:

{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 22,
  "result" : "noop",
  "_shards" : {
    "total" : 0,
    "successful" : 0,
    "failed" : 0
  },
  "_seq_no" : 25,
  "_primary_term" : 1
}
---
{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 22,
  "result" : "noop",
  "_shards" : {
    "total" : 0,
    "successful" : 0,
    "failed" : 0
  },
  "_seq_no" : 25,
  "_primary_term" : 1
}

可见, 多次请求无变化, post+_update的方式会检查内容的变化, 如果无变化, 则不更新, 有变化才更新; 单纯post和put的方式, 是会每次更新的, 不检查内容; 另外需要注意的是 post+_update的调用方式, 需要把数据放在 doc结构里:

POST customer/_update/1
{
  "doc":{
    "name": "niewj",
    "age": 20,
    "gender": "male"
  }
}

4. get查询数据

4.1 指定ID的查询:

GET customer/_doc/1

查询结果:

{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 22,
  "_seq_no" : 25,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "name" : "niewj",
    "age" : 20,
    "gender" : "male"
  }
}

4.2 直接GET索引名:返回的是索引的定义信息

GET customer

查询结果:

{
  "customer" : {
    "aliases" : { },
    "mappings" : {
      "properties" : {
        "age" : {
          "type" : "long"
        },
        "gender" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "name" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "creation_date" : "1602493864109",
        "number_of_shards" : "1",
        "number_of_replicas" : "1",
        "uuid" : "j5DyzqgTTnG9aWRnAgHVmA",
        "version" : {
          "created" : "7040299"
        },
        "provided_name" : "customer"
      }
    }
  }
}

包括: mappings/alias/settings, 其中 mappings包括properties, 它的内容就是文档数据的字段类型的说明:

age/gender/name等索引字段的定义说明

5. 乐观锁和并发修改:_seq_no和_primary_term

5.1 乐观锁字段说明

_seq_no: 文档版本号, 作用同version

_primary_term: 文档所在位置

我们在并发修改的时候, 指定文档序列号(_seq_no):

5.2 并发修改模拟

我们先查询看看:

GET customer/_doc/1

数据:

{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 22,
  "_seq_no" : 25,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "name" : "niewj",
    "age" : 20,
    "gender" : "male"
  }
}

并发修改时:

PUT customer/_doc/1?if_seq_no=25&if_primary_term=1
{
  "name": "jack",
  "age": 22
}

结果:

{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 23,
  "result" : "updated",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 29,
  "_primary_term" : 1
}

在此查询看结果:

GET customer/_doc/1

结果:

{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 23,
  "_seq_no" : 29,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "name" : "jack",
    "age" : 22
  }
}

文档已更新;(注意是全量更新)

此时如果再次调用, 就会报错, 因为_seq_no已经发生了改变:

{
  "error": {
    "root_cause": [
      {
        "type": "version_conflict_engine_exception",
        "reason": "[1]: version conflict, required seqNo [25], primary term [1]. current document has seqNo [29] and primary term [1]",
        "index_uuid": "j5DyzqgTTnG9aWRnAgHVmA",
        "shard": "0",
        "index": "customer"
      }
    ],
    "type": "version_conflict_engine_exception",
    "reason": "[1]: version conflict, required seqNo [25], primary term [1]. current document has seqNo [29] and primary term [1]",
    "index_uuid": "j5DyzqgTTnG9aWRnAgHVmA",
    "shard": "0",
    "index": "customer"
  },
  "status": 409
}

6. 删除文档和索引

6.1 删除document

DELETE customer/_doc/1

结果:

{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 24,
  "result" : "deleted",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 30,
  "_primary_term" : 1
}
---再次删除时会:
{
  "_index" : "customer",
  "_type" : "_doc",
  "_id" : "1",
  "_version" : 1,
  "result" : "not_found",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 31,
  "_primary_term" : 1
}

第一次调用会成功删除deleted, 再次调用就: not_found

6.2 删除index

es去掉了type的概念之后(其实之前也没有删除type的机制), 就更明了了:

删除索引:

DELETE customer

result:

{
  "acknowledged" : true
}
---再次调用时报错:
{
  "error" : {
    "root_cause" : [
      {
        "type" : "index_not_found_exception",
        "reason" : "no such index [customer]",
        "resource.type" : "index_or_alias",
        "resource.id" : "customer",
        "index_uuid" : "_na_",
        "index" : "customer"
      }
    ],
    "type" : "index_not_found_exception",
    "reason" : "no such index [customer]",
    "resource.type" : "index_or_alias",
    "resource.id" : "customer",
    "index_uuid" : "_na_",
    "index" : "customer"
  },
  "status" : 404
}

第一次删除索引时成功acknowledged=true, ;再次删除因为没有了, 所以报错!

7. bulk批量导入数据:post+_bulk

7.1 批量执行3个文档的插入:

POST customer/_bulk
{"index": {"_id": "1"}}
{"name": "niewj"}
{"index": {"_id": "2"}}
{"name": "jack"}
{"index": {"_id": "5"}}
{"name": "michel"}

第一行是元信息, 随后的一行是具体的数据, 此处的操作是, 批量执行3个索引文档的操作, 每个操作有自己独立的结果item信息:

{
  "took" : 6,
  "errors" : false,
  "items" : [
    {
      "index" : {
        "_index" : "customer",
        "_type" : "_doc",
        "_id" : "1",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 9,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "customer",
        "_type" : "_doc",
        "_id" : "2",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 10,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "customer",
        "_type" : "_doc",
        "_id" : "5",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 11,
        "_primary_term" : 1,
        "status" : 201
      }
    }
  ]
}

每个item是有各自的操作结果的! -> “result” : “created”

7.2 混合的批量操作

POST customer/_bulk
{"delete": {"_id": "1"}}
{"delete": {"_id": "2"}}
{"delete": {"_id": "5"}}
{"index": {"_id": "1"}}
{"name": "niewj01"}
{"index": {"_id": "2"}}
{"name": "niewj02"}

混合的批量操作: 3个delete文档的操作, 2个索引文档的操作, 所以这个结果应该有5个items元素:

{
  "took" : 28,
  "errors" : false,
  "items" : [
    {
      "delete" : {
        "_index" : "customer",
        "_type" : "_doc",
        "_id" : "1",
        "_version" : 2,
        "result" : "deleted",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 12,
        "_primary_term" : 1,
        "status" : 200
      }
    },
    {
      "delete" : {
        "_index" : "customer",
        "_type" : "_doc",
        "_id" : "2",
        "_version" : 2,
        "result" : "deleted",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 13,
        "_primary_term" : 1,
        "status" : 200
      }
    },
    {
      "delete" : {
        "_index" : "customer",
        "_type" : "_doc",
        "_id" : "5",
        "_version" : 2,
        "result" : "deleted",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 14,
        "_primary_term" : 1,
        "status" : 200
      }
    },
    {
      "index" : {
        "_index" : "customer",
        "_type" : "_doc",
        "_id" : "1",
        "_version" : 3,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 15,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "customer",
        "_type" : "_doc",
        "_id" : "2",
        "_version" : 3,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 16,
        "_primary_term" : 1,
        "status" : 201
      }
    }
  ]
}

上面的批量操作都是在一个index下的, 其实也可以不指定index, 在元信息中指定, 这样, 可以对多个文档进行批量操作, 具体操作略~

7.3 批量初始化官网的accounts.json案例数据

accounts.json数据内容

POST bank/_bulk
{"index":{"_id":"938"}}
{"account_number":938,"balance":9597,"firstname":"Sharron","lastname":"Santos","age":40,"gender":"F","address":"215 Matthews Place","employer":"Zenco","email":"sharronsantos@zenco.com","city":"Wattsville","state":"VT"}
{"index":{"_id":"940"}}
{"account_number":940,"balance":23285,"firstname":"Melinda","lastname":"Mendoza","age":38,"gender":"M","address":"806 Kossuth Place","employer":"Kneedles","email":"melindamendoza@kneedles.com","city":"Coaldale","state":"OK"}
......数据的内容粘贴

结果:

{
  "took" : 612,
  "errors" : false,
  "items" : [
   {
      "index" : {
        "_index" : "bank",
        "_type" : "_doc",
        "_id" : "1",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 0,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "bank",
        "_type" : "_doc",
        "_id" : "6",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 1,
        "_primary_term" : 1,
        "status" : 201
      }
    },......
]}

转载请注明来源，欢迎对文章中的引用来源进行考证，欢迎指出任何有错误或不够清晰的表达。可以在下面评论区评论，也可以邮件至 hi@niewj.com