Lab 2.0: Elasticsearch Queries - squatchulator/Tech-Journal GitHub Wiki

Lab 2 - Elasticsearch Queries

Creating Indices and Documents

  • To see current indexes on Elasticsearch, run curl -XGET '<public IP>:9200/_cat/indices?v&pretty'
    • Should be a ton of system-created ones starting with a ".", and the one we created in the last lab should be there as well.
  • Now, we want to make an "app" index with the user John with an id of 4. We can do that with the following:
curl -X PUT '<public IP>:9200/app/users/4' -H 'Content-Type: application/json' -d '
{
  "id": 4,
  "username": "john",
  "last_login": "2023-01-25 12:34:56"
}
'
  • Let's create an index for logs and add a log in for John using:
curl -XPOST '<public IP>:9200/logs/my_app' -H 'Content-Type: application/json' -d'
{
	"timestamp": "2023-01-24 12:34:56",
	"message": "User logged in",
	"user_id": 4,
	"admin": false
}
'
  • Check the indices again with curl -XGET '<public IP>:9200/_cat/indices?v&pretty' and you should see "log" and "app" now.
  • We are going to add more documents to the "app" index now. We'll add 2 more users:
curl -X PUT '<public IP>:9200/app/users/5' -H 'Content-Type: application/json' -d '
{
  "id": 5,
  "username": "jane",
  "last_login": "2023-01-25 12:34:56"
}
'
curl -X PUT '<public IP>:9200/app/users/6' -H 'Content-Type: application/json' -d '
{
  "id": 6,
  "username": "jim",
  "last_login": "2023-01-25 12:34:56"
}
'
  • We also need to add 2 more entries to the "log" index for these two users:
curl -XPOST '<public IP>:9200/logs/my_app' -H 'Content-Type: application/json' -d'
{
	"timestamp": "2023-01-24 12:34:56",
	"message": "User logged in",
	"user_id": 5,
	"admin": false
}
'
curl -XPOST '<public IP>:9200/logs/my_app' -H 'Content-Type: application/json' -d'
{
	"timestamp": "2023-01-24 12:34:56",
	"message": "User logged in",
	"user_id": 6,
	"admin": false
}
'

Elasticsearch API Queries

  • The data is now indexed so we should be able to search it. Use a URI search like this to get the doc for id 4: `curl -XGET ':9200/app/users/4?pretty'
  • We can also use GET to do searches by calling the _search API endpoint with the 'q' parameter, and it will return the documents with the word "logged" in it: curl -XGET '<public IP>:9200/_search?q=logged'
  • Another way to search is by using Lucene queries like this: curl -XGET '<public IP>:9200/_search?q=username:john&pretty'

Elasticsearch Query DSL

In addition to URI Searches, Elasticsearch also provides a request body search with a Query DSL for more advanced searches. There is a wide array of options available in these kinds of searches, and can mix and match different options to get results. Query DSL contains two kinds of clauses:

  • leaf query clauses that look for a value in a specific field, and
  • compound query clauses (which might contain one or several leaf query clauses).
  • To test Query DSL:
curl -XGET '<public IP>:9200/logs/_search?pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "match_phrase": {
      "message": "User logged in"
    }
  }
}
'
⚠️ **GitHub.com Fallback** ⚠️