Blog Detail

Covid-19 Tracker Ask Question

preview image Programming
by Anurag Srivastava, Aug 4, 2018, 7:02:21 AM | 5 minutes |

Basics of Data Search in Elasticsearch

Day by day the volume of data is increasing as we are moving towards the digital age and transforming things into the Internet of things. Take a simple example of smartwatch what it does, it measures the steps, BP, heart rate, etc and push the data to a server from where we can access our health-related metrics. In the same way, there are different smart devices that keep on sending the regular data which is stored on a server. We are dumping lots and lots of data on servers which is there to help us find a trend, analyze them through data science to solve some serious problems or apply machine learning algorithms to forecast the future trend.

Data is important and we should capture them regularly to get meaningful information out of it. Now the question arrives how? What is best way to store and search data? Traditionally we were storing data in a RDBMS system and fetching them by using SQL queries but now things have changed because we want quick search response. We have no time to wait for a search result by seeing the loading icon moving in a direction. Another issue which we were facing was the uncertainty of data format, size and whether it is structured or unstructured? 

Now move to the search part as this blog is here to introduce you about the basic Elasticsearch query. It will explain the basics of query construction so that a novice person can install, index and search data from an Elasticsearch cluster. Elasticsearch is primarily used for its search capabilities and the Elastic-stack which can be applied to any set of applications to boost the performance and monitoring capabilities.  So let's start the process and learn the basic search query construction in Elasticsearch.

We have basically two types of search APIs in Elasticsearch: 'request URI based' and 'request body-based'. 


- In REST request URI we use to pass the search criteria in the URL itself like:

GET /blogs/technical/_search?q=topic:kibana

- In REST request body we use to construct the search block and write the search query inside the query block of Elasticsearch like:

GET /blogs/technical/_search
{
  "query": {
    "term": {
      "topic":"kibana"     
    }
  }
}

So the URI based search is quite a basic search where we just want to search a keyword whereas in request body we can construct the complex queries. So we have the query language to work with request body based searches. In this blog, I am not going into details to keep it simple so that everyone can understand what is going on.

The fielddata is disabled on text fields by default in Elasticsearch so we need to enable it for constructing the queries.

PUT blogs/_mapping/technical?update_all_types
{
  "properties": {
    "topic": {
      "type": "text",
      "fielddata": true
    }
  }
}

Match All Queries:

Now let's understand the basics of the query language, where first comes the match_all query:

GET /blogs/technical/_search
{
  "query": { "match_all": {} }
}

In match_all query, Elasticsearch returns all the documents. So this Elasticsearch query is basically like SQL "select * from technical" query.

Apply Limit:

Now we are going to set the offset and limit in a query to restrict the records, like:

GET /blogs/technical/_search
{
  "query": { "match_all": {} },
  "from": 1,
  "size": 5
}

In the above query, I am fetching 5 documents starting from the second one. In the same way, we can set the offset and limit in any Elasticsearch query.

Sort the results:

In Elasticsearch we can sort the documents as per our requirement like:

GET /blogs/technical/_search
{
  "query": { "match_all": {} },
  "sort": { "topic": { "order": "desc" } }

}

In the above expression, we are applying the ordering on the field topic.

Field Selection:

We limit the number of columns in SQL select queries, in the same way, we can do it in Elasticsearch queries, like:

GET /blogs/technical/_search
{
  "query": { "match_all": {} },
  "_source": ["category"]
}

In the above query, we will only get the category field in search results and the topic filed would not be displayed.

Match Queries:

We can run the match queries against the field name, like:

GET /blogs/technical/_search
{
  "query": {
    "match": {
      "topic": "kibana"
    }
  }
}

In the above query, we can pass the text to search against the topic field.

In this blog, I have explained the basics of Elasticsearch query construction. I the next blog I will cover filters, boolean queries, wildcard queries, etc and then will explain aggregation and its usage.

Other Blogs on Elastic Stack:
Introduction to Elasticsearch

Elasticsearch Installation and Configuration on Ubuntu 14.04
Log analysis with Elastic stack 
Elasticsearch Rest API
Basics of Data Search in Elasticsearch
Elasticsearch Rest API
Wildcard and Boolean Search in Elasticsearch
Configure Logstash to push MySQL data into Elasticsearch 
Metrics Aggregation in Elasticsearch
Bucket Aggregation in Elasticsearch
How to create Elasticsearch Cluster


If you found this article interesting, then you can explore “Mastering Kibana 6.0”, “Kibana 7 Quick Start Guide”, “Learning Kibana 7”, and “Elasticsearch 7 Quick Start Guide” books to get more insight about Elastic Stack, how to perform data analysis, and how you can create dashboards for key performance indicators using Kibana.


You can also follow me on:

- LinkedIn: https://www.linkedin.com/in/anubioinfo/

- Twitter: https://twitter.com/anu4udilse

- Medium: https://anubioinfo.medium.com



Comments (0)

Leave a comment

Related Blogs

Create Sequence Diagrams using PlantUML

Jun 27, 2021, 12:50:31 PM | Anurag Srivastava

Improving your productivity on Linux Terminal

Nov 29, 2020, 5:16:40 PM | Anurag Srivastava

Elastic Stack Interview Questions

Sep 12, 2020, 3:58:55 PM | Anurag Srivastava

Introduction to Kibana

Aug 1, 2020, 6:19:45 PM | Anurag Srivastava

Create a Chess board in PHP

Mar 9, 2020, 8:45:41 AM | Rocky Paul

Handling Excel file using Python (Part 2)

Dec 31, 2019, 1:33:53 PM | Anurag Srivastava

Top Blogs

Wildcard and Boolean Search in Elasticsearch

Aug 10, 2018, 7:14:40 PM | Anurag Srivastava

Elasticsearch REST APIs

Jul 31, 2018, 6:16:42 PM | Anurag Srivastava