Logs help answer 5 W's

1. What happened?
2. Why did it happen?
3. When did it happen?
4. Where did it happen?
5. How did it happen?

You want to be able to collect, process, and analyze your logs. You can use the information contained within them to track and investigate performance, as well as detect potential and ongoing threats and attacks.

Datadog decouples Log Ingestion and Log Indexing. You can cost-effectively ingest all of your logs and only index the logs you need for day-to-day analytics, monitoring, and investigations.

Log Explorer is your home base for querying and analyzing your indexed and ingested logs.

In Log Explorer, you can query and analyze indexed logs. You can also view log details. Indexed logs can be used in monitors, dashboards, and notebooks.

In the Live Search tab (shown below), you can query near real-time ingested logs and view log details. Ingested logs can be used for Watchdog (automated) Insights, Error Tracking, generating metrics, and Cloud SIEM detection rules.

The layout of the Log Explorer is shown below.

1. In the center, you'll see the streaming list of log events or the visualization for an aggregation of log events.
2. Near the top, there is a search field to filter log data and "Group into" options to aggregate and visualize log events.
3. On the left is the facets list. Facets are tags and attributes that can be used to query and analyze your log events.
4. In the top right, you can use the time picker to set the desired time range over which you're viewing the log data.
5. Below the search and visualization options, the Timeline Graph displays the count of log events. You can hover over the bars in the graph to see the count indicated by each bar, and you can highlight a region of this graph to focus on that time range.
6. On the right side above the Log List, you have export and column display options.
7. In the top left, you can open Views for the list of saved views. These are search queries and time ranges that you have saved for future reference.

Tags vs Attributes

Tags are key:value pairs that are assigned at the host or container level to provide metadata about the logs. Examples include env:prod, service:webstore, and app:rails-storefront.

Attributes are details that Datadog extracts from your logs during ingestion and processing. Attributes are listed in JSON format below the Event Attributes tab in the Log Details. They also have a key:value pair format. Examples include action:checkout, http.method:POST, and network.bytes_written:1024.

Syntax Basics

Tag

• format - key :value
• example - service: weblogic

attribute

• format - @key:value
• example - @http.method: POST

single term

• format - word
• example - Response

sequence

• format - group of words surrounded by double quotes
• example: "Response Fetched"

Full-text searches

Search text - *:prod

• Description - Searches all log attributes ( tag ) with the value prod

Search text - :prod

• Description - Searches all log attributes ( tag ) for strings that starts with prod

Search text - *:"product inventory"

• Description - Searches all log attributes (attributes ) with the value "product-inventory"
• example : @inventory.message:"Item low. Check product inventory." and @task.name:"Update product inventory"

Search text - prod

• Description - Searches only the log message for the term prod

Search text - prod*

• Description - Searches all log messages for strings that starts with prod

Operator

AND

• Description - Intersection: both terms are in the selected events
• Example - "authentication AND failure"

OR

• Description - Union: either term is contained in the selected events
• Example - "authentication OR password" , service:(store-ads OR store-discounts)

-

• Description - Exclusion: the following term is NOT in the event (apply to each individual raw text search)
• Example - "authentication AND -password"

sequence

• Description - group of words surrounded by double quotes
• Example - "hello world"

Facet and Measures

You must convert tags and attributes into facets and measures in Log Explorer in order to use them to query logs in the following:

• Log Analytics (Aggregations)
• Log Monitors
• Dashboards
• Notebooks

Visualizations

Timeseries

• Graph of the counts of logs or statistics for a single measure over a selected time frame based on the search query. You can optionally split by up to three available facets

Top List

• List of the top values of counts or statistics based on a search query over the selected time frame.

Table

• List of the counts or statistics based on a search query over the selected time frame.

Tree Map

• Organize and display data in nested rectangles to show percentage as a whole.

Pie Chart

• Organize and graph data as a percentage of a whole.

Event Attribute lists all attributes from that log that were either in JSON format when the log was ingested or were parsed from the log during processing after ingestion.

**Trace **shows the trace associated with the log, if log and trace correlation is configured. From here, you have the option to View Trace in APM.

**Metrics **displays Host and Container metrics associated with the logs. If you click any graph, you'll see options to view related data and dashboards in Datadog.

Processes are related processes running on the host and container.

To learn how to set up monitors with logs, use error tracking, generate metrics, and display log-based data in dashboards, take the Track Logs for Alerting and Investigations course.

To learn how to start processing logs to extract attributes and enrich logs details so that you can query logs from all of your sources, take the Process Logs Out of the Box with Integration Pipelines course.

Additional resources The Datadog documentation, blog, and YouTube channel are rich resources that can help you learn more about Datadog.