Changelog ‐ Lineage drill‐down for issue root cause analysis - data-drift/data-drift GitHub Wiki

Understand and solve the root cause of metric issues. Without SQL archeology.

Metric issues are time-consuming to investigate

Time is a scarce resource for analytics team. Data issues impacting metrics do not help. They are hard and time-consuming to investigate.

Let’s take an example. A data consumer notices a 50% day-over-day decrease in free-to-paid conversion rate. A data analyst is asked to investigate. What makes data issues hard to investigate is that the problem can come from the modeling, or the data itself or both. To find the issue, the analyst will check recent pull requests merged and recent data updates. This involves checking multiple tables across the metric’s lineage to understand the root cause.

A simple example like this one easily takes half a day to investigate. In addition to costing time and money, it has a high opportunity cost, diverting money from higher ROI projects.

Simple root cause analysis with lineage drill-down

As analytics practitioners, we know spending time investigating data issues is frustrating. That’s why we are happy to introduce lineage drill-down for faster and painless root cause analysis.

Lineage drill-down works by adding a monitor where your metric is computed. For dbt users, this monitor is metadata added to your model. It then understands how your metric is computed and on which upstream tables it depends. When an issue occurs, it pinpoints exactly which rows have been updated and introducing the change.

Reduce your data issues time-to-resolution

As we said, root cause analysis of data issues is hard and time-consuming. Datadrift provides analytics team with an open-source solution for faster and simpler troubleshooting.

Find here the documentation to get started with lineage drill-down.