Patrol Groups and Patrols - aivanzhang/panda_patrol GitHub Wiki
Patrol groups are used to organize related patrols. They are context managers that you use to create patrols. To use this library, you must first create a patrol group before defining data tests. Patrol groups are created using the patrol_group
method. This method takes the following parameters:
-
group_name: str
- Name of the patrol group. This name should be unique to the patrol group. -
dbt_model_uri=None: str
(Default:None
. For DBT use only) - The dbt model uri of the Python model that is being tested. For examplemodels/base/first_model.py
would bebase/first_model
. This is used to link the data tests to the dbt model so that Python code related to a DBT test is stored. If excluded, then the data test code will not be stored in the database.
The patrol_group
method returns a patrol decorator that can then be used for tagging data tests.
# Import the patrol_group method
from panda_patrol.patrols import patrol_group
...
# Create a patrol group and return the patrol decorator for tagging data tests in this group
with patrol_group(GROUP_NAME) as patrol:
# Define data tests here
...
Patrols are decorators that are used to tag and specify data tests. This decorator takes the following parameters:
-
name: str
- The name of the patrol. This name should be unique to the patrol group. -
severity: Severity
(Default:Severity.CRITICAL
) - The severity of the patrol. -
store_logs: bool
(Default:True
): A boolean that determines whether or not to store the logs of the data test. IfTrue
then all the logs of a data test will be stored. IfFalse
then no logs are stored.
Data tests are run automatically at the end of the context after all data tests have been defined and tagged. In order to determine if this test should execute, a backend request checks if the test is silenced (this is defined via the Frontend). If the test is silenced, then the test will not execute. The return value (which needs to be convertible to a string), exception, traceback (if any) are stored in the database. Logs are also stored unless explicitly prohibited. The code related to the data test is also stored in the database. This allows you to view the code of the data test that was run. Finally a data test is considered successful if no exceptions are raised. If a test fails, then a backend request checks if the test has alerting enabled and if so checks if the test is assigned to an user (also defined via the Frontend). If alerting is enabled and it's assigned to a user, then an alert is sent to the user and/or Slack channel.
# Import the Severity enum
from panda_patrol.data.patrol_result import Severity
# Import the patrol_group method
from panda_patrol.patrols import patrol_group
...
# Create a patrol group and return the patrol decorator for tagging data tests in this group
with patrol_group(GROUP_NAME) as patrol:
# The data test is named PATROL_NAME, has a severity of Severity.INFO, and stores logs.
@patrol(PATROL_NAME, severity=Severity.INFO, store_logs=True)
def DATA_TEST_NAME():
print("Hello World") # This log is stored in the database
# Test data. Throw an exception if the test fails.
...
return "value" # This return value is stored in the database and tracked on the frontend
...