python logging - ghdrako/doc_snipets GitHub Wiki

import logging;
logging.basicConfig(filename='mylog.txt',level.logging.ERROR)
logging.methodname('message')
try:
  lst = [1,2,3]
  print(lst[3])
except Exception as e:
  logging.exception(e)

import logging
logger_good = logging.getLogger("task_app")   # not use constructor: logger_not_good = logging.Logger("task_app")

As a best practice, if you’re creating a module-level logger for each module in your application, I recommend that you create the logger by running logging.getLogger (name) . name is a special attribute for a module name. When you name the module taskier.py , for example, the module’s name attribute is taskier

Always use getLogger to retrieve the same logger for your module or application. For module-level loggers, it’s best to use getLogger (name) to get the logger.

logger = logging.getLogger(__name__)
file_handler = logging.FileHandler("taskier.log")
logger.addHandler(file_handler)

Add stream loger to see message in console

stream_handler = logging.StreamHandler()
logger.addHandler(stream_handler)
logger.warning("Just a random warning event.")
# output the following: Just a random warning event.

In Python’s logging module, we have access to five levels ( DEBUG , INFO , WARNING , ERROR , and CRITICAL ) plus a base level ( NOTSET ), which has a numeric value of 0 and isn’t typically used.

Set logging level

logger = logging.getLogger(__name__)
logger.setLevel(logging.WARNING)

Setting a handler’s level

logger.setLevel(logging.DEBUG)                                  # Sets the logger’s level to DEBUG
handler_warning = logging.FileHandler("taskier_warning.log")
handler_warning.setLevel(logging.WARNING)                       # Adds a handler at the WARNING level
logger.addHandler(handler_warning)
handler_critical = logging.FileHandler("taskier_critical.log")
handler_critical.setLevel(logging.CRITICAL)                     # Adds a handler at the CRITICAL level
logger.addHandler(handler_critical)
logging_messages_all_levels()
warning_log_records = check_log_content("taskier_warning.log")
print(warning_log_records)
# output the following lines:
--Critical message
--Error message
--Warning message
critical_log_records = check_log_content("taskier_critical.log")
print(critical_log_records)
# output the following line:
--Critical message

import logging
# Generate two logs with different severity levels
logging.warning('This is a warning message')
logging.info('This is an info message')

The logs are, by default, routed to stdout

Three basic elements used for logging in Python:

A formatter, which describes how the full log is going to be presented, attaching metadata like the timestamp or the severity.
A handler, which decides how the logs are propagated. It sets the format of the logs through the formatter, as defined above.
A logger, which produces the logs. It has one or more handlers that describe how the logs are propagated.

import sys
import logging
# Define the format
FORMAT = '%(asctime)s.%(msecs)dZ:APP:%(name)s:%(levelname)s:%(message)s'
formatter = logging.Formatter(FORMAT, datefmt="%Y-%m-%dT%H:%M:%S")
# Create a handler that sends the logs to stdout
handler = logging.StreamHandler(stream=sys.stdout)
handler.setFormatter(formatter)
# Create a logger with name 'mylogger', adding the handler and setting
# the level to INFO
logger = logging.getLogger('mylogger')
logger.addHandler(handler)
logger.setLevel(logging.INFO)
# Generate three logs
logger.warning('This is a warning message')
logger.info('This is an info message')
logger.debug('This is a debug message, not to be displayed')

Detecting expected errors

import logging
import requests
URL = 'https://httpbin.org/status/500'
response = requests.get(URL)
status_code = response.status_code
if status_code != 200:
    logging.error(f'Error accessing {URL} status code {status_code}')

he https://httpbin.org/ service, a simple HTTP request and response service that can be used to test code. In particular, the https://httpbin.org/status/ endpoint returns the specified status code, making it easy to generate errors.

Capturing unexpected errors

import logging
import requests
URL = 'https://httpbin.org/status/500'
logging.info(f'GET {URL}')
response = requests.ge(URL)
status_code = response.status_code
if status_code != 200:
    logging.error(f'Error accessing {URL} status code {status_code}')

By default in Python, it will show the error and stack trace in the stdout. When the code is executed as part of a web server, this is sometimes enough to send these messages as ERROR logs, depending on how the configuration is set up.

If you need to create a script that needs to be running endlessly and is protected against any unexpected errors, be sure to use a try..except block as it's generic, so any possible exception will be captured and handled.

import logging
import requests
from time import sleep
logger = logging.getLogger()
logger.setLevel(logging.INFO)
while True:
    
    try:
        sleep(3)
        logging.info('--- New request ---')
    
        URL = 'https://httpbin.org/status/500'
        logging.info(f'GET {URL}')
        response = requests.ge(URL)
        scode = response.status_code
        if scode != 200:
            logger.error(f'Error accessing {URL} status code {scode}')
    except Exception as err:
        logger.exception(f'ERROR {err}')

while True:
    try:
        code
    except Exception as err:
        logger.exception(f'ERROR {err}')

The try..except block is inside the loop, so even if there's an error, the loop will be uninterrupted. If there's any error, except Exception will capture it, no matter what the exception is.

Any unexpected error should be logged as ERROR

Log strategies

Log level	Action to take	Comments
DEBUG	None.	Not tracked. Only useful while developing.
INFO	None.	INFO logs show generic information about the flow of the actions in the app to help track systems.
WARNING	Track the number of logs. Alert on raising levels.	WARNING logs track errors that are automatically fixed, like retries to connect to an external service, or fixable format errors in a database. A sudden increase may require investigation.
ERROR	Track the number of logs. Alert on raising levels. Review all errors.	ERROR logs track errors that can't be recovered. A sudden increase may require immediate action. All of them should be periodically reviewed to fix common occurrences and mitigate them, perhaps moving them to WARNING level.
CRITICAL	Immediate response.	CRITICAL logs indicate a catastrophic failure in the application. A single one indicates the system is not working at all and can't recover.

In web applications, as a rule of thumb, ERROR logs should only be created when the status code is one of the 50X variants (like 500, 502, and 503). Remember that the 40X errors mean that the sender has a problem, while 50X means that the application has the problem, and it's your team's responsibility to fix it.

Handlers https://docs.python.org/3/howto/logging.html#useful-handlers.

StreamHandler - the destination of the stream to be sys.stdout
FileHandler to send the logs to a file
SysLogHandler to send logs to a syslog destination
TimeRotatingFileHandler, which rotates the logs based on time, meaning it stores the last defined time, and archives older versions.