Logging - nstarman/utilipy GitHub Wiki

Logging

Logging can be hacked together when constructing a simple script: throw in a couple print statements, maybe save the output to a file, call it a day. However, as soon as the codebase becomes more complicated, espcially with functions spanning multiple files, logging becomes simultaneously essential and challenging. That's why utilipy offers easy-to-use and powerful logging functions which can easily be incorporated into functions and scripts. Making a legible logfile should be simple. Now it is.


Simple Scripts

Adding logging to a script is simple. First we will instantiate a logfile. Then, while doing stuff in the script, we will periodically report on what is going on. The best part is that the loggers can both print and save the output to a file. Now we can have a real-time account and a keepsake for posterity.

This is an example python script using utilipy's logging functions.

##########################################################################
# IMPORTS

from utilipy import LogFile  # import the standard logging 
... # other imports

##########################################################################
# SETUP

# setup the logger
# this logger will both print the output and save it to script.log
_LOGFILE = LogFile('script.log')

##########################################################################
# SCRIPT

... # do stuff
_LOGFILE.write('did stuff here')  # printing and saving


### Doing Other Stuff
_LOGFILE.newsection(title='Title')  # making a new section
... # do more stuff
_LOGFILE.write('did more stuff here')  # printing and saving

##########################################################################
# END

the printed output and saved logfile, script.log, will look like:

script.log Log:

===============================================================================

did stuff here

-------------------------------------------------------------------------------
Title 

did more stuff here

For simple scripts like this one, the same result can be accomplished with print statements. However, even at this most basic level we think that this logger method is cleaner and more readable than the print statement equivalent. The next sction, on functions, demonstrates the real power of LogFile.




Writing Functions with Logging

A Good Template

function_file.py

##########################################################################
# IMPORTS

from utilipy import LogFile  # import the standard logging 
... # other imports

##########################################################################
# SETUP

# instantiate a blank logfile to use as a default value
# this PrintLog can only print, not write to a file
_LOGFILE = PrintLog(show_header=False,  # prevent displaying 
                    verbose=0)          # set report verbosity level

##########################################################################
# CODE

# define a function, including a logger and verbosity
# the logger is defaulted to the printlog
# the verbose=None will use the default verbosity of the logger
def function(*args, logger=_LOGFILE, verbose=None):
    ...  # function does stuff here

    # reporting on the workings of the function
    logger.report('verbose = 1 message',
                  'verbose = 2 message',
                  'verbose >= 3 message',
                  verbose=verbose)

    return output
# /def

##########################################################################
# END

Now, function can give reports, with different levels of detail, on what it is doing.

Calling function without specifying a logger or a verbosity will neither print nor write to a file.

function()
>> # nothing

Calling function without specifying a logger, but giving a verbosity, will print but not write to a file.

function(verbose=1)
>> 'verbose = 1 message'  # not saved to file

Calling function with a logger will print and write to a file, unless the verbosity is 0 or the logger is a PrintLog, which only prints.

function(logger=LogFile('log.txt'), verbose=2)
>> 'verbose = 2 message'  # printed and saved

Here is an example script implementing LogFile.

script.py

##########################################################################
# IMPORTS
from utilipy import LogFile  # import the standard logging function
from .function_file import function

##########################################################################
# SETUP
# instantiate a real logfile and set a verbosity level
_LOGFILE = LogFile('name_of_log.log',  
                   verbose=1)

##########################################################################
# SCRIPT

# call the function with built-in verbosity (verbose=1)
output = function(arg1, arg2, ..., logger=_LOGFILE, verbose=None)

_LOGFILE.newsection('set verbosity')

# call the function with set verbosity
output = function(arg1, arg2, ..., logger=_LOGFILE, verbose=0)

output = function(arg1, arg2, ..., logger=_LOGFILE, verbose=10)

##########################################################################
# END

The output to this script, on both the console and saved file, is:

name_of_log.log Log:

===============================================================================

'verbose = 1 message'  # b/c verbosity = 1

-------------------------------------------------------------------------------
set verbosity

'verbose >= 3 message'

Note that the verbose=0 message was neither printed, nor recorded.





The Logging Functions

All the logging functions are designed to have identical methods, for guaranteed compatibility when passed between functions, even if those methods accomplish slightly things. For instance, PrintLog has a .write method even though it cannot write to a file. For PrinLog, this will print the output (whereas for LogFile it both prints and writes).

PrintLog

__init__

open

open_to_write

open_to_read

read_log

print

write

record

verbort

report

newsection

close

LogFile

Docstring

    a basic logger which can both print and record to a file
    ** this class uses `open', not a more extensive logger, like `logging'

    The arguments filename - opener are all for `open`
    their descriptions are in
        https://docs.python.org/3/library/functions.html#open

    Parameters
    ----------
    filename : str, optional
        the file name / path at which to save this log
        If no filename, makes a PrintLog() instead
    verbose : int
        the verbosity level to use in .report / .verbort 
    mode : str  (default 'w')
        recommend either 'w' or 'a'
    sec_div : str
        the section divider used in `newsection'
    header : None, str  (default None)
        the header for the file
        None -> filename
    show_header : bool  (default True)
        whether to print the header
    ...

    Notes
    -----
    mode options:
        'r' open for reading
        'w' open for writing, truncating the file first
        'x' open for exclusive creation, failing if the file already exists
        'a' open for writing, appending to the end of the file if it exists
        'b' binary mode
        't' text mode
         NOT ALLOWED '+' open a disk file for updating (reading and writing)

    Inherited Methods
    -----------------
    .print
    .newsection
    .write  (reimplemented for docstring)
    .record  (reimplemented for docstring)
    .verbort  (reimplemented for docstring)

    Overwritten Methods
    -------------------
    ._write: writes to file
    ._print_and_write: prints and writes

__init__:

initialize a FileLog with a filename

if no filename is provided, it switches itself to a PrintLog

if no header is provided it uses the filename + Log, if header is True, it just uses Log. If header is False, then no header is made.

all the arguments are given in the docstring. Most are for open and should be consulted accordingly.

open

open_to_write

open_to_read

read_log

print

write

record

verbort

report

newsection

close

utilipy wiki
code documentation at readthedocs.

⚠️ **GitHub.com Fallback** ⚠️