PythonCookBook - henk52/knowledgesharing GitHub Wiki

Python cookbook

Introduction

Vocabulary

  • hashmap
    • In Python, “hashmap” and “dictionary” refer to the same data structure.
  • dictionary - data type is implemented as a hash table, which allows for fast access to values using keys.
    • Dictionaries are used to store key-value pairs, where each key must be unique and immutable.

References

General

Hello world

#!/usr/bin/env python3

"""This script will ..."""

def main():
    """Main function"""
    print("Hello world")
  
if __name__== "__main__":
  main()

Documenting the code

"""A simple function that says hello... Richie style"""

Functions

See: 

def WriteKey(filename, Oentry):
  filehandle = open(filename, "w")
  filehandle.write("{\n")
  filehandle.close()

Command line parameters

import argparse

jsonFile = ''

parser = argparse.ArgumentParser(description='read given json file.')
parser.add_argument('--json', dest='jsonFile', action='store',
                    default='statistics.json',
                    help='json file')

args = parser.parse_args()

print(args.jsonFile)

Basic types

String operations

def _removeNonAscii(s): return "".join(i for i in s if ord(i)<128)

Codecs

String to float

fTemp = 20.3 + float(fields[0])

Format string

The number before the ':' is the zero base index to the list in the 'format()' temperature_message = '{0:s}.temperature {1:0.1f}C {2:0.1f}%\n'.format(szHostname, fTemp, fHumity)

concatenate strings

See:

get substring

# Start text at 7th char and end 2 before end
# [TEST] some string..
# test = 'some string'
test = line[7:-2]

split string

See: 

x = ‘blue,red,green’
x.split(",")

>>> x 
[‘blue’, ‘red’, ‘green’]

a,b,c = x.split(",")
 
>>> a
‘blue’

For split vs partition, see: partition(): Split the string at the first occurrence of sep, and return a 3-tuple containing the part before the separator, the separator itself, and the part after the separator.

Pattern matching and extraction

# https://docs.python.org/2/library/re.html
import re
        # (PDH-CSV 4.0) (UTC)(0)
        szVersionAndTimeZone = arRawHeader.pop(0)
        #arMatches = re.search('\((.+?)\).*', szVersionAndTimeZone)
        arMatches = re.search('\((.+?)\)\s+\((.+?)\)\((.+?)\)', szVersionAndTimeZone)
        # index 0 seems to be the whole string.

        # PDH-CSV 4.0
        szEsxTopCsvVersion = arMatches.group(1)
        # UTC
        szTimeZone = arMatches.group(2)
        nOffsetMaybe = arMatches.group(3)

r = requests.get('http://172.17.0.3:9568/metrics')
        metricLines = r.text.split("\n")
        for line in metricLines:
            # perf_node_packets_clientside_client_deltatime_maximum{server="172.17.0.2"} 14
            arMatches = re.match('perf_node_packets_clientside_client_deltatime_maximum\{(.+?)\} (\d+)', line)
            if arMatches is not None:
                if ( int(arMatches.group(2)) > 10 ):
                    # server="172.17.0.2" 14
                    print(arMatches.group(1), arMatches.group(2))

Array operations

See:

Initialize/create an empty array

arNiceHeaderList = []

Return and remove first element

szVersionAndTimeZone = arRawHeader.pop(0)

Get length of array/list

len(list)

Iterate through an array

See: 

for line in outputLines:
    print(line)

platform

Date/Time

get epoch

import calendar
import time
ts = calendar.timegm(time.gmtime())

Convert from string to epoch

timestampString = data['timestamp']
# https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior
# 2020-04-01 10:11:48.462247
datetime_object = datetime.strptime(timestampString, '%Y-%m-%d %H:%M:%S.%f')
print(round(datetime_object.timestamp()))

# 02/25/2019 13:31:36
    arDateTimeMatches = re.search('(\d+)/(\d+)/(\d+)\s+(\d+):(\d+):(\d+)', szDateTime)
    timestamp = datetime.datetime(int(arDateTimeMatches.group(3)), int(arDateTimeMatches.group(1)), int(arDateTimeMatches.group(2)), int(arDateTimeMatches.group(4)), int(arDateTimeMatches.group(5)),int(arDateTimeMatches.group(6))).strftime('%s')

Delta dates

Calculate date X days ago

from datetime import datetime, timedelta

daysToKeep = 2

timestampNDaysAgo = datetime.now() - timedelta(days=daysToKeep)

print(datetime.now())
print(timestampNDaysAgo)

Calculate delta between two dates

from datetime import date

f_date = date(2014, 7, 2)
l_date = date(2014, 7, 11)
delta = l_date - f_date
print(delta.days)

Modules

sudo apt install python3-pip

  • pip3 install
  • pip3 show
  • pip3 uninstall

Something intelligent here

Default installation paths

  • /usr/lib/python2.7
  • /usr/local/python2.7

Show where the module is loaded from

import inspect
from m import f

print inspect.getmodule(f)

Show where a module is installed

pip show azure-identity

How to find which pip package owns a file?

See: 

pip list | tail -n +3 | cut -d" " -f1 | xargs pip show -f | grep "filename"

OS

OS I/O

Handle output from sub process

import subprocess

listOfSystemName = subprocess.run(['grep', 'system:', 'some_log_file'], stdout=subprocess.PIPE)
commandOutput = listOfSystemName.stdout.decode('utf-8')
firstLine = commandOutput.partition('\n')[0]

Get list of files in directory

import os

dir_path = './test'

def main():
    for path in os.listdir(dir_path):
        # check if current path is a file
        if os.path.isfile(os.path.join(dir_path, path)):
           print(path)

File

Text file

Write as UTF: CSV_FILE.write(sCsvLine.encode('utf8'))

  filehandle = open(filename, "w")
  filehandle.write("{\n")
  filehandle.write("}\n")
  filehandle.close()

  filehandle = open(filename, "r")
  line = filehandle.readline()
  while(line != ''):
      print("DDD line: ", line)
      line = filehandle.readline()

  filehandle.close()

See also:

get list of files in directory

See: [](

from os import listdir
from os.path import isfile, join
onlyfiles = [f for f in listdir(mypath) if isfile(join(mypath, f))]

CSV

Read CSV file

        # esxtopData is an iterator
        esxtopData = csv.reader(csvfile, quotechar='"')
        #   https://docs.python.org/3.1/tutorial/datastructures.html
        # Removes the first element in the list
        arRawHeader = esxtopData.next()

Iterate through CSV file

        for row in esxtopData:
            print ': '.join(row)

XML

  • Built in XML lib:

Json

Read json file

selecting-fields-from-json-output

import json
jsonFile = open('your_filename.json', 'r')
values = json.load(jsonFile)
jsonFile.close()

Access loaded json structure

idValue = values['criteria'][0]['id']

{
    "accountWide": true,
    "criteria": [
        {
            "description": "some description",
            "id": 7553,
            "max": 1,
            "orderIndex": 0
        }
     ]
}

Web

import requests

r = requests.get('https://api.github.com/events')
print(r.text)

GUI

Tkinter

*TKinter