How Python runs the source code

When people talk about “compiled languages”, they often think about languages like C. In C, the process of running a program looks something like this (simplified):

you write the source code;

you use a compiler which turns your source code into machine language that the CPU of your computer understands;

every different family of CPUs uses a different machine language, so computer architectures (e.g. Windows versus Mac) needs a different compiler;

but once you have the compiled machine language, you no longer need either the source code or the compiler;

on the other hand, the compiled code will only run on one family of machines, so you need a separate Windows and Mac version of your program.

When people talk about “interpreted languages”, they usually are thinking of something like this:

you write the source code;

you use an interpreter which looks at the source code and directly executes the statements, without compiling it to machine language first;

so to run code in an interpreted language, you must have both the source code and the interpreter;

and a single program (the source code) will run on any machine with the right interpreter.

Python, like most languages for, oh, about forty years, combines elements of both:

you write the source code;

when you use the interpreter, it first compiles the source code into “byte code”;

unlike machine code, which is specific to a family of CPUs and machine architectures, the byte code is independent of what sort of machine you are using;

the interpreter then has a “virtual machine” which knows how to run the byte code; think of it as a simulated CPU;

like compiled languages, there is an intermediate form (the byte code) between what you write (source code) and what gets run by the actual CPU;

so you can compile Python code into a .pyc file and then run that compiled version instead of the source code;

but like interpreted languages, the byte code doesn’t run directly on the CPU of your computer; you still need the interpreter to run the byte code;

which means that the same .py or .pyc file can be used on any machine that has the right interpreter;

some Python interpreters, like PyPy, include a “Just In Time” compiler which turn parts of the byte-code into machine language as needed, for extra speed, without losing the advantages of having an interpreter.

These days, most modern languages have elements of both. Interpreters nearly always have an intermediate compiled form that runs in a virtual machine; compilers sometimes have a built-in interpreter.

Refer link

---

Ellipsis

---

CWD in Python

Current working directory in a file will not give the directory path of the file in which we use cwd. It will return the dir path where the execution happens. Use OS module for that

---

Parameter kind

Official link

Refer link

PEP 3102 – Keyword-Only Arguments defines the use of the asterisk in terms of an argument, while PEP 570 – Python Positional-Only Parameters defines the slash in terms of a parameter, although symbol also shows up briefly. PEP 570 also uses the words parameter and argument interchangeably.

Positional argument, Default argument, Variable length arguments, Named arguments, Variable length keyword arguments

If you want to call a function by passing one argument only with the name of the argument then u ca use the named argument.

def printf(val):
    print(val)

printf("Positional")
printf(val="Keyword argument")

What if you want to call the printf function with the keyword name, for this u can use named argument style.

def printf(*, val):
    print(val)

If you try to call this function like this printf("Positional") then you will get an error like this TypeError: printf() takes 0 positional arguments but 1 was given

Becoz we made this function to accept only keyword argument.

only this printf(val="Keyword argument") and this will work

def printf(*, val):
    print(val)

d = {"val": "Keyword argument"}
printf(**d)

Orders

def get_student(name, grade='Five', *args, age, **kwargs):
    print(name, age, grade)

---

Single underscore and double underscore

Dict keys - why not list

Because lists are mutable, dict keys need to be hashable, and hashing mutable objects is a bad idea because hash values should be computed on the basis of instance attributes.

Python doc

---

list.sort() sorted(iterable)

fabs

abs (built in) - positive number fabs (provided by math module) - positive floating point number

format specifier

formatted-string-literals

format-examples

a = "Logesh"

print(f'{a:20}')  # Logesh              
print(f'{a:<20}') # Logesh              
print(f'{a:_<20}')# Logesh______________
print(f'{a:>20}') #               Logesh
print(f'{a:^20}') #       Logesh       

a = 0.96349       
print(f'{a:.2%}') # 96.35%

a = 10000.242353  
print(f'{a:.3f}') # 10000.242
print(f'{a:,.2f}')# 10,000.24
print(f'{a:.0e}') # 1e+04

template string

Boolean type conversion

int(Boolean)

float(Boolean)

list(Boolean)

int attributes

dir()

globals()

The globals() method returns a dictionary with all the global variables and symbols for the current program.

The globals() method doesn't take any parameters.

locals()

The locals() method returns a dictionary with all the local variables and symbols for the current program.

The locals() method doesn't take any parameters.

help()

help gives the information about the python thing.

__name__ and __doc__

count words in doc

iter()

extended iter()

Extended iter and iter() are not the same. iter() takes iterable as the input whereas the extended iter(callable, end_match).

isinstance()

issubclass()

issubclass(ClassToChk, BaseClass))

Class

vars()

__dict__

Modules and Packages

Packages and Modules are the way to organize the related code.

A module typically corresponds to a single source file, and you load modules into programs by using the import keyword. When you import a module, it is represented by an object of type module, and you can interact with it like any other object.

A package in Python is just a special type of module.

Packages are generally represented by **directories **in the file system, while modules are represented by single files.

Here fastapi is the Package. fastapi.middleware.cors is Module.

sys.path

The Python looks for the imported module in the system path variable.

PYTHONPATH

It is an environmental variable that has the system path from where the Python program will look for the modules.

docs

absolute imports

Importing a module by giving the full path.

relative imports

Importing a module by giving the relative path from the current file.

Conditionals

Conditional Expression

lambda

callable

The callable() function returns True if the specified object is callable, otherwise it returns False.

*args **kwargs

• *args and **kwargs are special Python keywords that are used to pass the variable length of arguments to a function

• When using them together, *args should come before **kwargs

• The words “args” and “kwargs” are only a convention, you can use any name of your choice

arguments after the *args must be passed as the key word argument else it will raise an error.

Simply, in the function definition, the *args is the last parameter that is positional after this everything must be key word. **kwargs must be the last parameter in the function definition.

its all about packing and unpacking. * in the arguments is to say to unpack the iterable elements. if * is not specified at the time of calling the function with the iterables for the *args then it will take as the single positional arguments. Since we haven't specified to unpack the list or tuple that i sent as the individual positional arguments. Instead they are treated as the single positional argument.

* unpacks the iterables(list and tuple in our example) and calls the function and each element will be passed as the individual arguments.

The same goes for the **kwargs. we need to give ** while calling the function with the **kwargs paramater with the dict argument.

zip

To loop through the two or more iterable at the same time.

If one iterable contains more items, these items are ignored.

Zip does the transposition. Converting rows to colums colums to row

inner(local) function

The function defined inside another function is called inner function. That inner function is not an attribute of the outer function. The inner function is just an another function but that is defined inside a function.

first class function

In Python, all fucntions are first class function. Meaning they are treated like any other object type.

Closures

A Closure is a function object that remembers values in enclosing scopes even if they are not present in memory.

enclosing scope - the scope that encloses that

enclosing function - outer function

Firstly, a Nested Function is a function defined inside another function. It's very important to note that the nested functions can access the variables of the enclosing scope. However, at least in python, they are only readonly. However, one can use the "nonlocal" keyword explicitly with these variables in order to modify them. link

Closures maintains the reference to objects from earlier scopes.

Here the value of x is binded to the local fucntion but we haven't called that local function in its scope(inside the outer function). instead we have called that inner function from the global scope by making the outer fucntion to return the inner function. While doing so we need to remember the previous scope of that function to use the variable x while invoking that inner funtion outside of the outer function.

When the new binding happen it doesn't modifies the existing values of the different scopes

global

to modify the global variable values inside a function instead of creating new variable we could use the global keyword.

nonlocal

nonlocal keyword introduces the enclosing namespace variable to the local function (local namespace)

(introduces names from enclosing namespaces into the local namespace). You will get an error (SyntaxError) if the name doesn't exists

decorators

modify or enhance functions without changing their definition

The Decorator takes callable object and returns a callable object.

The callable can be of - class, function..

The last function object result is compiled.

classes as decorators

instances as decorators

multiple decorators

First, nearby decorator is called and got the callable return and repeat the steps.

then,

finally,

Attributes

class attributes

class attributes are class variables that are inherited by every object of a class.

The value of class attributes remain the same for every new object.

class attributes are defined outside the __init__() function.

instance attributes

Instance attributes, which are defined in the __init__() function, are instance variables that allow us to define different values for each object of a class.

class method

A class method is a method that is bound to the class and not the object of the class.

They have the access to the state of the class as it takes a class parameter that points to the class and not the object instance.

It can modify a class state that would apply across all the instances of the class.

For example, it can modify a class variable that will be applicable to all the instances.

A class method receives the class as an implicit first argument, just like an instance method receives the instance.

A class method can be called either on the class (such as C.f()) or on an instance (such as C().f()).

Class method is intended to access, modify and work with the class attributes.

Class methods also used as the factory methods.

Factory method is a creational design pattern which solves the problem of creating product objects without specifying their concrete classes. The Factory Method defines a method, which should be used for creating objects instead of using a direct constructor call ( new operator).

named constructor

Able to create instances.

have a class method that returns the object of the class. (which means we use this class method to create an object that is the contructor.)

static method (kindoff utility method)

Py Doc

Static methods in Python are extremely similar to Python class level methods, the difference being that a static method is also (like class method) bound to a class rather than the objects for that class.

This means that a static method can be called without an object for that class.

This also means that static methods cannot modify the state of an object as they are not bound to it.

We generally use static methods to create utility functions. like finding cubes, like that stuffs...only utilities

Static methods have a very clear use-case. When we need some functionality not with respect to an Object but with respect to the complete class, we make a method static. This is pretty much advantageous when we need to create Utility methods as they aren’t tied to an object lifecycle usually. Finally, note that in a static method, we don’t need the self to be passed as the first argument.

A static method can be called either on the class (such as C.f()) or on an instance (such as C().f()).

Simply, the Static method is a normal function inside the class. It has no self or cls parameter to be passed.

Trying to access the instance variable inside the static method

trying to access and modify the class variables

inheritance and overriding static method

Now, due to the explicit mention of which static method to use we lose our inheritance ability.

TO solve this,

here overriding of the static method works.

If it is explicit its done. we can't do anything.

static vs class vs instance methods

• static method is the utility method inside the class and it doesnot modify the state of the class or instance.

• class method deals with the class variables. and controls the state of the class across all the instances

• instance method deals with the instance variables and stuffs. It doesnot intend to change the class state. only controls the instance state.

@property

To make the method as an attribute to access the value of the variable.

We can use the @property decorator on any method of a class and use the method as a property.

The @property decorator also makes the usage of getters and setters much easier/

• Lets say we have to make the some attribute to be protected and not modified directly, and only get and set by the getters and setters.

The property name should be always same (we can name anything but it should be same) in the highlighted places.

Specifically, you can define three methods for a property:

• A getter - to access the value of the attribute.
• A setter - to set the value of the attribute.
• A deleter - to delete the instance attribute.

You don't necessarily have to define all three methods for every property. You can define read-only properties by only including a getter method. You could also choose to define a getter and setter without a deleter.

We could do this job without the @property decorator but using the @property is more Pythonic way of doing it.

Providing Write-Only Attributes

Properties are the Pythonic way to create managed attributes in your classes.

By using the @property decorator we could create new attribute to the class. But by convention it should give some meaning(should manage the attributes)

Useful property method

property in inheritance

this raises the error in line 12. name 'person_age' is not defined

*The Property name is not inherited we need to explicitly specify that.

properties and class methods

String representation of Python objects

str() and repr() functions for making string representations from Python objects which call the methods __str__() __repr__()

why do we need 2 methods?

repr()

str()

Print(Obj) calls str() this calls repr()

If str() is not defined then calling print(Obj) it calls repr(). If defined then it calls the corresponding method.

__format__

ascii()

ord()

Python ord() and chr() are built-in functions. They are used to convert a character to an int and vice versa. Python ord() and chr() functions are exactly opposite of each other.

chr()

float

Python float types can fail at many times due to precisions.

decimal

We can use decimal library to over come the issue faced by the Python float.

The reason for the issue in Decimal(x)-Decimal(y) is that in Python the base 10, 0.8 value can't be exactly represented in base 2

So it is always recommended to use the quote literals to avoid this.

abs()

The abs() function returns the absolute value of the specified number.

round()

bin() oct() int() hex()

map()

The map() function executes a specified function for each item in an iterable. The item is sent to the function as a parameter.

map(function, iterables)

Map sequence terminated when one of the given sequence ends. In other words, the output of the map is the sequence whose len is equal to the smallest len of the input sequence.

filter()

The filter() function returns an iterator were the items are filtered through a function to test if the item is accepted or not.

filter(function, iterable)

reduce()

functools.reduce(function, iterable)

• The first argument in reduce() is a function. This function will be applied to all the elements in an iterable in a cumulative manner to compute the result.

• The second argument is iterable. Iterables are those python objects that can be iterated/looped over, includes like lists, tuples, sets, dictionaries, generators, iterator, etc.

In other words, reduce will reduce to sequence to the single value.

Ex: if we have list of numbers and if we pass the list to the reduce function that maps it to another fucntion so in the end the list is reduced to the single value as the ouptu of reduce.

Multiple Inheritance

Here Base1 class __init__ is called since it was the first preference while inheriting.

Method Resolution Order

MRO has some rules,

Here since the Class B and C inherits A, A has to come first before B for the Class D.

super()

class bound proxy

instance bound proxy

super() alone in the instance methods and class methods

then,

Collection Protocol

Ref

ABC

Exception

Exception Hierarchy

To handle the KeyError and IndexError we could use LookupError. Becasue both the KeyError and IndexError inheirts the LookupError.

e.args

our custom Exception

Custom Exception class with the overriding some methods.

raise Error() from e

Assertion

Context manager

A context manager usually takes care of setting up some resource, e.g. opening a connection, and automatically handles the clean up when we are done with it.

Simply, the Context manager does the setup and teardown parts.(enter and exit)

PEP

Backend code of the Context manager

context manager protocol

custom Context manager class

enter()

exit()

The Exception is not propagated outside the with statement if the exit returns True.

contextlib

contextlib.contextmanager a decorator you can use to create new context managers.

The function shloud yield should not return

Deque

A deque (short for "double-ended queue") in Python is a data structure that allows fast and efficient addition and removal of elements from both ends (front and back). It is implemented in Python's collections module as collections.deque.

Key Features of a Deque

• Efficient Operations: Unlike lists, which have (O(n)) time complexity for inserting or deleting elements at the beginning, deques provide (O(1)) time complexity for such operations.
• Thread-Safe: Deques are thread-safe, making them suitable for multi-threaded applications.
• Dynamic Size: Unlike fixed-size arrays, a deque can grow or shrink dynamically.

Importing Deque

from collections import deque

Common Operations

1. Creating a Deque

   d = deque()  # Creates an empty deque
   d = deque([1, 2, 3])  # Creates a deque with initial elements
   

2. Adding Elements

   • Append to the right:

     d.append(4)  # Adds 4 to the right end
     

   • Append to the left:

     d.appendleft(0)  # Adds 0 to the left end
     

3. Removing Elements

   • Remove from the right:

     d.pop()  # Removes and returns the rightmost element
     

   • Remove from the left:

     d.popleft()  # Removes and returns the leftmost element
     

4. Accessing Elements You can access elements like a list:

   print(d[0])  # Access the first element
   print(d[-1])  # Access the last element
   

5. Rotating Elements Rotates the deque elements by a specified number of steps:

   d.rotate(1)  # Rotates right by 1 step. Default d.rotate()
   d.rotate(-1)  # Rotates left by 1 step
   

6. Checking Length

   print(len(d))  # Returns the number of elements
   

7. Clearing All Elements

   d.clear()  # Removes all elements from the deque
   

Example Usage

from collections import deque

# Initialize deque
d = deque([1, 2, 3])

# Add elements
d.append(4)
d.appendleft(0)

# Remove elements
d.pop()
d.popleft()

# Rotate elements
d.rotate(1)

# Print deque
print(d)  # Output: deque([3, 1, 2])

from collections import deque

l = ['1', '2', '3', '4']
d = deque(l)
print(d)

print('Extend')
d.extend(['5', '6'])
print(d)

print('Extendleft')
d.extendleft(['0', '-1',  '-2']) # extendleft will reverse and then extend to the left of the deck
print(d)

Output

deque(['1', '2', '3', '4'])
Extend
deque(['1', '2', '3', '4', '5', '6'])
Extendleft
deque(['-2', '-1', '0', '1', '2', '3', '4', '5', '6'])

Applications of Deques

• Queue and Stack Operations: Since a deque supports operations at both ends, it can be used as a queue (FIFO) or a stack (LIFO).
• Sliding Window Problems: Deques are efficient for handling problems that involve maintaining a window of elements.
• Breadth-First Search (BFS): In graph algorithms, a deque can be used for efficient level-order traversal.

Deques are versatile and ideal for scenarios where frequent additions and deletions from both ends are required.

Python Source

Thus, the pop() operation still causes it to act like a stack, just as it would have as a list. To make it act like a queue, use the popleft() command. Deques are made to support both behaviors, and this way the pop() function is consistent across data structures. In order to make the deque act like a queue, you must use the functions that correspond to queues. So, replace pop() with popleft() in your second example, and you should see the FIFO behavior that you expect.

Deques also support a max length, which means when you add objects to the deque greater than the maxlength, it will "drop" a number of objects off the opposite end to maintain its max size.

Queue - FIFO

A Queue in Python is a data structure that follows the FIFO (First In, First Out) principle. This means that elements are added to the back (rear) of the queue and removed from the front.

Python provides multiple ways to implement queues:

---

1. Using queue.Queue The queue module provides a thread-safe implementation of queues.

Import and Initialization

from queue import Queue

q = Queue(maxsize=5)  # maxsize is optional; default is infinite.

Common Operations

• Adding Elements: put() adds an element to the queue.

  q.put(1)  # Adds 1 to the queue
  

• Removing Elements: get() removes and returns the front element.

  item = q.get()  # Removes and returns the first element
  

• Checking Queue Size:

  print(q.qsize())  # Returns the current number of elements
  

• Check if Queue is Full or Empty:

  print(q.full())   # Returns True if full
  print(q.empty())  # Returns True if empty
  

Example

from queue import Queue

q = Queue(maxsize=3)

q.put(10)
q.put(20)
q.put(30)

print(q.get())  # Output: 10
print(q.get())  # Output: 20

---

2. Using collections.deque The deque from the collections module can also be used as a queue. It is faster but not thread-safe.

Import and Initialization

from collections import deque

q = deque()

Common Operations

• Adding Elements:

  q.append(1)  # Adds 1 to the rear
  

• Removing Elements:

  q.popleft()  # Removes and returns the front element
  

Example

from collections import deque

q = deque()

q.append(10)
q.append(20)
q.append(30)

print(q.popleft())  # Output: 10
print(q.popleft())  # Output: 20

---

3. Using queue.LifoQueue for Stack-like Queues The queue module also provides LifoQueue for LIFO (Last In, First Out) queues.

Import and Initialization

from queue import LifoQueue

stack = LifoQueue()

Common Operations

• Use put() and get() as with a regular queue, but elements are added and removed from the same end.

---

4. Using a Simple Python List Although not ideal for performance-critical applications, a list can mimic a queue:

Example

q = []

# Adding elements
q.append(10)
q.append(20)

# Removing elements
print(q.pop(0))  # Removes and returns the first element (FIFO)
print(q.pop(0))

Drawback: Removing elements from the front of a list is (O(n)) because all subsequent elements must be shifted.

---

Choosing the Right Queue

• For thread-safe operations: Use queue.Queue.
• For performance: Use collections.deque.
• For simple applications: A list may suffice.

Queues are essential for managing tasks, implementing algorithms (e.g., BFS), or handling resources in a controlled manner.

---

Stack - LIFO

A stack is a data structure that follows the LIFO (Last In, First Out) principle, where the last element added is the first to be removed. Python does not have a built-in stack data type, but you can implement stacks using various methods.

---

1. Using a List Python lists can be used to implement a stack, as they support append and pop operations.

Common Operations

• Push: Add an element to the top of the stack using append().
• Pop: Remove and return the top element using pop().
• Peek: Access the top element without removing it (using indexing).
• Check if Empty: Use len() to check if the stack is empty.

Example

stack = []

# Push elements
stack.append(1)
stack.append(2)
stack.append(3)

# Peek at the top element
print(stack[-1])  # Output: 3

# Pop elements
print(stack.pop())  # Output: 3
print(stack.pop())  # Output: 2

# Check if the stack is empty
print(len(stack) == 0)  # Output: False

Drawback: Lists are not ideal for large-scale stack operations because they allocate extra memory and resizing may affect performance.

---

2. Using collections.deque The deque (double-ended queue) in the collections module is a better option for implementing a stack due to its optimized memory and performance.

Import and Initialization

from collections import deque

stack = deque()

Common Operations

• Push: Use append() to add an element to the top.
• Pop: Use pop() to remove the top element.
• Peek: Use indexing to access the top element.

Example

from collections import deque

stack = deque()

# Push elements
stack.append(1)
stack.append(2)
stack.append(3)

# Peek at the top element
print(stack[-1])  # Output: 3

# Pop elements
print(stack.pop())  # Output: 3
print(stack.pop())  # Output: 2

# Check if the stack is empty
print(len(stack) == 0)  # Output: False

Advantage: Deques are more memory-efficient and faster for stack operations.

---

3. Using queue.LifoQueue The queue module in Python provides a thread-safe implementation of a stack using LifoQueue.

Import and Initialization

from queue import LifoQueue

stack = LifoQueue(maxsize=5)  # maxsize is optional

Common Operations

• Push: Use put() to add an element to the stack.
• Pop: Use get() to remove the top element.
• Check if Full/Empty:

  print(stack.full())   # Returns True if full
  print(stack.empty())  # Returns True if empty
  

Example

from queue import LifoQueue

stack = LifoQueue()

# Push elements
stack.put(1)
stack.put(2)
stack.put(3)

# Pop elements
print(stack.get())  # Output: 3
print(stack.get())  # Output: 2

# Check if the stack is empty
print(stack.empty())  # Output: False

Advantage: This is thread-safe but slightly slower than deque due to locking mechanisms.

---

Comparison of Methods

Method	Use Case	Performance	Thread-Safe
List	Small-scale, non-threaded applications	Moderate	No
deque	Performance-critical, non-threaded stack	Excellent	No
LifoQueue	Thread-safe stack operations	Good	Yes

For most applications, deque is the recommended choice due to its speed and simplicity. Use LifoQueue for multi-threaded environments.