Session 13: HTTP python library - myTeachingURJC/2018-19-PNE GitHub Wiki
-
Goals:
- Learn how to implemente web servers using the HTTP python module
- Debugging the HTTP protocol with the Browser developer's tools
- Duration: 2h
- Date: Week 7: Tuesday, March-5th-2019
- This session consist of the teacher's guidelines for driving the Lecture, following the learn by doing approach
- Introduction
- HTTP module web server
- Web Server 1: Using the http.server handler
- Web Server 2: Implementing our own handler
- Debugging the HTTP protocol
- Exercise 2
- Authors
- Credits
- License
Python includes an HTTP module for implementing both HTTP clients and servers very easily. In the previous week we learn how to implement them from sockets. This week we will learn how to program servers using the HTTP module
We will use a top-down approach: starting with the fully-working server provided by the http module. Then we will focused on the internal details for implementing our own servers
The HTTP module includes a fully working web server. Let's test it
- Create a new folder in your 2019-2020-PNE-practices project. Call it: Session-14
- Download these html example files and store them into the Session-14 folder:
- Select the Session-14 folder. Right-click on it and chose the Open in terminal entry
- From that terminal execute the following command:
python -m http.server 8080
- It will launch the web server. Go to the browser and open this page: http://localhost:8000. This other URLs are equivalent: http://0.0.0.0:8000/, http://your-IP:8000. You should see something like this:
The server is showing the index.html (that is our main web page). You should be able to navigate to the other pages, just clicking on the links
- Modify the index.html file for adding your name, your IP address and your row number
- Add two links in your main page: one to the server's of the classmate located at your right, and another to the one located on your left, following this scheme:
You can use the following template for the index.html page
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Linked Servers</title>
</head>
<body style="background-color: tomato">
<h1>Your-name page</h1>
<h2>IP: 192.168.124.41</h2>
<h2>Row: 3</h2>
<p>Click on the page you want to access</p>
<a href="green.html">[Green page]</a>
<a href="blue.html">[Blue page]</a>
<a href="pink.html">[Pink page]</a>
<br><br>
<a href="">[PREV]</a>
<a href="">[NEXT]</a>
</body>
</html>This page looks like this (you can change the background color if you want)
When all the servers are linked, we have created our own small world wide web: html documents located in different machines and linked one to each other
This is our first version of the web server using the http.server module. Here you can find all the documentation of the http.server module
The Handler is the name given to an object that is called when a client is connected and need to be attended. In this example we are using the Handler provided by the http.server module
- Create a new python file called webserver1.py and type this code
import http.server
import socketserver
# Define the Server's port
PORT = 8000
# -- Use the http.server Handler
Handler = http.server.SimpleHTTPRequestHandler
# -- Open the socket server
with socketserver.TCPServer(("", PORT), Handler) as httpd:
print("Serving at PORT", PORT)
# -- Main loop: Attend the client. Whenever there is a new
# -- clint, the handler is called
httpd.serve_forever()Initially a server socket is created, bound to our IP and port and changed to the listen mode. Then, serve_forever() method is called. It makes the server waits until a client is connected. When it happens, the handler function is called.
- Execute it from pycharm. It should work exactly the same as the previous example
Let's define our own Handler for attending the clients. As always, we will start with something very simple and we will be adding more functionality on every example
This example just prints a message on the console telling us that we have received a GET request message
import http.server
import socketserver
# Define the Server's port
PORT = 8002
# Class with our Handler. It is a called derived from BaseHTTPRequestHandler
# It means that our class inheritates all his methods and properties
class TestHandler(http.server.BaseHTTPRequestHandler):
def do_GET(self):
"""This method is called whenever the client invokes the GET method
in the HTTP protocol request"""
# We just print a message
print("GET received!")
# IN this simple server version:
# We are NOT processing the client's request
# We are NOT generating any response message
return
# ------------------------
# - Server MAIN program
# ------------------------
# -- Set the new handler
Handler = TestHandler
# -- Open the socket server
with socketserver.TCPServer(("", PORT), Handler) as httpd:
print("Serving at PORT", PORT)
# -- Main loop: Attend the client. Whenever there is a new
# -- clint, the handler is called
try:
httpd.serve_forever()
except KeyboardInterrupt:
print("")
print("Stopped by the user")
httpd.server_close()
print("")
print("Server Stopped")We should create a Class derived from the http.server.BaseHTTPRequestHandler base Class. For attending the GET request message, we should implement the do_GET method. This method will be called whenever there is a GET request from the client. In this example we are just printing a message on the console
Notice that we are not processing the client requests. And we are not generating any response message. For the later reason the browser will show us an error message
We have also improved how the server is stopped: whenever the control-c is detected (or the server is stopped within pycharm) the server_close() method is called and a message is printed on the script
- Type the code in the webserver2-1.py file and execute it
- In the server you should see an error like this:
In our console there should appear the message: GET received! (many times, because as the browser is not getting any response from the server, it is sending the request message many times)
As our TestHandler is derived from the Base class BaseHTTPRequestHandler, it inheritates all its properties and methods. You can find all the properties and method in the http.server documentation (Check it!!)
In this example we will print the following properties:
- self.requestline: It contatins the full request line (remember that the request line is the first line of the HTTP request messages)
- self.command: The command that the clients is invoking. It should be always GET as we are inside the do_GET() method
- self.path: It contains the resource that the client is asking for
This is the new web server:
import http.server
import socketserver
import termcolor
# Define the Server's port
PORT = 8003
# Class with our Handler. It is a called derived from BaseHTTPRequestHandler
# It means that our class inheritates all his methods and properties
class TestHandler(http.server.BaseHTTPRequestHandler):
def do_GET(self):
"""This method is called whenever the client invokes the GET method
in the HTTP protocol request"""
# We just print a message
print("GET received! Request line:")
# Print the request line
termcolor.cprint(" " + self.requestline, 'green')
# Print the command received (should be GET)
print(" Command: " + self.command)
# Print the resource requested (the path)
print(" Path: " + self.path)
# IN this simple server version:
# We are NOT processing the client's request
# We are NOT generating any response message
return
# ------------------------
# - Server MAIN program
# ------------------------
# -- Set the new handler
Handler = TestHandler
# -- Open the socket server
with socketserver.TCPServer(("", PORT), Handler) as httpd:
print("Serving at PORT", PORT)
# -- Main loop: Attend the client. Whenever there is a new
# -- clint, the handler is called
try:
httpd.serve_forever()
except KeyboardInterrupt:
print("")
print("Stoped by the user")
httpd.server_close()
print("")
print("Server Stopped")- Execute the new web server and connects from the browser. Check your console messages:
Now we will add a response message to our handler so that the browser will show some contents instead of an error message. Our server will always send the same message, no matter what resources is the client requesting: is a happy server :-)
In our TestHandler Class we can use the following methods for generating the response very easily:
- self.send_response(code) : Creates a response header with a status line with the error CODE passed as an argumento. The response is not really sent yet, but stored into a buffer. The status codes can be found in this link
- self.send_header(): Add a header to the response message (Ex. Content-Type, Content-Length...)
- self.end_headers(): Add a blank line to the response message (indicating that the header is finished)
The message body is sent using the self.wfile.write() method
import http.server
import socketserver
import termcolor
# Define the Server's port
PORT = 8000
# Class with our Handler. It is a called derived from BaseHTTPRequestHandler
# It means that our class inheritates all his methods and properties
class TestHandler(http.server.BaseHTTPRequestHandler):
def do_GET(self):
"""This method is called whenever the client invokes the GET method
in the HTTP protocol request"""
# Print the request line
termcolor.cprint(self.requestline, 'green')
# IN this simple server version:
# We are NOT processing the client's request
# It is a happy server: It always returns a message saying
# that everything is ok
# Message to send back to the clinet
contents = "I am the happy server! :-)"
# Generating the response message
self.send_response(200) # -- Status line: OK!
# Define the content-type header:
self.send_header('Content-Type', 'text/plain')
self.send_header('Content-Length', len(str.encode(contents)))
# The header is finished
self.end_headers()
# Send the response message
self.wfile.write(str.encode(contents))
return
# ------------------------
# - Server MAIN program
# ------------------------
# -- Set the new handler
Handler = TestHandler
# -- Open the socket server
with socketserver.TCPServer(("", PORT), Handler) as httpd:
print("Serving at PORT", PORT)
# -- Main loop: Attend the client. Whenever there is a new
# -- clint, the handler is called
try:
httpd.serve_forever()
except KeyboardInterrupt:
print("")
print("Stoped by the user")
httpd.server_close()
print("")
print("Server Stopped")- Run the server. Now the browser should display the following message:
And this is what you get in the pycharm console:
The servers are sometimes hard to debug. Maybe one of our server is not working fine, but we do not know why. The browsers have tools for helping the developers to understand what is going wrong. We will learn how to use the Firefox's developer toools
- Run the previous web server
- Open a new tab in firefox
- click on the top-right menu and select the entry webdeveloper/network
- Connect to the server: localhost:8000 (or the port you have configured)
- You will see all the requests done by the browser, and their status codes
- Clicking on the first request will give you more information about the response and request messages interchanged:
Modify the previous server for send back the index.html file when the / resource is requested. It should send the error.html page (and the correspoding error code) if any other resource is requested
- Boni García
- Juan González-Gómez (Obijuan)
- Alvaro del Castillo. He designed and created the original content of this subject. Thanks a lot :-)
