Teaching Guide for Implementing Web Requests and Responses in Python

1. Web request and response basis

(I) Definition and composition of web requests and responses

Web requests and responses are the basis of web communication, where web requests are initiated by the client and the server returns a response after processing.

Composition of request

Request line: Contains request methods (such as GET, POST, PUT, DELETE, etc.), URL, and HTTP protocol version (such as HTTP/1.1). For example, the request line for a GET request might be "GET/HTTP/1.1".

Request header: Contains metadata about client information, request body type, browser type, etc. Common request headers include User - Agent (used to identify client type), Content - Type (representing the data type of the request body), etc.

Request body: The POST request usually contains data submitted by the user, such as form data or files, while the GET request generally does not have a request body.

The composition of the response

• Response line: Includes HTTP protocol version, status code, and status messages. For example, "HTTP/1.1 200 OK", where 200 is the status code and OK is the status message.
• Response header: Contains information about the response, such as content type (Content - Type), server information (Server), etc.
• Response body: contains the actual returned data, which may be HTML pages, JSON data, etc.

(II) Overview of HTTP protocol

HTTP (Hypertext Transfer Protocol) is a protocol for transmitting data on the web and is responsible for communication between the browser and the server. Here are the common HTTP methods:

GET: The request server obtains resources, which is usually used to read data. Its characteristic is that the request parameters are attached behind the URL, separated by "?", and multiple parameters are connected by "&". For example“/search?keyword=python”。

POST: Submit data to the server, which is usually used for form submission, file upload, etc. The data is transmitted through the request body and will not be displayed in the URL, which is relatively safe.

PUT: Used to update resources on the server.

DELETE: Used to delete resources on the server.

(III) Common HTTP status codes

• 200 OK: Indicates that the request is successful and the server returns the requested data.
• 301 Moved Permanently: Indicates that the resource has been moved permanently and the client will automatically redirect to the new URL.
• 404 Not Found: Indicates that the requested resource does not exist.
• 500 Internal Server Error: Indicates an internal error of the server.

2. Python's requests library

(I) Install the requests library

Before using the requests library, you need to install it first. If you already have Python's package management tool pip installed on your system, you can install it on the command line through the following command:

pip install requests

(II) Send GET request

GET requests are usually used to get data. We can send GET requests through the () method and process the returned response. Here is a simple example code:

import requests
 
# Send a GET requestresponse = ("")
 
# Output the status code of the responseprint("Status Code:", response.status_code)
 
# Output response contentprint("Response Body:", )
 
# Output response headerprint("Response Headers:", )
 
# Get the length of the response contentprint("Content Length:", len())

Code explanation:

• () is used to send a GET request, passing the URL to be requested in brackets. This method will return a Response object, containing all the information returned by the server.
• response.status_code is used to obtain the HTTP response status code, through which the status code can be used to determine whether the request is successful.
• Used to get the body content of the response, usually HTML or JSON data, and returns a string type.
• Used to get the response header, it is a dictionary-like object that can access the corresponding value through keys.
• len() is used to return the length of the response body. This value can help us understand the size of the returned content.

(III) Send POST request

POST requests are used to submit data to the server, usually used in scenarios such as form submission or uploading files. We use the () method to send POST requests. Here is the sample code:

import requests
 
# Send a POST requesturl = "/post"
data = {"name": "Alice", "age": 25}
response = (url, data=data)
 
# Output response status codeprint("Status Code:", response.status_code)
 
# Output response content (JSON format)print("Response Body:", ())

Code explanation:

• () is used to send a POST request. The first parameter is the URL and the second parameter is data. It contains the data to be submitted. The requests library automatically encodes it in application/x-www-form-urlencoded format.
• () is used to parse the returned JSON data. It will convert the response content in JSON format into data structures such as Python dictionary or list, which is convenient for us to process.

(IV) Processing response header and status code

The response header provides information about the server, and the status code tells us whether the request is successful. We can get the response header and status code through the following code:

import requests
 
# Send a GET requestresponse = ("")
 
# Get the response headerprint("Response Headers:", )
 
# Get the response status codeprint("Status Code:", response.status_code)
 
# Get content typeprint("Content - Type:", ("Content - Type"))

Code explanation:

Returns the response header, which contains information such as Content - Type, Date, Server, etc. We can obtain the value of the specified key through the get () method of the dictionary, such as getting the content type.

response.status_code returns an HTTP status code. By determining whether the status code is a successful status code such as 200, it can determine whether the request is successfully executed.

(V) Send GET request with query parameters

In a GET request, we can pass query parameters through the URL. For example, to access a URL containing parameters, we can pass the query parameters through the params parameter of the () method. Here is the sample code:

import requests
 
# Send a GET request with query parametersurl = "/get"
params = {"name": "Alice", "age": 25}
response = (url, params=params)
 
# Output response contentprint("Response Body:", ())

Code explanation:

params is a dictionary containing the query parameters to be passed. The () method will automatically encode these parameters into the URL, and the generated URL is similar to "/get?name=Alice&age=25".

(VI) Send POST requests with form data

A POST request can be used to submit form data. Here is a sample code for sending a POST request with form data using requests:

import requests
 
# Send POST request with form dataurl = "/post"
data = {"username": "testuser", "password": "mypassword"}
response = (url, data=data)
 
# Output response contentprint("Response Body:", ())

Code explanation:

The data parameter is a dictionary that contains the data submitted by the form. The requests library will automatically encode the data into application/x-www-form-urlencoded format, which is a common format for form data submission.

3. Process JSON response

Many Web APIs return data formats JSON, and Python's requests library provides convenient JSON processing methods. Here is the sample code:

import requests
 
# Send a GET request and get a JSON responseurl = "/users/octocat"
response = (url)
 
# parse JSON datadata = ()
 
# Output user's GitHub informationprint("User Login:", data["login"])
print("User Name:", data["name"])

Code explanation:

The () method parses the content of the response into a Python dictionary, so that we can easily process JSON data like operating a normal dictionary, and get the corresponding value through keys.

4. File operation

(I) Open file mode

Python uses the built-in open () function to open a file. When opening a file, you need to specify the file mode (that is, the way to operate the file). The following are common file opening modes:

• Read-only mode (r): Default mode, the file must exist. If the file does not exist, a FileNotFoundError exception will be thrown.
• Write mode (w): If the file exists, the file content will be overwritten; if the file does not exist, a new file will be created.
• Append mode (a): If the file exists, the written data will be appended to the end of the file; if the file does not exist, a new file will be created.
• Exclusive Creation Mode (x): If the file already exists, the operation fails and a FileExistsError exception is thrown. This mode is usually used to prevent overwriting existing files when creating files.
• Binary Reading Mode (rb): Used to read non-text files (such as pictures and audio files).
• Binary write mode (wb): used to write to non-text files.
• Read and write mode (+): The file must exist, and it can both read the file content and write data.
• w + mode: read and write mode, if the file exists, the file content will be overwritten; if the file does not exist, a new file will be created.
• a + mode: read and write mode. When the file exists, the data will be appended to the end of the file; if the file does not exist, a new file will be created.
• rb + mode: binary read and write mode.

Here is a sample code for opening a file using different modes:

# Open the file in read-only modewith open('', 'r') as file:
    content = ()
    print(content)
 
# Open the file in write mode, the file content will be overwrittenwith open('', 'w') as file:
    ("This is the new file content.\n")
 
# Open the file in append mode, new content will be appended to the end of the filewith open('', 'a') as file:
    ("Add content.\n")
 
# Open a file in binary mode (for example, read an image)with open('', 'rb') as file:
    binary_data = ()
    print("Readed binary data:", binary_data[:20])

(II) Read the file

File reading functions in Python are very powerful. Here are several common reading methods:

read () method: used to read all content in the file, and the read content will be returned as a string. The sample code is as follows:

with open('', 'r') as file:
    content = ()
    print(content)

readline () method: read one line of file content at a time, which is suitable for situations where files need to be processed line by line. The sample code is as follows:

with open('', 'r') as file:
    line = ()
    while line:
        print(())  # strip() is used to remove line breaks at the end of the line        line = ()

readlines () method: It will read all lines in the file at once and store each line of data as an element of a list, suitable for situations where the entire file needs to be read and processed in rows. The sample code is as follows:

with open('', 'r') as file:
    lines = ()
    for line in lines:
        print(())

(III) Write to the file

Python provides several methods to write data to files. Write operations are often used in logging, data export and other scenarios.

Write to a file using the write () method: This method writes the specified string to the file. If the file is opened in "w" mode, the original file content will be overwritten; if it is opened in "a" mode, the content will be appended to the end of the file.

The sample code is as follows:

with open('', 'w') as file:
    ("This is the first line of data.\n")
    ("This is the second row of data.\n")

Use the writelines () method to write multiple lines of data: This method accepts an iterable object (such as lists, tuples, etc.), writes its elements into the file, and each element will be written to the file as one line. The sample code is as follows:

lines = ["First row of data.\n", "Second line of data.\n", "Third row of data.\n"]
with open('', 'w') as file:
    (lines)

(IV) Download file example

We can download the file through the requests library and save it locally. Here is the sample code for downloading an image file:

import requests
 
url = "/"  # Image URLresponse = (url)
 
# Check whether the request is successfulif response.status_code == 200:
    # Write files in binary mode    with open('downloaded_image.jpg', 'wb') as file:
        ()
    print("The picture download was successful!")
else:
    print(f"Download failed,Status code:{response.status_code}")

Code explanation:

First, use the () method to send a GET request to obtain the content of the image file and set the URL to the image address.

By determining whether the status code of the response is 200, it is determined whether the request is successful.

If the request is successful, open the file in binary write mode ('wb') using the with open () statement, and then write the response content (binary data) to the file via (). Returns byte stream data, suitable for processing binary files.

(V) Precautions in file operation

Whether the file exists: When opening the file, you must make sure that the file path is correct. If the file does not exist, you can use the () function to check whether the file exists, or use the try - except statement to catch the FileNotFoundError exception. The sample code is as follows:

import os
 
if (''):
    with open('', 'r') as file:
        content = ()
else:
    print("The file does not exist!")

File permissions: When operating files, you may encounter insufficient permissions. For example, try to write to a read-only file, or access a file that does not have read permissions. In this case, you can use the try - except statement to catch the PermissionError exception. The sample code is as follows:

try:
    with open('readonly_file.txt', 'w') as file:
        ("Try to write to a read-only file")
except PermissionError:
    print("Insufficient permissions, cannot write to the file.")

Automatic file closing: When using the with open () statement, Python automatically manages the opening and closing of files without explicitly calling the () method, which helps avoid the problem of not closing the file and reduces the risk of resource leakage.

(VI) Other common file operations

Get file information: Python provides os and modules, which can obtain file size, modification time and other information. The sample code is as follows:

import os
 
file_path = ''
print("File Size:", (file_path), "byte")
print("File modification time:", (file_path))

Delete files: Use the () function to delete files. The sample code is as follows:

import os
 
file_path = ''
if (file_path):
    (file_path)
    print(f"{file_path}Deleted!")
else:
    print("The file does not exist!")

5. Error handling and exception capture

Various errors may occur when making web requests, such as network timeouts, server errors, etc. The requests library helps us catch these errors through exception handling mechanism. Python's try statements can catch and handle exceptions in code blocks, thus avoiding program crashes and providing an opportunity to handle errors.

(I) Use of try statements

The try statement is used to catch and handle exceptions. It consists of the following three parts (where else block and finally block are optional):

• try block: contains code that may throw exceptions. When an error occurs during the code running, the program will jump to the corresponding except block for processing.
• except block: When an exception occurs in the code in the try block, the program will jump to the except block for execution. In except, you can specify the exception type to be caught, such as Timeout, HTTPError, etc.
• else block (optional): If the code in the try block does not throw an exception, the code in the else block will be executed.
• finally block (optional): The code in the finally block will be executed regardless of whether an exception occurs, and is usually used to clean up resources (such as closing files, database connections, etc.).

(II) Example: Catch common exceptions

Here is a sample code for catching common exceptions:

import requests
from  import RequestException, Timeout, HTTPError
 
try:
    # Send a GET request and set the timeout to 5 seconds    response = ("", timeout=5)
 
    # If the status code is not 200, an HTTPError exception is thrown    response.raise_for_status()  # If the status code is 4xx or 5xx, an exception is thrown 
    # If the request is successful, output the response content    print("Response Body:", )
 
#Catch request timeout exceptionexcept Timeout:
    print('Request timed out')
 
# Capture HTTP errors (such as status codes 404, 500, etc.)except HTTPError as http_err:
    print(f'HTTP error occurred: {http_err}')
 
#Catch other network-related errorsexcept RequestException as req_err:
    print(f'Request error occurred: {req_err}')
 
# You can clean resources (such as closing files or connecting) in finally blocksfinally:
    print("Request attempt completed.")

Code explanation:

try block: First initiate an HTTP request, and set the timeout to 5 seconds through the timeout=5 parameter. Then use the response.raise_for_status() method to check the status code of the response. If the server returns an incorrect status code (such as 404, 500, etc., that is, the status code of 4xx or 5xx), the method will throw an HTTPError exception. If the status code is 200 or other successful status code, then the subsequent operation of printing the response content is performed.

except block:

• Timeout: When the request timeout (more than 5 seconds set), the program will catch the Timeout exception and print "Request timed out" to prompt the user to request timeout.
• HTTPError: If the status code of the response indicates an HTTP error (for example, 404 means that the page is not found, 500 means that the server is internal error, etc.), the program will catch an HTTPError exception and print related error information, such as "HTTP error occurred: 404 Client Error: Not Found for url:", which contains the specific error status code and error description, so that users can understand the cause of the error.
• RequestException: This exception is the base class for all requests library exceptions, and can catch any exception thrown by requests library, such as connection problems, DNS resolution failures, etc. When other network-related errors occur, the program will catch a RequestException exception and print "Request error occurred: [Specific error message]", allowing the user to know that an error related to network request has occurred.

Finally block: The code in it will be executed regardless of whether an exception occurs. Only "Request attempt completed." is printed here, which is used to indicate the end of the request. Usually, some resource cleaning work can be carried out here, such as closing files, database connections, etc., to ensure that the program's resources are reasonably released.

(III) Summary of exception handling

Avoid program crash: The exception handling mechanism allows us to catch errors during the program operation and handle them accordingly, avoiding the program suddenly crashing due to exceptions, improving the stability and reliability of the program.

Precise exception handling: Through the try...except structure, you can accurately capture and process different exception types, and execute different processing logic for different error situations, so that the program can more flexibly deal with various exception situations. For example, for a timeout exception, a request can be re-initiated, and for an HTTP error, it can be prompted to check whether the requested URL is correct, etc.

Resource Cleanup: The finally block is used to perform cleanup work. Regardless of whether an exception occurs, the code in it will be executed after the request processing is completed, ensuring that resources (such as file handles, network connections, etc.) can be released correctly and avoid resource leakage problems. This is particularly important for long-running programs.

6. Try statement

(I) Basic syntax and structure

try:
    # Block of code that may throw exceptions    result = 10 / 0  # Delete Zero Errorexcept ZeroDivisionError as e:
    # Catch a specific exception and get the error object    print(f"mistake: {e}")  # Output: division by zeroelse:
    # Execute only when there is no exception in the try block    print("Computation is successful:", result)
finally:
    # Execute regardless of exception or not (commonly used for resource cleaning)    print("Execution Completed")

(II) Capture multiple exceptions

1. Handle different exceptions separately

try:
    num = int("abc")  # ValueError
    result = 10 / num  # ZeroDivisionError
except ValueError:
    print("Input is not a valid integer")
except ZeroDivisionError:
    print("The divisor cannot be zero")

2. Merge and handle similar exceptions

try:
    file = open("")  # FileNotFoundError
    data = (file)  # JSONDecodeError
except (FileNotFoundError, ) as e:
    print(f"File Error: {e}")

(III) Capture all exceptions

Abuse is not recommended, which may cover up serious problems:

try:
    #Arbitrary codeexcept Exception as e:
    print(f"Unknown error: {e}")
    # Recommended record stack information for debugging    import traceback
    print(traceback.format_exc())

(IV) Actively throw exception (raise)

def validate_age(age):
    if age < 0:
        raise ValueError("Age cannot be negative")
    return age
 
try:
    validate_age(-5)
except ValueError as e:
    print(e)  # Output: Age cannot be negative

(V) Custom exception

class AuthError(Exception):
    """Custom authentication error"""
    def __init__(self, message="Certification failed"):
         = message
        super().__init__()
 
#User Exampletry:
    if not user_authenticated():
        raise AuthError("Invalid Token")
except AuthError as e:
    print(f"Authentication exception: {e}")

(VI) Exception chain (raise ... from)

Keep the original exception information to facilitate locating problems:

try:
    data = ("invalid json")
except  as e:
    raise RuntimeError("Configuration parsing failed") from e

Output:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
: Expecting value...
 
The above exception was the direct cause of the following exception:
 
Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
RuntimeError: Configuration parsing failed

(VII) Context Manager (with statement)

Alternative try-finally for automatic resource management:

try:
    with open("", "r") as file:
        content = ()
except FileNotFoundError:
    print("The file does not exist")
# The file will be closed automatically without manual call ()

(8) Common exception types

(9) Best Practices

1. Accurately capture exceptions

try:
    result = 10 / num
except (ZeroDivisionError, TypeError) as e:
    print(f"Calculation error: {e}")

2. Use logging to record exceptions

import logging
 
try:
    # Dangerous Operationexcept Exception as e:
    (f"System exception: {e}", exc_info=True)

3. Custom exception hierarchy

class AppError(Exception):
    """Application base exception"""
 
class DatabaseError(AppError):
    """Database operation exception"""
 
class ConnectionError(DatabaseError):
    """Connection exception"""

4. Resource cleaning is preferred with

# Not recommendedfile = open("")
try:
    data = ()
finally:
    ()
 
# recommendwith open("") as file:
    data = ()

(10) Exception handling process

Execute try block: If there is an exception, jump to the first matching except

Match except: Check in order, and execute the corresponding code block when found

Execute else: If try there is no exception

Execution finally: whether it is abnormal or not, the final execution

(11) Advanced usage: try-except-else-finally combination

try:
    # Try to operate    file = open("", "r")
except FileNotFoundError:
    # Handle missing configuration    print("Use default configuration")
    config = DEFAULT_CONFIG
else:
    # Read configuration when no exceptions    try:
        config = (file)
    except :
        print("Malfunctioning configuration format, use default values")
        config = DEFAULT_CONFIG
    finally:
        ()  # Make sure the file is closedfinally:
    # Application Configuration    apply_config(config)

(12) Exceptions and performance

Exception handling itself is not slow, but frequent throwing/catching will affect performance

Avoid using exceptions to control routine processes (such as looping through files)

Priority use condition judgment (such as if (file))

The above is the detailed content of the teaching guide for implementing web requests and responses in Python. For more information about Python Web requests and responses, please follow my other related articles!