Analysis of common ways of calling APIs in Python

1. Introduction

In data development, calling API is one of the core ways to connect to external services, obtain data, or implement system integration.
Recently, due to the security restrictions of Alibaba Cloud ODPS, a batch of API calls have been dealt with. Therefore, from the perspectives of principles, cases, advantages and disadvantages, I will introduce the common methods of writing Python to call APIs.

2. Introduction to common methods

The main methods of Python calling APIs can be divided into the following four categories. The characteristics and applicable scenarios of each method are somewhat different:

method	principle	Applicable scenarios
requests	A third-party library based on HTTP encapsulates GET/POST and other request methods, providing a concise API call interface.	Regular HTTP requests, rapid development, data interaction
	Python has built-in HTTP library, which requires manual construction of request headers and packets to finely control request scenarios.	Low-level operation and high compatibility requirements
urllib	Standard library modules, including submodules of other submodules, basic core.	Scenarios with simple requests and no dependence on third-party libraries
aiohttp	The HTTP library based on asynchronous IO is used to implement non-blocking requests through coroutines, which is suitable for high concurrency scenarios.	Asynchronous programming, high concurrency API calls

Implementation and cases

Principle: By encapsulating HTTP request methods, parameter delivery and response processing are simplified, URL encoding, connection pool management and response resolution are automatically processed, and functions such as JSON parsing, Session sessions, file uploads and other functions are supported.
advantage: Concise code and comprehensive functions
shortcoming: Request IO-intensive scenarios with low efficiency

import requests

def get_github_user(username):
    """
     Use the requests library to call the GitHub API to get user information
     """
    # Construct API requests    url = f"/users/{username}"
    
    try:
        # Send GET requests (automatically handle connection and closing)        response = (url, timeout=5)
        
        # Check HTTP status code        if response.status_code == 200:
            # parse JSON response data            user_data = ()
            return {
                "name": user_data.get("name"),
                "location": user_data.get("location"),
                "public_repos": user_data.get("public_repos")
            }
        else:
            print(f"Request failed，Status code：{response.status_code}")
    except  as e:
        print(f"Request exception：{str(e)}")

# Call exampleif __name__ == "__main__":
    print(get_github_user("octocat"))  # GitHub official test account

Principle: The built-in underlying HTTP client library, which manually manages the connection life cycle and message header structure, suitable for scenarios where HTTP protocol is needed
advantage: Directly available, fine-grained control
Disadvantages: The code is verbose, and the encoding and connection need to be processed manually

import 
import json

def get_httpbin_data():
    """
     Get httpbin test data using
     """
    # Create an object    conn = ("")
    try:
        # Send a GET request        ("GET", "/get")
        # Get the response object        response = ()
        # Read the response body and decode it        data = ().decode('utf-8')
        # parse JSON data        return (data)
    except Exception as e:
        print(f"Request exception：{str(e)}")
    finally:
        # Connection must be closed manually        ()

# Call exampleif __name__ == "__main__":
    print(get_httpbin_data())

(Lightweight solution)

principle: Compound HTTP tool, integrated modules such as request/error/parse, suitable for simple requests and dependencies without dependencies
advantage: Lightweight
shortcoming: The interface design is scattered and the lack of advanced functions

from  import urlopen
import json

def get_user_agent():
    """
     Use urllib to get client UA information
     """
    url = "/user-agent"
    try:
        # Open the connection and automatically handle HTTP/HTTPS        with urlopen(url, timeout=5) as response:
            # Read and parse JSON data            return (().decode())
    except Exception as e:
        print(f"Request exception：{str(e)}")

# Call exampleif __name__ == "__main__":
    print(get_user_agent())

(Async solution)

principle: Asyncio-based HTTP client, using coroutines to implement non-blocking IO operations, suitable for high-concurrency scenarios (such as crawlers, real-time systems)
Advantages: High throughput, high resource utilization
shortcoming: Needs asynchronous programming knowledge and high debugging complexity

import aiohttp
import asyncio

async def fetch_concurrent_data():
    """
     Use aiohttp to obtain multiple API data concurrently
     """
    urls = [
        "/posts/1",
        "/comments/1"
    ]
    
    # Create a shared session (automatically manage connection pools)    async with () as session:
        tasks = []
        for url in urls:
            # Create concurrent tasks            task = asyncio.create_task(
                (url, timeout=(total=3))
            )
            (task)
        
        # Wait for all tasks to complete        responses = await (*tasks)
        
        results = []
        for response in responses:
            # Check the response status            if  == 200:
                data = await ()
                (data)
        return results

# Call exampleif __name__ == "__main__":
    print((fetch_concurrent_data()))

4. Summary and comparison

method	Applicable scenarios	Performance	Difficulty
requests	Regular REST API calls	★★★★☆	Simple
	Underlying protocol control	★★☆☆☆	medium
urllib	Simple request/dependency-free environment	★★★☆☆	Simple
aiohttp	High concurrency/asynchronous tasks	★★★★★	complex

This is the end of this article about the common ways to parse Python APIs. For more related content on Python API calls, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!