PMEve--Everyone is a Product Manager

Home >> Tech Dev >> Python

Ten Tips for Making Your Python Code Safer When Handling Network Requests

Time： Column：Python views：265

In Python web programming, using the requests library to make HTTP requests is a common and efficient approach. This library not only provides a simple and easy-to-use API but also supports various advanced features. This article will introduce how to leverage the requests library to enhance the safety and stability of your code, covering aspects ranging from basic GET requests to complex session management and error handling.

Ten Tips for Making Your Python Code Safer When Handling Network Requests

Tip 1: Use the requests Library for HTTP Requests

In Python, requests is one of the most commonly used libraries for sending HTTP requests. Compared to the standard library urllib, requests provides a more concise and user-friendly API while supporting more functionalities.

Example Code:

import requests

# Send a GET request
response = requests.get('https://api.github.com')

# Output the response content
print(response.text)

Code Explanation:

  1. Import the requests library.

  2. Use the get() method to send a GET request to GitHub's API.

  3. Print the content returned by the server.

Tip 2: Set a Timeout

Setting a reasonable timeout for network requests is crucial. This can prevent the program from freezing due to network delays or unresponsive servers.

Example Code:

import requests

try:
    response = requests.get('https://api.github.com', timeout=5)  # Set timeout to 5 seconds
    print(response.text)
except requests.exceptions.Timeout:
    print("Request timed out, please check your network connection!")

Code Explanation:

  1. Use the timeout parameter to set a timeout period.

  2. Use a try...except structure to catch timeout exceptions and provide a prompt message.

Tip 3: Validate SSL Certificates

When interacting with HTTPS websites, requests validates the server's SSL certificate by default. However, there are times when we may need to disable this feature (e.g., in a testing environment) or specify a custom CA certificate.

Example Code:

import requests

# Disable SSL certificate verification
response = requests.get('https://api.github.com', verify=False)

# Specify CA certificate file
response = requests.get('https://api.github.com', verify='/path/to/cert.pem')

Code Explanation:

  1. Use verify=False to disable SSL certificate verification.

  2. Use the verify parameter to specify the path to the CA certificate.

Tip 4: Use a Proxy Server

In certain cases, we may need to access the internet through a proxy server. requests supports setting up a proxy server for network requests.

Example Code:

import requests

proxies = {
    'http': 'http://10.10.1.10:3128',
    'https': 'http://10.10.1.10:1080',
}

response = requests.get('https://api.github.com', proxies=proxies)
print(response.text)

Code Explanation:

  1. Define a dictionary containing the proxy server addresses.

  2. Use the proxies parameter to set the proxy server.

Tip 5: Set Request Headers

To better simulate browser behavior, we can customize request headers, including User-Agent, Accept-Encoding, etc.

Example Code:

import requests

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3',
    'Accept-Encoding': 'gzip, deflate',
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'Connection': 'keep-alive',
}

response = requests.get('https://api.github.com', headers=headers)
print(response.text)

Code Explanation:

  1. Define a dictionary containing the request header information.

  2. Use the headers parameter to set the request headers.

Tip 6: Handle Redirects

When making network requests, you may encounter redirects. By default, requests will automatically handle redirects, but you can also control this manually.

Example Code:

import requests

# Allow automatic redirects
response = requests.get('http://github.com', allow_redirects=True)

# Disable automatic redirects
response = requests.get('http://github.com', allow_redirects=False)

Code Explanation:

Use the allow_redirects parameter to control whether automatic redirects are allowed.

Tip 7: Handling Cookies

When making network requests, cookies are essential, especially when maintaining a logged-in state. The requests library provides convenient methods to handle cookies.

Example Code:

import requests

# Create a Session object
session = requests.Session()

# Send a request with cookies
url = 'https://example.com'
cookies = {'session_id': '12345'}
response = session.get(url, cookies=cookies)

# Output the response content
print(response.text)

Code Explanation:

  1. A Session object is created to share cookies across multiple requests.

  2. The cookies parameter is used to pass cookie information.

Tip 8: Using Authentication

For some APIs or websites that require authentication, we need to provide credentials like a username and password. requests offers various methods to handle authentication.

Example Code:

import requests

# Basic authentication
url = 'https://api.example.com/data'
username = 'user'
password = 'password'
response = requests.get(url, auth=(username, password))
print(response.text)

# OAuth authentication
url = 'https://api.example.com/data'
token = 'your_oauth_token'
headers = {'Authorization': f'Bearer {token}'}
response = requests.get(url, headers=headers)
print(response.text)

Code Explanation:

  1. The auth parameter is used to pass basic authentication credentials.

  2. The headers parameter is used to pass OAuth authentication credentials.

Tip 9: Handling Error Responses

During network requests, it’s common to encounter various error responses such as 404 Not Found or 500 Internal Server Error. Proper handling of these errors is key to ensuring the stability of the program.

Example Code:

import requests

url = 'https://api.example.com/data'

try:
    response = requests.get(url)
    response.raise_for_status()  # Raises an HTTPError if the status code is not 200
    print(response.text)
except requests.exceptions.HTTPError as e:
    print(f"HTTP Error: {e}")
except Exception as e:
    print(f"Other Error: {e}")

Code Explanation:

  1. The raise_for_status() method is used to check the response status code.

  2. A try...except structure is used to catch exceptions and provide error messages.

Tip 10: Using Session Management

When handling a series of consecutive network requests, using session management can improve efficiency. The requests library provides a Session class to manage sessions.

Example Code:

import requests

# Create a Session object
session = requests.Session()

# Set default request headers
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3',
    'Accept-Encoding': 'gzip, deflate',
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'Connection': 'keep-alive',
}
session.headers.update(headers)

# Send multiple requests
url1 = 'https://api.github.com'
url2 = 'https://api.github.com/users/octocat'

response1 = session.get(url1)
response2 = session.get(url2)

print(response1.text)
print(response2.text)

Code Explanation:

  1. A Session object is created.

  2. The update() method is used to set default request headers.

  3. The session.get() method is used to send multiple requests, sharing the same headers and session information.

Practical Case Study: Retrieving Weather Information

Suppose we are developing a simple weather query system that retrieves weather information by calling an external API. Here, we will use the OpenWeatherMap API.

Example Code:

import requests

# API base URL and API key
base_url = "https://api.openweathermap.org/data/2.5/weather"
api_key = "your_api_key"

# Query parameters
params = {
    'q': 'New York',
    'appid': api_key,
    'units': 'metric'  # Units in Celsius
}

# Send a GET request
response = requests.get(base_url, params=params)

# Check the response status code
if response.status_code == 200:
    weather_data = response.json()
    city_name = weather_data['name']
    temperature = weather_data['main']['temp']
    description = weather_data['weather'][0]['description']

    print(f"City: {city_name}")
    print(f"Temperature: {temperature}°C")
    print(f"Description: {description}")
else:
    print("Request failed, status code:", response.status_code)

Code Explanation:

  1. Set the API base URL and API key.

  2. Define query parameters including the city name, API key, and unit.

  3. Use requests.get() to send a GET request.

  4. Check the response status code, and if it's 200, parse the JSON data and output relevant information.

Summary

This article introduced the basic methods and advanced techniques for making HTTP requests using the requests library, covering aspects such as setting timeouts, verifying SSL certificates, using proxy servers, setting request headers, handling redirects, managing cookies, using authentication, handling error responses, and session management. A practical case study demonstrated how to use these techniques to develop a simple weather query system. Mastering these tips can help developers handle network requests more efficiently and improve the robustness and security of their programs.

Recommended