The Python requests
module is a commonly used HTTP library that simplifies sending HTTP requests to websites and retrieving response results.
Compared to the urllib
module, requests
is more concise and user-friendly.
To send HTTP requests using requests
, you must first import the module:
import requests
After importing, you can send HTTP requests using methods provided by requests
to a specified URL. For example:
Example:
# Import the requests package import requests # Send a request x = requests.get('https://www.pmeve.com/') # Return the web content print(x.text)
After calling a request using requests
, a response
object is returned, which contains detailed response information such as status code, headers, and content:
print(response.status_code) # Get the response status code print(response.headers) # Get the response headers print(response.content) # Get the response content
More response information is detailed below:
Attribute or Method | Description |
---|---|
apparent_encoding | The encoding of the response |
close() | Closes the connection to the server |
content | Returns the response content as bytes |
cookies | Returns a CookieJar object containing cookies |
elapsed | Returns a timedelta object showing the time between sending the request and receiving the response |
encoding | The encoding used to decode r.text |
headers | Returns the response headers in a dictionary format |
history | Returns a list of response objects for each redirection (URL history) |
is_permanent_redirect | Returns True if the response is a permanent redirect, otherwise False |
is_redirect | Returns True if the response was redirected, otherwise False |
iter_content() | Iterates over the response content |
iter_lines() | Iterates over the lines in the response |
json() | Returns the JSON object from the response (only works if the response is in JSON format, otherwise raises an error) |
links | Returns the parsed link headers |
next | Returns the next request object in the redirection chain |
ok | Checks the status_code value and returns True if less than 400, otherwise False |
raise_for_status() | Returns an HTTPError object if an error occurred |
reason | Description of the HTTP status (e.g., "Not Found" or "OK") |
request | Returns the request object that generated this response |
status_code | Returns the HTTP status code (e.g., 404 for "Not Found" or 200 for "OK") |
text | Returns the content of the response as a Unicode string |
url | Returns the URL of the response |
Example:
# Import the requests package import requests # Send a request x = requests.get('https://www.pmeve.com/') # Return the HTTP status code print(x.status_code) # Return the status description print(x.reason) # Return the encoding print(x.apparent_encoding)
Output:
200 OK utf-8
Requesting JSON Data
To request a JSON file and retrieve its content:
Example:
# Import the requests package import requests # Send a request x = requests.get('https://www.pmeve.com/try/ajax/json_demo.json') # Return the JSON data print(x.json())
Output:
{ "name": "Website", "num": 3, "sites": [ { "name": "Google", "info": ["Android", "Google Search", "Google Translate"] }, { "name": "PMeve", "info": ["PMeve Tutorials", "PMeve Tools", "PMeve WeChat"] }, { "name": "Taobao", "info": ["Taobao", "Online Shopping"] } ] }
Requests Methods
The requests
methods are shown in the following table:
Method | Description |
---|---|
delete(url, args) | Sends a DELETE request to the specified URL |
get(url, params, args) | Sends a GET request to the specified URL |
head(url, args) | Sends a HEAD request to the specified URL |
patch(url, data, args) | Sends a PATCH request to the specified URL |
post(url, data, json, args) | Sends a POST request to the specified URL |
put(url, data, args) | Sends a PUT request to the specified URL |
request(method, url, args) | Sends a request using the specified method to the specified URL |
Example Using requests.request()
:
# Import the requests package import requests # Send a request x = requests.request('get', 'https://www.pmeve.com/') # Return the HTTP status code print(x.status_code)
Output:
200
Setting Request Headers
To set request headers:
Example:
# Import the requests package import requests # Define parameters kw = {'s': 'python tutorial'} # Set request headers headers = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36" } # Use params to pass a dictionary or string query parameters response = requests.get("https://www.pmeve.com/", params=kw, headers=headers) # Check the response status code print(response.status_code) # Check the response encoding print(response.encoding) # Check the complete URL print(response.url) # Check the response content (Unicode format) print(response.text)
Output:
200 UTF-8 https://www.pmeve.com/?s=python+%E6%95%99%E7%A8%8B ... other content...
POST Request Example
The post()
method sends a POST request to a specified URL, typically formatted as follows:
requests.post(url, data={key: value}, json={key: value}, args)
The url
is the target URL.
The
data
parameter is a dictionary, list of tuples, bytes, or file objects to send to the specified URL.The
json
parameter is a JSON object to send to the specified URL.The
args
include additional parameters such as cookies, headers, or verify options.
Example:
requests.post(url, data={key: value}, json={key: value}, args)
Output:
<p style='color:red;'>This content was requested using the POST method.</p><p style='color:red;'>Request time: 2024-05-26 17:30:47</p>
POST Request with Parameters
Example:
# Import the requests package import requests # Form parameters myobj = {'fname': 'PMeve', 'lname': 'Boy'} # Send a POST request x = requests.post('https://www.pmeve.com/try/ajax/demo_post2.php', data=myobj) # Return the web content print(x.text)
Output:
<p style='color:red;'>Hello, PMeve Boy, how was your day?</p>
Additional Request Parameters
You can include additional parameters such as headers, query parameters, and request bodies in your requests:
headers = {'User-Agent': 'Mozilla/5.0'} # Set request headers params = {'key1': 'value1', 'key2': 'value2'} # Set query parameters data = {'username': 'example', 'password': '123456'} # Set request body response = requests.post('https://www.pmeve.com', headers=headers, params=params, data=data)
This code sends a POST request with custom headers, query parameters, and request body.
In addition to basic GET and POST requests, requests
supports other HTTP methods such as PUT, DELETE, HEAD, and OPTIONS.