How to send a post with python-requests
A POST request may be a particular sort of HTTP method used once we send data to services online. We use them on internet sites that use forms – once we log in, once we send messages or post a picture. Applications also use POST requests to interact with other services to send data with JSON being a standard format for exchange.
The Requests library is one of the foremost popular HTTP client libraries for Python. It currently has over 45k stars on Github, with downloads on PyPI of 115M a month! It makes sending POST requests much simpler programmatically than having to send data via a headless browser. With a headless browser, we’d got to write scripts to navigate to a site, move between fields, populate each of them with the specified data before finally submitting the info. By sending a POST request, we skip straight to the ultimate submission step. Requests are also a much smaller library than a browser leading to better performance and memory usage.
In this article, we’ll cover the way to construct a POST request using Requests and the way it can make the method much simpler for us.
Building a JSON POST Request with Requests
- Set the Request Method to POST
- Set the POST data
- Set the POST Headers
- POST-JSON Data
As an example, start by building a JSON POST request the hard way. don’t be concerned. Requests will simplify this for us later! We’re using the httpbin.org service, which returns a JSON response detailing the content that was sent.
-
Set the Request Method to POST
Requests feature a really simple HTTP verb-based design, meaning that to urge a response we call it’s.post() method supplying our URI as an argument. It also provides similar methods for GET, PUT and PATCH requests.
import requests
r = requests.post("https://httpbin.org/post")
2. Set the POST data
To actually send some data, we supply a knowledge argument. By default, data is shipped using an HTML form as we might get by submitting any form online. Requests set the content type to ‘application/x-www-form-URL encoded, so it is not necessary to line any headers.
<code class="language-python" data-lang="python">import requests
import json
r = requests.post("https://httpbin.org/post",
data={"key": "value"},
)
Inspecting the response from the httpbin.org service, we will see what was sent as form data under the “form” key.
<code class="language-python" data-lang="python">>>> r.text
'{\n "args": {}, \n "data": "", \n "files": {}, \n "form": {\n "key": "value"\n }, \n "headers": {\n "Accept": "*/*", \n "Accept-Encoding": "gzip, deflate", \n "Content-Length": "9", \n "Content-Type": "application/x-www-form-urlencoded", \n "Host": "httpbin.org", \n "User-Agent": "python-requests/2.25.1", \n "X-Amzn-Trace-Id": "Root=1-60df1a04-0384d3ce7d9ac00b5855064b"\n }, \n "json": null, \n "origin": "**.***.**.***", \n "url": "https://httpbin.org/post"\n}\n'
If we would like to send JSON, we will supply a JSON formatted string.
<code class="language-python" data-lang="python">import requests
import json
r = requests.post("https://httpbin.org/post",
data=json.dumps({"key": "value"}),
)
3. Set the POST Headers
Our JSON example isn’t getting to work because it is now. With many services, we’ll likely get a 400 HTTP status code. To stop this we also have to inform the service we’re calling that our data is JSON so it is often handled correctly. To do so, we set the ‘Content-Type’ to ‘application/json’ within the request headers:
<code class="language-python" data-lang="python">import requests
import json
r = requests.post(
"https://httpbin.org/post",
data=json.dumps({"key": "value"}),
headers={"Content-Type": "application/json"},
)
-
POST-JSON Data
Requests make it very easy to scale back all this right down to a way simpler call. we will just supply a ‘json’ argument with our data. Requests will correctly set our headers and encode the JSON formatted string for us automatically.
import requests
r = requests.post('https://httpbin.org/post', json={'key':'value'})
Reading JSON Responses
By inspecting the response from our service, we will see the info returned may be a JSON formatted string too. This is often only text, so we’d got to parse it ourselves to use it within our Python script.
<code class="language-python" data-lang="python">>>> r.text
'{\n "args": {}, \n "data": "{\\"key\\": \\"value\\"}", \n "files": {}, \n "form": {}, \n "headers": {\n "Accept": "*/*", \n "Accept-Encoding": "gzip, deflate", \n "Content-Length": "16", \n "Content-Type": "application/json", \n "Host": "httpbin.org", \n "User-Agent": "python-requests/2.25.1", \n "X-Amzn-Trace-Id": "Root=1-60df0aaa-3105fc35436042571335fa22"\n }, \n "json": {\n "key": "value"\n }, \n "origin": "**.***.**.***", \n "url": "https://httpbin.org/post"\n}\n'
- Making POST requests within a Session
As a final example, let’s log in to Hacker News using Requests and submit some form data to log in together with a request.Session() object. Any cookies we receive are going to be stored within the object, and can be reused in later .get() calls to the session.
For our example, we’ll search the response text of the news page to verify if we’re shown the logout link indicating we’re being served a logged-in page.
<code class="language-python" data-lang="python">import requests
session = requests.Session()
r = session.post("https://news.ycombinator.com/login", data={
"acct": "username",
"pw": "***not-telling***"
})
r = session.get("https://news.ycombinator.com/news")
logout_pos = r.text.find("logout")
print(r.text[logout_pos-20:logout_pos+20])
# <a id='logout' href="logout
Our link is within the response indicating we’re logged in. we will now still interact with the location using the session object via Requests.
Conclusion
We have seen just how simple Requests makes it to post data to sites or services and the way we will reduce much of the common code in our applications by using it. This easy design has certainly contributed to its success within the Python community, making it a firm favorite with developers.
Abhishek Kumar
More posts by Abhishek Kumar