Requests — how to tell if you’re getting a 404
Solution 1: Look at the attribute: Demo: If you want to raise an exception for error codes (4xx or 5xx), call : You can also test the response object in a boolean context; if the status code is not an error code (4xx or 5xx), it is considered ‘true’: If you want to be more explicit, use . Solution 1: Interpreter is your friend: Also, exceptions: Also notice that by default doesn’t raise exception if status is not : There is raise_for_status() method that does it:
Requests — how to tell if you’re getting a 404
I’m using the Requests library and accessing a website to gather data from it with the following code:
I want to add error testing for when an improper URL is entered and a 404 error is returned. If I intentionally enter an invalid URL, when I do this:
I want to know how to test for that. The object type is still the same. When I do r.content or r.text , I simply get the HTML of a custom 404 page.
Look at the r.status_code attribute:
if r.status_code == 404: # A 404 was issued. >>> import requests >>> r = requests.get('http://httpbin.org/status/404') >>> r.status_code 404 If you want requests to raise an exception for error codes (4xx or 5xx), call r.raise_for_status() :
>>> r = requests.get('http://httpbin.org/status/404') >>> r.raise_for_status() Traceback (most recent call last): File "", line 1, in File "requests/models.py", line 664, in raise_for_status raise http_error requests.exceptions.HTTPError: 404 Client Error: NOT FOUND >>> r = requests.get('http://httpbin.org/status/200') >>> r.raise_for_status() >>> # no exception raised. You can also test the response object in a boolean context; if the status code is not an error code (4xx or 5xx), it is considered ‘true’:
if r: # successful response If you want to be more explicit, use if r.ok: .
If your request is made inside another function, but you want to catch the error in a higher level, it is good to know that you can also get the status code directly from the exception. In my case I could not access the response since the HTTPError was raised before my function was able to pass on the response. I ended up doing the following:
try: r = function_calling_request(the_request) except HTTPError as e: if e.response.status_code == 404: return do_stuff_if_not_found() Why python requests get a 404 error?, The server could reject delivering the content if it does not like the «user agent» identification. It’s not nice to respont with a 404 error in
How To Handle Errors & Exceptions with Requests and Python
http — HTTP modules¶
http is a package that collects several modules for working with the HyperText Transfer Protocol:
- http.client is a low-level HTTP protocol client; for high-level URL opening use urllib.request
- http.server contains basic HTTP server classes based on socketserver
- http.cookies has utilities for implementing state management with cookies
- http.cookiejar provides persistence of cookies
The http module also defines the following enums that help you work with http related code:
A subclass of enum.IntEnum that defines a set of HTTP status codes, reason phrases and long descriptions written in English.
>>> from http import HTTPStatus >>> HTTPStatus.OK HTTPStatus.OK >>> HTTPStatus.OK == 200 True >>> HTTPStatus.OK.value 200 >>> HTTPStatus.OK.phrase 'OK' >>> HTTPStatus.OK.description 'Request fulfilled, document follows' >>> list(HTTPStatus) [HTTPStatus.CONTINUE, HTTPStatus.SWITCHING_PROTOCOLS, . ]