Python find 404 errors

How To Handle 404 Error In Python Flask

In this tutorial, you’ll learn how to handle 404 error in Python Flask. While trying to access a non existing route, you will encounter a not found error. You can handle non existing route error or 404 error gracefully. You can either use render_template with error message to show a customized error message or return a 404 JSON error response. Let’s have a look at the code implementation.

Assuming you have a basic understanding of Flask web application framework. Let’s start by creating a file called app.py and add the following Python code :

from flask import Flask, abort app = Flask(__name__) @app.route("/") def hello(): return "Welcome to Python Flask." 

Save the above changes and run the app using the following terminal command :

# command to run flask app FLASK_APP=hello.py flask run 

Point your browser to http://localhost:5000 and you will the app running. Now if you try to access a non existing route you’ll encounter a 404 not found error. For example : http://localhost:5000/abc.

Handling 404 Error In Flask

To handle 404 Error or invalid route error in Flask is to define a error handler for handling the 404 error.

@app.errorhandler(404) def invalid_route(e): return "Invalid route." 

Now if you save the changes and try to access a non existing route, it will return “Invalid route” message.

You can either use render_template method to show a custom error page or return a JSON error response.

from flask import Flask, abort from flask import jsonify app = Flask(__name__) @app.errorhandler(404) def invalid_route(e): return jsonify('errorCode' : 404, 'message' : 'Route not found'>) 

Hope the above code helps. Happy Python coding 🙂

Источник

How to catch HTTP 404 error in Python

Hi guys! In this tutorial, we will learn how to catch an HTTP 404 error in Python. There are many instances when we encounter an HTTP error with error code 404 on the internet. This error indicates that the requested page was not found. This tutorial will teach you how you can use Python to spot pages with such an error. Let’s see more on this with example programs.

Catch HTTP 404 error in Python

There are many methods for catching a 404 error. We will import the urllib or urllib3 library in this post for catching a 404 error. These library has required methods and attributes for our purpose. Let’s see the code and understand how it’s done.

Method 1

See the below example to find out a page with an HTTP 404 error.

import urllib3 http = urllib3.PoolManager() error = http.request("GET", "https://www.google.com/404") if (error.status == 404): print("HTTP 404 ERROR")

The output for the above code is:

In the above example, we have used the PoolManager of urllib3 to create an http object. A PoolManager instance is needed to make requests. Then we have used the request method to get an HTTPResponse object for the web page ‘https://www.google.com/404’. This HTTPResponse object contains status, data, and header. We check whether the status for this page is 404 if so we print that this is an HTTP error.

Method 2

In this method, we will import the Python urllib library. Have a look at the given code and try to understand.

import urllib.request, urllib.error try: con = urllib.request.urlopen('http://www.google.com/404') except urllib.error.HTTPError as err: print('HTTP', err.code, 'ERROR')

The output of the program:

As you can see, the urlopen() method opens the given URL for us. We have used try and except blocks to catch the HTTP error using urllib.error.HTTPError. If any error is raised while opening the URL, the control is given to except block and there we print the error code for HTTP error.

Источник

Find and test invalid links and 404 errors using Python & Pytest

So, I recently managed to break something on CloudBytes/dev>. All the internal links on the site were broken, and I published the website.

I only noticed the error when the number of 404 erros increased significantly in the analytics report. So I set about creating a Python script to find the broken links and highlight them during the Continous Integration process I have setup.

The Solution Workflow to validate the links

I came up with the following set of steps to first populate the links and then validate if they exist.

1. Fetch the sitemap.xml file from the website & create a list of all links on the website
2. For each link in the list, check if it exists on the website and fetch the webpage
3. Find all the links in the webpage
4. Then request the webpage for each link and check if it exists
5. If the page exists, then add it to the list of valid links

Pytest Program to Scrawl & Test Website

      Step 0: Set the Site URL and Base URL

I set BASE_URL to localhost and SITE_URL to cloudbytes.dev. The reason for doing both is that in the CI process I use the localhost server to run the tests, but you can use the same program above to test a live website with minor changes.

Step 1: Fetch the sitemap.xml file from the website & create a list of all links on the website

  I also create a function to get all the links from a webpage passed as an argument

  Step 2 For each link in the list, check if it exists on the website and fetch the webpage

We do this in the test_internal_links function, where we get all the links in the sitemap

 Step 3 Find all the links in the webpage

 Step 4 Then request the webpage for each link and check if it exists

 Step 5 If the page exists, then add it to the list of valid links

 Finally, run this script by running the following command (You need to have pytest installed):

And this will scrape through the entire website and check if all the internal links are valid.

Источник

How to Fix HTTPError in Python

The urllib.error.HTTPError is a class in the Python urllib library that represents an HTTP error. An HTTPError is raised when an HTTP request returns a status code that represents an error, such as 4xx (client error) or 5xx (server error).

HTTPError Attributes

The urllib.error.HTTPError class has the following attributes:

• code : The HTTP status code of the error.
• reason : The human-readable reason phrase associated with the status code.
• headers : The HTTP response headers for the request that caused the HTTPError .

What Causes HTTPError

Here are some common reasons why an HTTPError might be raised:

• Invalid or malformed request URL.
• Invalid or malformed request parameters or body.
• Invalid or missing authentication credentials.
• Server internal error or malfunction.
• Server temporarily unavailable due to maintenance or overload.

Python HTTPError Examples

Here are a few examples of HTTP errors in Python:

404 Not Found

 import urllib.request import urllib.error try: response = urllib.request.urlopen('http://httpbin.org/status/404') except urllib.error.HTTPError as err: print(f'A HTTPError was thrown: ') 

In the above example, an invalid URL is attempted to be opened using the urllib.request.urlopen() function. Running the above code raises an HTTPError with code 404:

 A HTTPError was thrown: 404 NOT FOUND 

400 Bad Request

 import urllib.request try: response = urllib.request.urlopen('http://httpbin.org/status/400') except urllib.error.HTTPError as err: if err.code == 400: print('Bad request!') else: print(f'An HTTP error occurred: ') 

In the above example, a bad request is sent to the server. Running the above code raises a HTTPError with code 400:

401 Unauthorized

 import urllib.request import urllib.error try: response = urllib.request.urlopen('http://httpbin.org/status/401') except urllib.error.HTTPError as err: if err.code == 401: print('Unauthorized!') else: print(f'An HTTP error occurred: ') 

In the above example, a request is sent to the server with missing credentials. Running the above code raises a HTTPError with code 401:

500 Internal Server Error

 import urllib.request import urllib.error try: response = urllib.request.urlopen('http://httpbin.org/status/500') except urllib.error.HTTPError as err: if err.code == 500: print('Internal server error!') else: print(f'An HTTP error occurred: ') 

In the above example, the server experiences an error internally. Running the above code raises a HTTPError with code 500:

How to Fix HTTPError in Python

To fix HTTP errors in Python, the following steps can be taken:

1. Check the network connection and ensure it is stable and working.
2. Check the URL being accessed and make sure it is correct and properly formatted.
3. Check the request parameters and body to ensure they are valid and correct.
4. Check whether the request requires authentication credentials and make sure they are included in the request and are correct.
5. If the request and URL are correct, check the HTTP status code and reason returned in the error message. This can give more information about the error.
6. Try adding error handling code for the specific error. For example, the request can be attempted again or missing parameters can be added to the request.

Track, Analyze and Manage Errors With Rollbar

Managing errors and exceptions in your code is challenging. It can make deploying production code an unnerving experience. Being able to track, analyze, and manage errors in real-time can help you to proceed with more confidence. Rollbar automates error monitoring and triaging, making fixing Python errors easier than ever. Try it today!

Источник