How to find broken links on the site and get rid of them

Some errors associated with search engine optimization sites lose their relevance over time. They either cease to be significant, or the level of automation of processes within the site almost completely negates the possibility of such problems.

Alas, the problem of broken links is not one of them: it has been a source of headache for webmasters and SEO specialists for many years.

1. What are broken links

First, let's define the concept of "broken links".

Broken links are links that lead to an inaccessible or non-existent page. First of all, these include all links with a 4xx server response code.

1.1. Causes and sources of appearance

The emergence of broken links can be caused by both incorrect configuration of the CMS (content management system), and a simple human factor (the webmaster made a mistake or did not update the link in a timely manner). Broken links appear if:

• the page address was specified incorrectly;
• the page was permanently deleted, but links to it still exist and do not redirect to an alternative page;
• after moving the page or moving the site, 301 redirects were not correctly configured and checked;
• due to a system error, the working page gives a 404 response code instead of 200 OK;
• there was a failure in the automatic update of data from an external source.

1.2. Implications for website optimization

The presence of broken links is regarded as a negative factor both in terms of search engine optimization and in terms of user experience. There are a number of reasons for this:

1. They spoil the user's experience of working with the site.
   Finding himself on a page with a 404 error instead of an important page for him, the user with a very high probability can leave the site and never return to it. This leads to an increase in the bounce rate, on the basis of which the search engine can draw conclusions that are very disastrous for your site. These, in turn, will directly affect your organic rankings.
   
   
2. They make it difficult for search robots to work.
   If too many broken links are found on a site, search engines can reduce the site's crawling budget (the number of links that a search engine robot crawls over a specific period of time) and lower the site's position in search results.
   
   
3. They are "sprayed" with reference weight.
   Broken links disrupt the natural distribution of link power within a site.

2. How to check a site for broken links

The main problem with broken links is that they appear all the time, and you can only partially prevent them from appearing. Based on this, you should regularly check the site for broken links manually. However, if the site includes thousands of pages, this method will take too much of your time and effort.

In total, there are several ways to find broken links, which differ from each other in terms of convenience and speed:

1. Online Services check broken links ( Online for Broken Link Checker ).
2. Dedicated Google Search Console tool.
3. Crawler ( Netpeak Spider ).
4. Special browser plugins ( Check My Links for Chrome, for example).

In order to better understand in what cases to apply to a particular tool, we will analyze the features of each of them separately.

2.1. Google Search Console

In the old GSC interface, data on errors, as well as on pages giving a 4xx response code, will be collected in the "Scanning" / "Scanning errors" section.

If you are using the updated interface of the Google Webmasters Panel, go to the "Index" / "Coverage" section and click on the "Errors" filter to see a list of pages with any problems, including a 4xx response code.

The disadvantage is that the list will include all links leading to your site, including links that the search robot will find on other sites. The list is updated after the next crawl of the site, which makes it impossible to track all technical problems that arise in real time.

2.2. Broken Link Checker

The free version of the Online Broken Link Checker service allows you to scan your site for broken links online. You just need to specify the site address, write captcha characters in the appropriate field and start a search.

A huge plus of the service is its accessibility, simplicity and no need to install any software on your computer.
But at the same time, there are a number of disadvantages, including:

• limitation of 3000 URLs per crawl;
• inability to find broken links to images, videos, documents and any files other than HTML;
• relatively low analysis speed, which is especially noticeable in the case of giant sites.

The service is suitable for a cursory analysis of small sites with a minimal likelihood of any broken links, except for links to HTML documents.

2.3. Netpeak spider

Using Netpeak Spider, you can scan an unlimited number of pages at any time and get an up-to-date report on all errors in the technical optimization of the site, including broken links, broken images and broken redirects (redirects leading to non-existent pages). In addition, with a crawler, you can limit the scan area yourself. For example, check a certain category in which the risk of broken links for some reason is higher than in others.

To find broken links using Netpeak Spider, you need:

1. Launch Netpeak Spider.
2. Enter the site address in the "Start URL" line, then click "Start". If necessary, you can restrict the search to a list of URLs (“List of URLs” in the main menu), a category (“Settings” / “General”) or a list of pages that meet a particular rule (“Settings” / “Rules”).
3. After Netpeak Spider completes the process, go to the "Reports" / "Errors" tab on the side panel and click on the "Broken links" item, if one is displayed in the list of errors found.
4. By clicking in the main table, an additional tab will open, which will display a list of links that give the server a response code of 4xx and higher.
   
   
   
5. To see where the found broken link is located, right-click on the page URL and select "Incoming Links" in the context menu.
   
   
   
6. When a table opens with a list of pages containing this or that broken link, in the upper left corner you need to click on the "Export" button. Save the received data in a convenient format: .xlsx or .csv.
   
   
   
7. You can also use a special report "Broken links", which can be found in the corresponding menu.
   

This solution is ideal for sites with any CMS and any number of pages, since the standard license does not imply any restrictions.

2.4. Special plugins for Chrome

For most popular browsers, such as Google Chrome, there are many plugins available that allow you to quickly scan a page in the current tab for broken links. Among them is Check My Links .

With the help of the plugin, all working links on the page are highlighted in green, broken ones - in red. In addition, if you specify the appropriate settings, the program can focus your attention on links with GET parameters, "empty links", links with the rel = "nofollow" attribute, and other links other than normal (a regular dofollow link with anchor text enclosed in <a> tag with href attribute).

The solution is suitable for those who work with sites with a small number of pages, but are very jealous of the purity of links within the site :)

3. How to fix broken links

Depending on the scale of the tragedy, to eliminate broken links, you can use several techniques - from the most straightforward to complex in terms of technical implementation. Below we have outlined a few scenarios that you can follow to get rid of broken links on your site.

3.1. Manual removal

If we are talking about eliminating no more than a hundred links that have accumulated over a long time or appeared only once due to technical work on the site, they can be deleted manually. For this procedure, we recommend the following algorithm:

1. Check if the address is correct and correct it if possible (and if you know what to fix it).
2. If the fix does not work, and the page or site that you previously successfully linked to is no longer there, find other relevant material and replace the link.
3. If none of the steps are successful, remove the link.

3.2. Setting up redirects

If broken links appear on your site constantly, and this happens due to some regular updates or changes (for example, deleting pages of inaccessible products), we recommend using a redirect. This could be:

1. 301 redirect to a relevant page with similar content, which can solve the user's problem.
2. 301 redirect to a category page or a section of the site to which the non-existent page belonged (here you need to be careful not to provoke a 404 SOFT error).
3. Refresh redirect from a custom 404 error page, which will notify you about the absence of the desired page and redirecting to the relevant category or section.

3.3. Creating a custom 404 page

Do you find building redirects too complicated and time-consuming to eliminate broken links? Well, there is another way to lower the bounce rate and push the user to continue surfing deeper into the site - creating a custom 404 page that will:

• entertain the user;
• show him options for further movement around the site;
• offer interesting, relevant content / pages.

In fact, you will not eliminate existing broken links in this way, but you can improve usability and keep users on the site.

Briefly about the main

Broken links are one of the critical technical errors that directly affect website optimization and its subsequent promotion in organic search. Broken links can be controlled using a panel for Google webmasters, specialized online services and plugins, as well as using crawlers such as Netpeak Spider .

There are several ways to get rid of found broken links:

1. Replace or remove links manually.
2. Set up redirects.
3. Create a custom 404 error page that will help keep the user on the site.

How do you deal with broken links on your website? Share your experience in the comments: maybe you know the techniques that would be worth adding to this material;)