Screaming Frog SEO Audit. 10 Useful Tricks
The journey to the first page of Google or Bing is a long one, and SEO is the mechanism to move your website to these prime positions. However, SEO is more than just keywords and fresh content; it is an ever-developing and changing set of rules and regulations that requires a site to change with the times. This is why SEO audits are an essential part of any website regular operation.
An SEO audit is a look at the technical aspects of a website. This check will look at everything on the site to ensure that the overall health of a website is excellent. These audits ensure that a website continues to maintain and improve its position by keeping it up to date, and in good standing via the crawlers. Failure to audit a site can lead to significant issues, including a dropped ranking in the leading search engines.
Screaming Frog is one of the best 'free' tools on the internet to complete an SEO audit. We have worked with this program for years, and below you will find some of our favourite useful tricks to ensure you get the most out of this fantastic tool.
1. 404 Pages and Where They are Placed - Response Codes Tab
Screaming Frog reporting can show a user where the various pages that are broken, otherwise called a 404 error, or which pages are providing a 500x response code. The 500x response code is in response to issues on the server-side, which in turn causes the page not to load for a prospective visitor. On the Google Search Console, you will see the same results, but Screaming Frog provides you with the primary source where the link is located.
As well, Screaming Frog is smart enough to detect if the specific link is placed in the navigation of the site, or even in the content of the site itself. This way, you can go into the site, and change that specific link with ease, saving you time, and your users a headache.
This trick will help you detect if you are linking to internal or external 404 pages or 500x pages, which in turn can help boost your SEO and user experience.
2. Indexable / Non Indexable Pages - Protocol Tab
In the protocol tab, you will find a great tool that will help you with indexing and deindexing. Screaming Frog allows you to go in and check if you have robot's txt robots tag which has made essential pages deindexed. The traditional way to check is to go through the "site:" operator, but with this tool, you can skip this entire step!
Plus, with Screaming Frog, you will be able to not only check if a page is indexed or not, but the program will also showcase the reason why the page is not indexed. This is extremely helpful and will save you hours, especially if it is a large site.
3. Broken and Heavy Images - Images Tab
Screaming Frog is a smart program and allows you to check for broken and heavy images within your site. Many SEO and website owners forget to check for broken images, and this can significantly affect a site's user experience. If a broken image is detected, the program will showcase the path for a picture so you can easily swap it out for the proper image.
As well, for those larger sized images, the program will flag them for your attention. The reason for this is that larger images can slow down a site as they require significant bandwidth to load. Reducing the overall image size can increase website speed and usability.
4. Multiply Canonicals - Canonicals Tab
Another great trick with Screaming Frog to use is to check if the canonical URL is correct and matching up with the URL that you want to be there. Google does treat a canonical as a suggestion or recommendation, but having everything in your site being consistent will help with your ranking.
Even if you have two canonicals on the same page, Screaming Frog will find these and check them. The default in Screaming Frog is only to check one. If you are operating multiple canonicals make sure to change the filter in Screaming Frog to multiply, to ensure that the program checks this.
5.Title / Description Length
The title and description length are essential to ensure that your SEO is as good as possible. Screaming Frog is a great tool to see what is visible in search results, and help you craft the perfect title and description for any search engine. You can use this trick to test our different lengths to ensure that you are encompassing everything your need to without losing words to the ether.
To use this tool, you will check the right tab, and click
Page titles -> Over 60 characters or for descriptions;
Page Description -> Over 155 characters.
This trick will not only improve your CTR, but will make your site just a little cleaner, and that is something we should all be aiming for in our work.
6. Zombie Pages - API Integration
"Zombie pages" have long been an issue in the SEO world, and Screaming Frog makes it a breeze to find, identify and remove pages that have not received traffic for years. Removing zombie pages will help a page improve its performance and increase its crawl ability with fresh content. For this trick you will need to do the following in Screaming Frog:
Go the Configuration Tab and click API Access
Next, you will connect and link your Google Analytics and Search Console accounts to Screaming Frog.
Then, you will set the date range for what you want to see, we suggest a year, but it could be less.
Then, you will go into your Google Search Console settings from within Screaming Frog and set the primary country (the country you are located in or selling into)
Finally, you will check the pages, and you will be able to see which specific pages are receiving the lowest impressions, clicks or good old traffic.
This tool makes removing zombie content a dream, and no matter if you are working with a newer site, or an established one, Screaming Frog makes finding zombie pages simple.
7. The Title is The Same as H1 Tag
One of the more consistent mistakes that we see on websites is that the title is the same as the H1 on a page. This Is not only an issue of bad content, but it can lead to harm to the site and the overall ranking.
Screaming Frog makes it easy to identify these issues, and the program can provide you with a report. To create and access it, you will need to do the following:
Click on the right tab and click Page Titles -> Same as H1
8. Duplicate or Missing H1 Tags
Related to the issues we usually see in useful trick 7, a duplicate or missing H1 tag can be a significant concern. Having no tag, or a copy will impact your page structure for Google, and it can directly negatively influence rankings as soon as the crawlers spot it.
Screaming Frog ensures that you can spot this and will provide you with a report to detect these issues and fix it asap. To access this, you will need to do the following:
Click the right tab, and then click H1 -> Duplicate or Missing and the report will generate.
9. Orphan Pages
When you are building a site, sometimes you add pages and forget to delete the pages. These are what are called orphan pages and are not always in the sitemap. Orphan pages are still part of your website, and depending on the content, especially if it is duplicate content, can negatively affect your ranking.
Screaming Frog can highlight any pages that are not in the sitemap XML so you can either delete them or add them into the sitemap. For this check you will need to:
Click sitemap -> Orphan URLs.
10. Crawl Treemap
One of the biggest timesavers on Screaming Frog is the ability to visualize the structure of the entire website with a single click of the button. SEO and website owners have had to build out these treemaps manually, and it is a bit of a headache.
To showcase these crawl treemap, you will need to:
Click Check -> visualizations.
Extra Tip
Compare Json Rendered and Text Only Version Crawl
Did you know that Google processes HTML, and then only when the Googlebot has resources available to process the content of the site will they then process your JavaScript content? That is right; all of that work that you have put into extra touches on JavaScript could be deferred for months. Thus, Screaming Frog empowers you to compare both the HTML and JavaScript sites to see what the Googlebot sees when it is crawling the site.
To check do this:
For the HTML
Run 1st crawl - Configuration - > spider - > Rendering - > Text Only
For the JavaScript
Run 2nd crawl - Configuration - > spider - > Rendering - > Javascript
Once these reports are done, you can then compare the two and find out what essential links that are on the JavaScript site will need to be moved over the HTML site. It can be a bit of a backbreaking process, but the benefits will be huge once the Googlebot arrives on your website!