What is the Screaming Frog SEO Spider?
Produced and maintained by the UK based SEO agency Screaming Frog, the SEO Spider is a desktop-based crawling tool that allows you to quickly crawl, analyse and audit a sites’ onsite SEO.
The SEO Spider allows you to quickly export all the key onsite SEO elements (e.g. URL, page title, meta description, headings, status codes etc) to Excel so you can easily build an effective list of SEO recommendations and areas for improvement.
What does Screaming Frog do?
The Screaming Frog SEO Spider crawls your website and gathers key data about the onsite SEO health of the site, giving you all the information you need to make informed decisions and build effective SEO strategies.
Some of the insights it can provide include:
Broken links
The crawler will give you a list of all the broken links (404 pages) on your site, which you can export to fix, or send to your developer.
Redirect and canonical issues
Find all the temporary (302) and permanent (301) redirects on your site, and identify any problematic redirect loops.
You can also see all the canonical tags that are used across your site, to make sure there are no mistakes with how they are being used.
Page titles and meta data
See the title tags and meta description tags for all of your pages, sort them by length and identify all the duplicated ones.
Duplicate and thin content
As well as showing any duplicated pages, Screaming Frog can also identify partially duplicated elements across your site like title tags, meta descriptions and headings.
The word count of every URL is also displayed so you quickly identify pages with thin content.
Robots directives
A mis-placed X-Robots-Tag directive such as ‘noindex’ or ‘nofollow’ can be a costly SEO mistake. Using Screaming Frog you can get a quick review of all the Robots tags in use across your site, as well as a list of all URLs blocked by your Robots.txt file.
Create XML Sitemaps
If you want to have full control of what URLs are in your XML Sitemap, then you can configure Screaming Frog to produce an optimised file for you – including the priority, last modified and change frequency settings.
How much does Screaming Frog cost?
You can download the Screaming Frog SEO Spider for free, which allows you to crawl and analyse 500 URLs on your website.
If you have a small/medium website with not too many pages this is a great cost-effective option – however, as well as the 500 page crawl limit, the free version also doesn’t allow for any configuration or integration of third-party data like Google Analytics.
For the complete Screaming Frog crawling experience, the paid version costs £149.00 per year for one licence.
If you are part of a team that would all benefit from full access to Screaming Frog, there are discounts available for multiple licence purchases:
- 1 – 4 licences – £149.00 per licence per year
- 5 – 9 licences – £139.00 per licence per year
- 10 – 19 licences – £129.00 per licence per year
- 20+ licences – £119.00 per licence per year
