Is your website easily accessible by search engines? If a search engine has trouble crawling your site then all of your SEO efforts will be wasted. Imagine how much energy you might waste. However, there is a good way to crawl your site like a search engine would!
The SEO Spider tool from Screaming Frog is a free application that lets you crawl your site like a search engine and find any trouble spots you can correct.
Enjoy secure, fast hosting for your WordPress site with our WordPress Hosting plans.
- Why you should use a site crawler
- How to install the SEO Spider tool
- How to Crawl Your Site With SEO Spider
For this article, you’ll just need access to the Internet and your computer’s admin password for installing software.
The spider tool is free for up to 500 URLs of your site. If you have more than 500 URLs you’ll need to upgrade to the paid version.
Why You Should Use a Site Crawler
Site crawlers are an important tool for every webmaster or casual user who manage websites large or small. The crawler will give you detailed information that would otherwise take you hours to discover—and it does it in mere minutes. All you need to is provide your URL and set the crawler loose to do its work. After the crawl is complete, you’ll have important data like:
- Which URLs are being crawled and which are blocked
- What pages have broken links
- What pages have images
And much more. Having this data at hand will give you a laundry list of optimizations you can make to improve your site’s crawl efficiency.
How To Install The SEO Spider Tool
Below, we’re going to learn how to grab the installer scripts for the SEO Spider tool and activate them on your computer. The installation procedure may be slightly different for various operating systems, but if you use Mac, Windows, or Ubuntu, it should be self-explanatory.
- Go the Screaming Frog homepage
- Click Download under SEO Spider
- Select your operating system
When your download completes, open the download package and follow the installation instructions for your operating system.
How to Crawl Your Site With SEO Spider
Now it’s time to run a crawl of your site. For this part of the tutorial, make sure you have the primary domain of your website handy.
- Open SEO Spider on your computer
- Fill in your URL
- Click Start
When the crawl is complete, all of your data will be displayed in neat columns and tables. If you’re unsure about what any of the data means, just leave a comment below and we’ll help you figure it out.
Well done! You’ve just installed the SEO Spider tool and ran a crawl on your website to help you accomplish more with your SEO goals. For more SEO tips, check out our articles on Search Engine Optimization!