Menu
Table of contents
It is one of the most useful tools that really saves you hours of time. So also useful for entrepreneurs who occasionally work on the findability of their website. In this article I will give you 9 beginner tips, so you can get started with Screaming Frog a little faster.
Tip 1: Don't buy a license right away: the free version will get you a long way too
As a diehard SEO specialist, you absolutely need a paid version of Screaming Frog (SF), but as a novice online marketer or entrepreneur, it is not necessary to have the paid version.
The free version already contains enough features and functions.
Especially if you only work with small websites with less than 500 URLs. After all, that's the number of URLs the free version of Screaming Frog accepts.
Bonus tip: possibly go find someone who also has SF and ask if you can use his/her license. After all, you can use a license multiple times.
Tip 2: Above all, use the HTML filter
When you use Screaming Frog, the filter defaults to "All. That means you'll also see all the images, JavaScript and all sorts of other things that won't be of much use to you as a beginning Screaming Frogger.
Therefore, make sure the filter in the upper left corner is set to "HTML.
If you have this filter turned on, you will see all the text/HTML pages of your website including all sorts of useful SEO indicators and On Page SEO elements. Think about:
- Whether the page can be indexed
- SEO title (and the length of the title)
- Meta description (and the length of the meta description)
- H1s
- H2s
- Canonical
- No-follow
- Meta keywords
- Internal links
- Etc.
In other words, all things that are important for the findability of your website.
Tip 3: Ignore, when convenient, the robots.txt
I always refer to the robots.txt as your website's manual. In this manual, you can say to Google: I don't want you to include that page in Google.
By default, Screaming Frog will respect the robots.txt. But that's not always convenient for your Screaming Frog printout.
Fortunately, Screaming Frog has a handy feature for that. A feature that lets you ignore the robots.txt.
In case you want to know where this function is: you can find it at the top under configuration and robots.txt.
Tip 4: Take advantage of the bulk export feature
A few inches to the right of the filter function, you have an export function. If you click on that, you will export all the information you see in front of you now.
But within Screaming Frog there is another export feature: the bulk export. With the bulk export you can run an export of all kinds of useful information with 2 clicks. For example:
- All canonicals
- All images (without alt tag)
- All 4XX links
- All internal links
- All outgoing links
- All orphan pages
- And much more.
Tip 5: Export the sitemap with Screaming Frog
Chances are your website already has a sitemap, because it was generated by the Yoast plugin, for example. But if you don't have a sitemap, if you want to know how Screaming Frog sees your sitemap or if you want to know what your image sitemap looks like, Screaming Frog has a handy feature for that as well.
The position can be found at the following location:
Then you can adjust all kinds of settings here, too. Think of pages that are on noindex, PDFs, pages with a canonical and more.
Tip 6: Analyze the Crawl Tree & Directory Tree Graph
I personally find this one of the most unusual and interesting features of Screaming Frog. Namely, this is a feature that is completely focused on the internal link structure / internal link building of a website.
And that is one of the most underappreciated parts within SEO.
With the Crawl Tree and Directory Tree Graph, you can see at a glance how your website is internally linked. You can find this feature in the following place:
Such a Tree Graph looks like this (and when you see it, I'm sure you'll understand what you got from this):
Tip 7: Change the order of the dates you see (by clicking on a heading)
Let's go back to the HTML filter for a moment. Suppose you want to know which pages have no meta description, for example.
Then you can click on 'meta description' and it will adjust the order of the data. If you click 1 time then all pages without meta description are on top. If you click one more time, then all pages with meta description are on top.
By the way, this does not only apply to the HTML filter or to the meta description, but this works with every component within Screaming Frog.
Tip 8: Play with the user agent (when Screaming Frog doesn't work)
Sometimes you may put a Web site in Screaming Frog and all you see is the home page, without all the data.
There may be something wrong with the website then, but Screaming Frog may also be blocked by the hosting party / SEO specialist in question.
You can get around this by modifying the user-agent. This means that it then appears to the Web site as if Google, for example, is trying to come along instead of Screaming Frog.
Or another user-agent.
Should you want to know which user agents are all there, I recommend testing the function.
Tip 9: Upload a list of URLs to analyze (list function)
By default, Screaming Frog's mode is set to spider. This means that Screaming Frog crawls through the website you enter like a spider to collect all the data.
But what if you want to use Screaming Frog to analyze a specific list of URLs? You can! With the handy 'list' feature.
By setting Screaming Frog to list, you then have the ability to upload specific URLs yourself.
This upload can be done in the following 4 ways:
Written by: Daniel Kuipers
Daniel is the founder of Online Marketing Agency. He constantly scours the Internet for the latest gadgets and tactics and blogs about them in understandable language. Well, sometimes.
1 response to "9 beginner tips for Screaming Frog"
Hello Daniel,
Good blog to read, thanks!
Only certain topics in your blog you can no longer change in the free feature, like robot.txt. Just thought I'd mention it.