Seven tips to improve your SEO via Screaming Frog

Screaming Frog can do a lot of things. Some of the features are difficult to use, while others are significantly easier to get to grips with. I've collected seven tips that are mainly easy to use. Of course, it depends on which CMS you use. For example, if you use WordPress, then you're already in a good position to improve your website SEO-wise.

1. Check if metatags are the correct length - and catchy!

2. Get an overview of your image sizes and alt-texts

3. Find broken links (404 pages)

4. Analyse redirects when migrating a page

5. Integration of G Suite products and other SEO tools

6. Crawl your page the way Google wants to

7. Verification of your schema markup

It's important that your meta tags are the right length, but also catchy. When your website is exposed on Google based on a specific search criterion, you only have a certain number of characters to make good use of. An example is the following:

The above example shows that the meta description is too long and needs to be corrected. But what if you have several pages where the meta description or title is too long? To find out, this is where we turn to Screaming Frog. By entering the domain into the program and then navigating to Overview -> Meta Descriptions we get an insight into which meta descriptions are too long, too short, duplicated or other critical things that need to be improved.

Once you have an overview of which pages need to be optimised, you can find them in the backend of your website. If you use WordPress, you can install Yoast SEO (I wonder if you've already done that? ;)), and navigate to SEO -> Tools -> Mass-editing. Here you can edit your meta titles and descriptions in bulk - which is easier than clicking into each page.

At Webamp , we've even created a small application to help you write compelling meta-titles and descriptions. This can help increase your CTR (click-through-rate) and thus increase the number of visitors via the organic search results on Google. You can find our CTR tool right here.

A website needs to load quickly. That's why there are demands on how your content loads - especially images. The most popular search engine, Google, primarily analyses your website via the mobile version - also called mobile-first indexing. So what does this mean for you? It means you need to focus on how images load, how much space they take up and what format your images are in.
In this context, alt-texts are also important. In short, alt-texts describe your images in a written context. This is particularly relevant for users with partial or full sight impairment, or users who use a screen reader or have a poor internet connection. This allows them to get a description of what the image contains and means if they cannot see it.

You can read much more about what Google recommends as best practice in image optimisation right here.

Posting a picture on a website is therefore not as simple as uploading a party picture on your favourite social media. You need to take into account the size of the image itself, as well as its dimensions. Via Screaming Frog, you can see both how much your images take up and which ones are missing alt captions. This can be found under Overview -> images.

Once you have found the list of images, you can sort them by size and then click on ''Inlinks'' at the bottom of the application. This will give you an idea of which images are taking up the most space and where they are located on your website.

Next, find a way to crop and/or compress your images. You can do this in Photoshop, for example. If you only need to compress your images, you can use online services such as Compress PNG or TinyPNG.

With Screaming Frog, you also have the option to find broken links that point to either external sources or previous subpages on your site. It is generally accepted that these links are fixed so that they are redirected to the correct new page or equivalent content that appeared on the now ''broken'' page. Not only for the sake of a search engine, but also for the user experience.

For example, it is very obvious to shut down the site if you are not redirected to the right content - this will create a poor user experience all other things being equal. That's why it's important to get these links right. This is how you do it:

As with the list of images, you have the option to find the location of the links on your site - and in any anchor text.

In addition to 404 pages, you can also see which redirects actually work, which links are blocked by robots.txt and much more. In general, you should be aware of what response codes your server sends to the client visiting your website, as this can influence how a search engine indexes it.

If you've created a new website and want to ensure that the existing SEO value will be ''transferred'', it's important that your redirects are just right. Otherwise, you could potentially lose SEO value if Google indexes one of your old landing pages high and you don't tell the search engine that the content may have changed location.

With Screaming Frog you have the possibility to make sure that the migration from the new to the old site is done in a proper way and that all redirects are done.

First you need to switch modes from Spider to Cunning. This is done under Mode in the menu:

Once this is done, upload the list of your old URLs. There are several options, but I have chosen to do it manually:

Then you need to set the program to follow your redirects. This is done under Spider -> Advanced -> Always Follow Redirects.

Once the crawl has been completed, select Reports -> Redirects -> All Redirects, after which you can download a .CSV file, which you can import into e.g. Excel or Google Sheets and analyse the results there.

For the sake of clarity, I've made a sample where you can see that one of Webamp's clients' redirects is done properly and sends traffic to the desired URL.

In this example I have done it with three different ones; Google Analytics, Google Search Console and Ahrefs. Next, I have the ability to compare data from the three sources to analyze each landing page and how they can be optimized.

Screaming Frog has its way of crawling a website when it needs to visualise and tell you how to optimise it. But in fact, you can also have the program crawl the website as if it were Google. Why is that relevant, you ask?

That's because it's Google that decides how your website will be indexed - and therefore how it will rank. Furthermore, the search engine also indexes your website based on a mobile-first principle (as mentioned earlier in this blog) - i.e. it crawls your website as if it were a mobile device. That you can change this setting in Screaming Frog is important because of two factors:

1. By mimicking the Googlebot Smartphone crawler, you can quickly find out what problems Google may have when crawling and rendering your website from a mobile-first principle

2. Using the Google Smartphone User-Agent in the application, you can compare your crawls and Google's own when analysing server logs - to see where the difference lies.

This is done relatively easily. By choosing Configuration -> User-Agent in the menu, you can easily switch this.

After the program has done its work, you can take a look under Overview -> Structured Data to see if there is anything that needs to be fixed:

Those were the seven tips I had to offer. You can use Screaming Frog for much, much more, but these tips will at least equip you to optimise the most basic of opportunities - especially if you're only now starting to explore search engine optimisation and your options for increasing organic traffic.

By researching which keywords your main competitors are targeting, you can get inspiration for your own keyword analysis.

Do they rank on keywords you should rank on too? And are there gaps in the market?