Google Updates How Web Crawler Works

The world’s biggest search engine has recently modified how Google web crawler works. A recent update on Googlebot’s help document revealed that it would only crawl the first 15MB of a webpage’s HTML to determine its search ranking. Any page element after the first 15MB will be excluded from the calculation process. 

A Few Exemptions

For those who are worried that Google won’t crawl the images, videos and other vital elements on your web pages, don’t be. The search giant’s web crawlers will separately fetch resources referenced in the HTML, including images, videos, JavaScript, and CSS. The file size limit is only applicable to uncompressed data.

Google Search Advocate John Mueller went on to further clarify the issue after certain members of the SEO community asked if Googlebot would disregard the text placed under images beyond the cutoff. Mueller explained that it is specific to the HTML, adding that embedded content or resources pulled in with IMG tags are not part of an HTML file. 

Why You Should Care

With Google updating its search algorithms yet again, business owners and their respective SEO teams must pay close attention and apply the necessary fixes to ensure they are following the best practices. This means placing important content near the top of your web pages to ensure Google’s search bots will crawl and index them.

Furthermore, you must structure your code to place SEO-relevant information within the first 15MB of your HTML. Instead of encoding videos and images directly into the HTML file, you should compress them to ensure the best results.

As SEO experts, we at Springhill Marketing recommend that you keep the size of your HTML pages to a maximum of 100KB. If your HTML files are smaller, it is unlikely that Google’s latest update will affect your website. If you want to be sure, you can check your page size using tools such as Google Page Speed Insights. 

Tips on Reducing Your Page Size

With 15MB being a significantly large amount of HTML, surpassing Google’s size limit can be difficult, particularly if your webpage only contains text. However, if you go over the limit, there’s a risk that your site has underlying issues that you should address immediately.

Also, it is good SEO practice to keep your web pages small. How Google web crawlers work notwithstanding, today’s web users have a significantly shorter attention span. Recent studies showed that 1 in 2 people would leave websites if they took longer than three seconds to load. Keeping your web pages small means faster loading times, translating to more visits and more chances of conversion.

To reduce page size, here are a few tips you should consider. 

1. Resize web images

Web images can easily take up your online resources, so you should never upload pictures without resizing them first. Also, you should pay attention to the size of the image’s frame or “container”. If it has a maximum width of 500 pixels, see that you don’t publish a photo bigger than 500 pixels.

If you want to provide visitors with high-resolution images, here’s a workaround that you can use. Upload a photo of the same size as its container. Then, link it to its high-resolution version to prevent the picture from weighing down your webpage. 

2. Compress pictures

Besides resizing web images, you should also consider compressing them. This will help reduce their file size, making them load faster. Besides photo editing software such as Adobe Photoshop and Gimp, there are various online tools that you can use to compress pictures before uploading them. They include:

• TinyPNG
• io
• JPEGmini

If your website runs on WordPress, you should consider installing plugins such as Optimole and EWWW Image Optimiser. These plugins will automatically compress web images as you upload them online. 

3. Limit custom fonts

While they help set your website apart from the others, custom fonts can easily add weight to your web pages and slow your site down. This is why you should limit their use as much as possible. While using a few custom fonts won’t add that much weight, overdoing them can hurt your site’s overall loading speed.

If you can’t help but use several custom fonts on your website, consider creating a WOFF2 file to reduce their file size. WOFF2 is a web font file created using the 2.0 version of the Web Open Font Format and used as a compressed container. It can support TTF and OTF fonts. 

Let Our SEO Experts Improve Your Web Pages

With Google updating how its search bots crawl web pages, you should make the necessary improvements to ensure your pages will be indexed properly. Also, it would be best to keep your page sizes small to prevent them from going over Google’s 15MB limit and slowing down your website.

If you require the assistance of seasoned SEO experts, we at Springhill Marketing are happy to help. Please contact our team to get access to our reliable, effective, and award-winning SEO services.