Toll Free: 1-877-695-7388

GTA: (647) 699-2838

Search Engine People
  • SEO
  • SEM
  • CRO
  • Display
  • Blog
  • Why Us
  • Contact
  • Join Our Team
  • Get A Quote

Toll Free: 1-877-695-7388

GTA: (647) 699-2838

Crawlability: Just How Well Can Google Crawl JavaScript

Jordan Kasteler | September 8th, 2016
Tweet67
Share26
Share
Pin
93 Shares

google-crawl-javascript

Google has crawled JavaScript since 2008. The idea that Google can crawl these types of pages is nothing new. The problem? Google wasn’t able to gather a whole lot of amount of information from JavaScript pages, limiting their ability to be rendered and found over HTML websites.

Now, Google has evolved, or so it seems.

“Blogs that use JavaScript have started to see more pages crawled and indexed over the past year, which clearly means Google’s evolving its support of this language.”
-- Matt Banner, OnBlastBlog.

Which Google’s Bot Does The Crawling?

Some industry experts have speculated whether Google uses separate bots to crawl JavaScript websites versus regular HTML sites.

The quick answer is no, according to John Mueller, Webmaster Trends Analyst at Google. Still, Mueller was quick to follow up and say that JavaScript won’t always render the first time Google crawls the page.

“The timing of when Google crawls the code on the page and when it renders is an important one when testing and analyzing SEO tactics with JavaScript. With this in mind, you can have more realistic expectations of when you should see a difference in your page’s performance.”
-- Scott Lazerson, Connector.

With this in mind, what can you do to make your JavaScript sites more search engine friendly? Here are a few expert tips.

JavaScript Redirects

Some tests have been done to see if a JavaScript redirect can have the same impact as a 301 redirect (the type of redirect that shares the link love from the original page with the page it’s being directed to). Some show that Google treats these redirects the same, while others show otherwise. There doesn’t seem to be any rhyme or reason to it.

The key is to test your redirects and keep a close eye on them. Sometimes, it might take a little longer for Google to start sending traffic to the new page because it takes longer for the bots to crawl JavaScript.

DOM Elements

Another test done by Search Engine Land contributor, Adam Audette, looked at whether Google is able to read changes to the Document Object Model (DOM) made by JavaScript.

The DOM is an essential part of how a webpage is accessed, read, and built by a browser.

Although the DOM isn’t tied to a specific programming language, like JavaScript, it is the connection point between the search engines and the web pages and programming language. This is important because it’s how search engines are able to read what’s happening on a webpage when ranking it.

Adam's conclusion:

"Dynamically inserted content, and even meta signals such as rel canonical annotations, are treated in an equivalent manner whether in the HTML source, or fired after the initial HTML is parsed with JavaScript in the DOM"
-- Adam Audette

Nix All Ajax Crawling Schemes

If you’re creating a new website, steer clear of using Google's 2009 AJAX crawling scheme. If your current website uses this now, consider making a switch.

In 2009, Google recommended using the AJAX Crawling Scheme. They have since changed their tune.

Google reportedly discontinued their support for the AJAX Crawling scheme in 2015. Google still crawls sites that use it but recommends not using it on new sites.

google-progressive-enhancement

Progressive enhancement is the preferred method of site building.

Review Your URLs

The symbol # creeps into many JavaScript-driven website URLs. If your URLs have # Google won’t index the page won't crawl and index the part after the #.

The exception to this rule is any URLs with “#!” in the URL. URLs with “#!” are still considered acceptable by Google. (See the above crawling scheme, now deprecated)

Use The Search Console Fetch And Render Tool

As mentioned earlier, testing JavaScript pages is important. Google doesn’t seem to have a method to its madness when it comes to these pages.

To see how your pages appear to Google, use the Search Console’s Fetch and Render tool. Although this tool still won’t see any URLs with “#!” or “#” in the URL, it will still give you a good glimpse into what the powerhouse search engine is seeing on your website.

Limit Your JavaScript Files

We’ve determined that Google can crawl JavaScript pages. Still, too many JavaScript files could be doing you more harm than good.

To maximize how well Google is crawling your pages, it’s a good idea to limit how many JavaScript files you have on your website. If you have a good amount of these files on your website, it can timeout without the files being loaded or rendered.

Upload The Correct Sitemap

This might seem like a small tweak but it’s an important one.

When uploading your sitemap, be sure to use the right “lastmod” dates. This sends Google a signal of what’s been changed and updated on your website so their bots know where to crawl and what to render.

One Last Consideration: Google Is Only One Search Engine

Remember, as you’re optimizing your website and incorporating JavaScript, Google is only one of the search engines you need to please. Although Google is the most widely used search engine, others, such as Bing and Yahoo, are also popular. These other search engines might not always support JavaScript, which could mean your pages are getting ignored by the bots that rank them.

Hand-Picked Related Articles:

  • No, Virginia, Google Doesn’t Index AJAX Sites
  • We Tested How Googlebot Crawls Javascript And Here’s What We Learned
  • What Is Unobtrusive JavaScript? (Part of Progressive Enhancement)

* Adapted images: Public Domain Dedication (CC0) Public Domain, pixabay.com via getstencil.com

Tweet67
Share26
Share
Pin
93 Shares
Posted in SEO, Web Design

About the Author: Jordan Kasteler

Jordan Kasteler is the SEO Director of Hennessey Consulting. His work experience ranges from co-founding BlueGlass Interactive, in-house SEO at Overstock.com, marketing strategy at PETA, and agency-level SEO & marketing. Jordan is also an international conference speaker, columnist, and book author of A to Z: Social Media Marketing.

Jordan Kasteler

2 thoughts on “Crawlability: Just How Well Can Google Crawl JavaScript”

  1. undertruck says:
    September 11, 2016 at 8:26 pm

    “The symbol “#” creeps into many JavaScript-driven website URLs. If your URLs have “#” Google won’t index the page.”

    This is incorrect. Google will only exclude the part after “#”. Remaining url would be indexed.

    1. Ruud Hein says:
      September 13, 2016 at 6:39 am

      Very well said, undertruck. I’ve edited in that stipulation.

Comments are closed.

Recent Posts

  • Maximizing Your E-Commerce Sales:
    A CRO Audit Guide
  • Movin’ On Up! Why Migrating to Google Analytics 4 (GA4) Should be a Priority
  • A Year in Review: The Digital Marketing Trends That Defined 2021
  • The Basics of Video Marketing
  • Just How Much Do Google Reviews Impact Your SEO Ranking?

Categories

  • Analytics & ROI Analysis
  • Company News
  • Content
  • Conversion Optimization
  • CRO
  • Display Advertising/RTB
  • Email Marketing
  • En Español
  • En Français
  • Inbound Marketing
  • Lead Nurture & Marketing Automation
  • Local Search
  • Marketing
  • Mobile
  • Partnership Marketing
  • PPC
  • PR
  • SEO
  • Social Media Marketing
  • Web Design

Additional Posts

How To Use AdWords Auction Insights in Your Marketing Strategy

September 7th, 2016 | by Troy Evans

25 Blog Post Templates To Make Blogging Faster

September 6th, 2016 | by Alicia Thomas

6 Deadly Sins That Can Kill Your Content Strategy

September 5th, 2016 | by Harsh Agrawal

LET'S TALK

Need more information or want to get in touch?

Get in touch!
  • SEO
  • SEM
  • Display
  • Blog
  • Why Us
  • Join Our Team
  • Contact Us
  • Local SEO
  • Small Business SEO
  • Enterprise SEO
  • International SEO

LOCATION

1305 Pickering Parkway,
5th Floor Pickering, L1V 3P2

PHONE

Toll Free: 1-877-695-7388
Greater Toronto Area: (647) 699-2838

Social

© Search Engine People Inc. 2023 – Canada’s Top Digital Agency
© SEP 2023 – A Search Engine People Company | Privacy Policy

Search Engine People