IndexNow - Sustainable Search Engine Indexing

In mid-October of 2021, Microsoft Bing announced they would be launching a protocol for website owners to instantly index their pages, thus hopefully increasing their organic rankings with the respective search engines.

"IndexNow" seems an appropriate name, because it allows webmasters to notify search engines when they have updated their webpages, by the push of a button.

The way it works today, the search engines are looking for this new information continuously, until they find the information they are looking for.

This makes the IndexNow protocol a potential dream for Search Engine Optimizers (SEOs) - meant to maximize the efficiency of indexing, causing web pages to be displayed in the search results pages.

The IndexNow protocol can also possibly minimize the wasted energy of crawler robots looking for updates across the world wide web, which takes a toll on our environment.

Wait, what, crawler bots? They are computer bots that analyze (ie. crawl) web pages for useful information that people search for. If you want to learn more about crawling and indexing, we recommend this article: How Search Engines Work.

Impact on Search Engine Optimization

For someone who wants their website visible in the search results, such as an SEO professional or website owner, this is also a great advantage to have their data shared faster and easier.

There are already some protocols in place for important and time-sensitive pieces of information, such as Google’s Indexing API for Job Posting and Live Streaming.

But what if you could tell the search engine immediately when you posted a guide on how to pick your next electrical bike, or what fantastic new products your industrial technology company has just released?

Experienced SEOs know that sometimes these things take time, and sometimes they take no time at all.

Director of Strategic Integrations at Duda (an award-winning website builder service), Russ Jeffrey, believes the traditional approach to search indexing using sitemaps is outdated, and that we should look towards the future and be able to push content to the search engines for them to crawl it.

I think all SEOs would agree that having a submit button and knowing that you have done your part would be relaxing - to say the least.

“World Wide Waste”

You may not be aware of this, but every time you use the internet for whatever reason, you are adding to the global pollution by cause of energy usage.

In his 2020 book “World Wide Waste” , Gerry McGovern gives some examples:

What the IndexNow resolves is the need for a search engine to crawl the web relentlessly, looking for updated information or new URLs to submit to the results pages.

This happens automatically, without any signaling from websites that they have published or updated anything at all, resulting in a waste of energy from all the computing power that runs continuously for these purposes.

The idea is as simple as it is splendid: You decide when new information or pages are to be submitted, thus saving the crawler bots time and energy.

Advantages of using IndexNow

Who can use IndexNow?

From the IndexNow documentation page , developers can implement the protocol as described.

But if you are not a developer or do not want to rely on one because [insert joke here], you can rest assured that many large platforms are embracing this protocol.

CloudFlare and Akamai are using it already, meaning that if you use these content delivery networks (CDN) you can also take advantage of its features.

Content Management Systems (CMS) such as WordPress and Duda have a plugin that supports IndexNow. While Duda has an automatic use of the protocol, Wix is rumored to tag along with the initiative.

Initially, Joost de Valk (CEO) of Yoast seemed a bit skeptical, but in June he announced that Yoast Premium will feature the IndexNow protocol.

For anyone using Optimizely/Episerver or Sanity, Geta will be happy to assist in the implementation of IndexNow.

Progress and news since the release

With Google Search possibly joining in on the functionality of IndexNow, the number of sites using the protocol would grow substantially.

Most of all, we hope that the initiative does improve Search Engine indexing and provide us all with a more sustainable world wide web.