In this post, I will walk readers through how I removed obsolete “Turbo pages” for our site from Yandex, a search engine with its own index and crawler that is primarily used in Russia and some other parts of the former Soviet Union.
Our use of Yandex
Yandex, which can be roughly described as a Russian version of Google, offers English-language webmaster tools for its search engine. I discovered this fact around the time I started running The New Leaf Journal back in the spring of 2020. Our original SEO solution, Yoast SEO, makes it easy to validate site ownership for Google, Bing, and Yandex webmaster portals. My priority was verifying our site with Google and Bing, but I also verified our site with Yandex for two reasons:
- It was easy to do; and
- At that time, DuckDuckGo drew some of its results from Yandex’s search index.
When I verified with Yandex, I was under the impression that it could help us appear in DuckDuckGo search results. That assumption was likely mostly mistaken. Even when DuckDuckGo did draw some results from Yandex’s search index, it never mixed results from Yandex with results from Bing, DuckDuckGo’s primary source. Thus, it is likely that my efforts to have our site verified with Yandex played little to no role in our eventually appearing on DuckDuckGo.
I wrote an article about my experiences in the Yandex Webmaster Portal back in October 2020. Therein, I gave an English webmaster’s perspective on Yandex’s tools. One thing I noted was that Yandex took its time indexing The New Leaf Journal. In furtherance of my objectives, I enabled a feature called Yandex Turbo. What is Yandex Turbo? Read on to find out.
Setting up Yandex Turbo
Yandex describes Yandex Turbo as follows (see original):
Turbo pages are lightweight versions of pages that open quickly even when you have a slow internet connection.
But how do they work? We read on to learn more…
These pages load about 15 times faster than their original versions. To achieve such speed, we use a page layout that is optimized for mobile devices, and the Yandex network infrastructure: Turbo page data is stored on the company’s servers.
In 2020, my goal was to have The New Leaf Journal indexed in as many places as possible. I was under the impression at the time that submitting our site to Yandex Turbo would help with Yandex (I am not sure to what extent that assumption was correct). While The New Leaf Journal was not the highest performing site in 2020 due to our then using cheap shared hosting, I was not particularly concerned about performance. I simply wanted to be indexed.
The only way to have Yandex create “Turbo” pages for a site is to feed it special Turbo RSS feeds. As I explained in my 2020 article, I used a WordPress plugin called RSS for Yandex Turbo. While the entire user interface is in Russian, it is readily readable with a machine translation and very easy to configure. I used the plugin to create Yandex Turbo feeds and entered the feeds in the proper fields in the Yandex Webmaster Portal. Everything worked exactly as expected and Yandex accepted our feeds. I tip my hat to Flector, the plugin developer, for making something easy enough for me to use when I really did not have a great idea of what I was doing.
Disabling Yandex Turbo in 2022
I opted to disable Yandex Turbo in the spring of 2022. By this point, we were well-established with Google and Bing (and by extension, DuckDuckGo) and had received very little traffic from Yandex. Additionally, DuckDuckGo suspended its relationship with Yandex in early 2022 on account of Russia’s invasion of Ukraine, negating one of the reasons for the time that I had spent ensuring that our site was included in Yandex’s index in the first place.
Moreover, I had some issues with how Yandex Turbo works. I had a better understanding of how Turbo worked in 2022 than I did in 2020, and I generally do not like the idea of having our site served by third parties (note that we have never used AMP or a CDN and we have not used third-party fonts since October 2020). Moreover, while the RSS for Yandex Turbo is a light plugin that has no noticeable effect on performance, I would rather not have plugins that we do not really need. I am sure that Yandex Turbo makes sense for many sites, especially those that rely heavily on Yandex for referrals – but for us it makes little sense.
I removed our Turbo feeds from the Yandex Webmaster Portal and then deleted the plugin. Now, if on the surface you think that it sounds like I did this correctly, you would be mistaken. I did not do this correctly. Had I only read the fine-print in the plugin, I may have done it correctly. What did I do wrong? Let us skip ahead about 10-11 months to January 2023.
Discovering a Yandex Turbo problem
For whatever reason, Yandex periodically revokes webmaster privileges – I assume the trigger involves not using the Webmaster account. I renewed my privileges on one occasion, but I let them lapse somewhere in 2022. It had little impact – there were a couple of months in 2022 wherein we received a decent number of Yandex referrals, and it ranked as high as our 5th best search engine referrer on a couple of occasions.
In mid-January 2023, Bing, without explanation or recourse for appeal, blacklisted The New Leaf Journal from its web search results (not image search results, oddly). I would say that DuckDuckGo followed suit, but since DuckDuckGo is little more than a Bing front-end, it is perhaps more accurate to say that it was dragged along for the ride. What is more, our fourth best search engine referrer, Brave Search, appears to have been affected by the Bing blacklisting even though New Leaf Journal articles can still be found in Brave.
Losing our second and third biggest search referrers and having our fourth biggest referrer crippled creates a sub-optimal situation for The New Leaf Journal. (Note: Google is our biggest search referrer by a wide margin and we have not seen any difference in our Google performance subsequent to the Bing blacklisting.) Yandex, however, did not blacklist The New Leaf Journal. It dawned on me that there is a non-zero chance that Yandex may now be in the running to be our second-largest search referrer. With this realization in mind, I decided to re-verify our site so I could access Yandex’s webmaster tools again.
I undertook some basic house-cleaning on my Yandex Webmaster account and checked to see if there were any errors or warnings that I could address to ensure that we were in good standing with a search engine that has not banned us. We had no serious issues, but I took the time to understand the few warnings and notes we had (far better than I did back in 2020) and address them.
On January 19, 2023, I received what I found to be a peculiar notice from Yandex.
Yandex warns me that my Turbo pages are not included in search results
I received the following notification from Yandex on January 19, 2023:
On one hand, I would hope that our Turbo pages are not included in search results. As I explained, I removed our Turbo sources in early 2022 and stopped outputting Turbo feeds at the same time. Why would they be in the search results?
However, the notification indicated to me that Yandex was a bit confused about my turbo page status. I decided to investigate.
I went to the Turbo data source section in Yandex and confirmed that we had no Turbo sources. However, Yandex remained under the impression that we had a few hundred Turbo pages. This was a problem, of course, because Yandex could not locate said pages. Sure enough, I confirmed that Yandex still had Turbo page URLs for nearly all of our articles prior to my removing our Turbo feeds in early 2022.
The Data Sources page contained a “Delete Turbo Page” link. This brought up a text box wherein I could insert a URL. Since we had about 500 URLs, this seemed like it would be annoying – but why not see what happened? I tried a URL and Yandex failed to delete it. This was, of course, because Yandex could not delete a Turbo Page that it could not load.
So how exactly could I delete the Turbo URLs from Yandex’s index?
I checked Yandex’s documentation for removing Turbo Pages. The second method suggested was the one that I had just tried – deleting individual Turbo Pages. This would not work since Yandex could no longer load the pages due to the fact that I had removed the data sources nearly one year prior. The second method involved adding code to all of the affected URLs. This sounded dreadful in light of the fact that there was about 500 of them.
I wondered whether there was an easy way out of the quagmire. It then occurred to me that I could re-install the plugin I had used and see if it had an answer.
RSS for Yandex Turbo has the answer
I reinstalled RSS for Yandex Turbo, not to start outputting Turbo Feeds again, but instead to see if I could solve Yandex’s confusion. I used Firefox Translate to translate the plugin settings page, which remains entirely in Russian.
Lo and behold, the very first page of the settings explained how to correctly delete Turbo feeds:
The settings explain that in order to make Yandex purge URLs it had created Turbo for, you must create a new Yandex Turbo feed which will add an attribute of “turbo=false” to URLs that you want removed.
Lest there was uncertainty, the plugin instructions explained unequivocally:
This is the only way to get Yandex to disable turbo pages for your site.
(Man, if only I had read this in the first place.)
The settings for RSS for Yandex Turbo have two options. The first is to delete selected articles and the second is to delete all Turbo pages. I checked both, even though one would do. For selected articles, I copied the list of Turbo URLs from Yandex into the “List of deleted links” field in the plugin and created a “trash” Turbo feed. I then submitted this feed to Yandex Turbo, which accepted it within 48 hours. Essentially, the idea is that Yandex will consume this trash Turbo feed and learn that the associated URLs should no longer have Turbo pages.
I submitted the feed and waited.
Yandex customer support
After I had changed some settings in Yandex Turbo while I was trying to figure out how to purge our former Turbo articles, I received an email from Yandex customer support asking me if I was having some issue with Turbo and if there was anything they could do to help. Out of curiosity, I explained that I was no longer using Turbo because it was not necessary for my site and asked for confirmation that I had followed the correct steps to remove the old Turbo articles and, if so, how long the removal would take. The support person quickly replied, stating that he understood and confirming that I had submitted the deletion feed correctly. He advised me to make sure that Yandex Webmaster indicated that all of the Turbo pages were deleted before removing the feed. I thanked the support person for his or her quick and helpful answer and took a moment to give a positive customer review.
What does the deletion feed look like?
We are no longer outputting our Yandex Turbo deletion feed since the Turbo pages have now been deleted.
However, I thought that it would be interesting to show you what it looked like. In light of the fact that it had contained about 500 articles, I will just copy a snippet with the first five articles to give you an idea of what it looks like and how it works (note, no indentation in sample).
<rss version="2.0">
<channel>
<turbo:cms_plugin>C125AEEC6018B4A0EF9BF40E6615DD17</turbo:cms_plugin>
<title>The New Leaf Journal</title>
<link>https://thenewleafjournal.com</link>
<description>Where the leaves are perennially virid</description>
<language>ru</language>
<generator>
</generator>
<item turbo="false">
<link>
https://thenewleafjournal.com/wordpress-resources-page/
</link>
</item>
<item turbo="false">
<link>
On Maury Povich’s Retirement
</link>
</item>
<item turbo="false">
<link>
Joining AlternativeTo and Rising to the Rank of Contributor
</link>
</item>
<item turbo="false">
<link>
On Not Mining Dogecoin 〜 A Vignette
</link>
</item>
<item turbo="false">
<link>
Read the Fine Print – Buying a Cheap Mini PC For a Home Server
</link>
</item>As you can see, all the feed does is tell Yandex that each of the listed URLs should not be in Turbo.
Reflections
Readers should take a few lessons from my story.
First, read the fine print. Had I carefully read the directions on the RSS for Yandex Turbo plugin before I deleted it and my Turbo feeds, I would have never had an issue with orphaned Turbo data in Yandex. While I doubt it made a significant difference, we would rather not confuse search engines.
Second, there is a specific lesson here for WordPress administrators. When you delete functionality from WordPress, things are often left behind in your database. This is actually not the case with RSS for Yandex Turbo itself – the plugin cleans up after itself. But many WordPress plugins do not. I recently deleted some old data from my database that had been left by plugins I installed in 2020. The Yandex Turbo lesson shows that this extends beyond WordPress itself. When you delete some feature or functionality, take care to ensure you understand how to fully delete it so you do not have problems later.
Thirdly, while I was impressed with Yandex’s customer service (especially compared to Bing’s utter lack of customer service), its process for deleting old Turbo data is unnecessarily byzantine. In a scenario like mine, it should be possible to purge old Turbo URLs from the Webmaster Tools panel without having to retrace my steps and create an entire RSS feed to do it. Yandex’s Turbo documentation presumes that once you set up Turbo feeds, you will only delete individual pages. I doubt that I am the only webmaster who at one time produced Turbo feeds and subsequently changed course – especially since many sites likely use additional tools such as plugins for the task. Yandex should make removing old Turbo data more intuitive.
Finally, while I doubt that I will have a reason to install Yandex for RSS Turbo again, I thank Flector, the plugin developer, for his great work. As I noted, the plugin itself has very clearly explained settings for creating Turbo feeds and for deleting Turbo data. It also carefully explains every option and how it works. The plugin is lightweight and fully cleans up after itself. If you are a WordPress admin who wants your site in Yandex Turbo, I give RSS for Yandex Turbo my full recommendation.