I have written regularly about using a feed reader to stay on top of interesting articles from around the web. My feed reader of choice at the moment is a Miniflux instance hosted using Pikapods. It works well in the majority of cases, but I have been running into an issue in recent months where a site which I subscribed to or was interested in subscribing to blocked my Miniflux instance from retrieving its feed (in some cases the site advertises its feed). I suspect that the culprit in most cases is Cloudflare or some other bot-protection mechanisms. Back in 2023 I wrote about running into a version of this issue when trying to save articles in Wallabag (an open source read-it-later app) and how I circumvented the problem with the help of the Internet Archive’s Wayback Machine. Unfortunately, the Wayback Machine does not work for staying on top of regularly updating RSS feeds. Fortunately, I employed a helpful and well-known tool to stay on top of feeds from sites which block my Miniflux instance: Google News.
Brief Introduction to Google News RSS and ATOM Feeds
Every Google News search has an RSS and ATOM feed. (If you are unfamiliar with feeds and feed readers as a general matter, please see my introduction article.) For example, the feed for Google News’s main algorithmic feed is https://news.google.com/news/rss. See a 2019 Stack Exchange post and a 2024 article by Artem Bugara for information about different Google News RSS search parameters, including regular search query- and geolocation-based feeds. But I was looking for a very specific type of Google News query feed: A domain-specific Google News feed.
Creating a Domain-Specific Google News RSS Feed
In August 2024, I wrote about how to use search engines to search a specific domain (also see my 2021 article on using DuckDuckGo to search Project Gutenberg). Google News, like regular Google Search, also allows for domain-specific searches.
We will use Anime News Network for this demonstration. As the name suggests, Anime News Network is an English-language anime news website (it also covers novels, manga, and video games) with sometimes questionable opinions. It not only has RSS and ATOM feeds, but it advertises them on a dedicated, albeit somewhat hard-to-find, Newsfeed page. I had been using an Anime News Network ATOM feed for several months without any problem. But one day I found that it was no longer updating because my Miniflux instance was being blocked. I tried the most obvious fixes to circumvent the issue but to no avail. It was then I decided to turn to Google News.
In theory, there are two ways of using Google News to pull up new Anime News Network articles. A blunt and imprecise method would be to use a search query, perhaps “Anime News Network” – and possibly add extra parameters to try to filter out anything from other websites that may sneak through (e.g., this article could appear in the feed because it includes the phrase “Anime News Network”). I did not want this method; I wanted a method which would only return results from animenewsnetwork.com.
First, we need a domain-specific search query. For Google News, the format is as follows:
https://news.google.com/search/?q=site:{domain}
If you are using Google News’s search box, you can conduct a domain-specific search by searching: site:{domain}
Now we need to run a search for Anime News Network. Using method one, the URL is:
https://news.google.com/search/?q=site:animenewsnetwork.com
For the RSS feed, we simply add rss/ between google.com and /search. See the generic URL below:
https://news.google.com/rss/search?q=site:{domain}
In the case of Anime News Network, the RSS feed is:
https://news.google.com/rss/search?q=site:animenewsnetwork.com
The feed can be added to any RSS feed reader or parser. Google News feeds include up to 100 results in chronological order.
Refining Google News Feeds With Additional Search Terms or Operators
Above, I demonstrated how to limit a Google News search to a specific domain. Because this is a regular Google News search, we can also add additional terms or operators to narrow the search and, by extension, the search’s feed. Ahrefs has published a handy list of Google Search operators.
Let us return to Anime News Network for an example. Let us imagine that I was only interested in Anime News Network stories which reference visual novels. I could run the following Google News Search:
https://news.google.com/search?q=site:animenewsnetwork.com “visual novel”
Like the basic domain search, this can be turned into a feed:
https://news.google.com/rss/search?q=site:animenewsnetwork.com “visual novel”
This feed will include only results from Anime News Network which include the exact phrase visual novel.
(One could also omit the quotation marks and search generally for visual novel without limiting the search to exact matches.)
Bonus: Other Google News Search Operator Feeds
It is worth noting that because ‘site:’ itself is a Google Search Operator, it is possible to use different operators to create unique Google News Feeds. While my focus in this article is on domain-specific feeds, I will give one fun example of how this could work. Ahrefs’ Google Search Operators directory includes the operator ‘stocks:’. This is for limiting searches to stock information for a ticker. It uses Apple’s aapl as an example. A stock ticker search can be turned into a feed in the same way as a domain-specific search:
https://news.google.com/rss/search?q=stocks:aapl
Again noting that this article is about Google News feeds for single domains, there are interesting feed-based use-cases for Google Search operators other than ‘site:’. (The Artem Bugara article I linked earlier in the article includes other interesting suggestions.)
Refining Domain-Specific Google News Feeds By Lengthening the URL
There are many cases wherein one may be interested in one section of a website but not another. Above, I showed how to limit an Anime News Network search items containing a specific phrase. It is also possible to limit the search to a specific part of the domain. Anime News Network includes the article categories in its URL strings. News articles include /news/ after the base domain. We can create a feed for articles with /news/ following the base domain:
https://news.google.com/rss/search?q=site:animenewsnetwork.com/news
This feed will exclude Anime News Network URLs which do not have /news/ after the base domain. For example, articles in Anime News Network’s review section use the animenewsnetwork.com/review/ format. These URLs would be excluded from the above example search. In some cases, you may be able achieve a similar effect by adding an inurl: operator to your search. See the following example:
https://news.google.com/rss/search?q=site:animenewsnetwork.com inurl:news
This trick can be used similarly to what I demonstrated in an article about category- and tag-specific feeds. However, it comes with the same limitation. The extent to which this trick may work depends on how a site structures its URL. For example, while The New Leaf Journal has category- and tag-feeds (see our feed directory), category and tag names are not part of article URLs. Thus, it would not be possible to create a category-based domain-specific search for our site on Google News.
Google News ATOM Feeds Instead of RSS
Most guides to feeds for Google News searches (including the two I liked above) discuss RSS feeds. I noticed that Google News searches also have ATOM feeds. The ATOM feeds are the same as RSS feeds save for the fact that the search string includes atom instead of rss. See my example below for the sort of domain-specific feed I detailed above:
https://news.google.com/atom/search?q={domain}
For the majority of people using these feeds in feed readers, RSS vs ATOM should not make any practical difference. Still, I thought it was worth noting since I have written about both types of feeds and advanced users may have reasons for preferring one type of feed over the other. (I only defaulted to RSS in this article because most online guides I came across on Google News feeds reference RSS feeds instead of ATOM feeds.)
Domain-Specific Google News RSS Feeds For Sites Without Feeds
I use Google News to fetch an Anime News Network RSS feed because Anime News Network blocks my Miniflux instance from fetching its feed. Domain-specific Google News RSS feeds can also be useful for sites which do not offer feeds. For example, I enjoy reading Tablet Magazine. Unfortunately, Tablet does not offer a feed and recently discontinued its free Substack newsletter (which I subscribed to via RSS). But if I add https://news.google.com/rss/search?q=site:tabletmag.com to my feed reader, I have all of the most recent Tablet Magazine articles as if this online magazine did offer a real RSS, ATOM, and/or JSON feed.
Domain-Specific Google News Feed Limitations
There are several downsides to using domain-specific Google News feeds.
Firstly, having to rely on Google is less than desirable, especially in light of the fact that I recommend feed readers as an alternative to big tech tools such as Google News (I will note however that domain-specific Google News searches necessarily limit Google’s algorithmic discretion). While I do not regularly use Google Search or stay logged into an account in the browsers I use to access my Miniflux instance, some people may prefer to avoid Google wholesale. It is also worth considering that Google has killed many of its own products and there is no guarantee that it will continue indefinitely to provide RSS feeds for Google News searches (or YouTube for that matter).
Secondly and more practically, Google News feed links do not go directly to the source article. Instead, they first go to Google News before redirecting to the source article. For this reason, Google News feeds cannot be used as full text feeds in Miniflux (it is possible some commercial feed readers or other options may handle Google News feed links more effectively – but this is not something I have personal knowledge of). I noted in an earlier article that people who use feed readers have different opinions on whether it is preferable to read full text articles in a feed reader or to navigate from the feed reader to the original website. For me the issue is site-specific and based on my set-up (primarily using the Miniflux progressive web app on mobile), I do not consider navigating to the original website to be a major hindrance in most cases. However, Google News feeds may be a less appealing option for people who prefer to read everything in their feed readers.
Thirdly, some feeds come with hitches. I have found in a few cases that domain-specific Google News feeds include non-post items such as About pages and category and tag archives. In some cases, adding additional search terms or operators or searching only a narrow section of the domain in question may resolve the issue. I have used feed filters in my feed reader to remove non-post items along with items that are otherwise not of interest to me. (I discussed feed reader-based feed filters generally in the context of my growing tired of seeing articles about Palworld in 2024.) This issue is site-specific, so there is no general rule for understanding whether non-post content will appear in a domain-specific Google News feed without checking the Google News search results or feed.
Alternatives to Using Domain-Specific Google News Feeds
My (currently) single-user Miniflux instance being blocked from accessing certain feeds is not necessarily a universal problem. Many people who use feed readers may never encounter the issue (I do not recall having this problem when I used a local Android app called Handy Reading). For those who do, feed readers sometimes offer built-in methods. For example, Miniflux has a feed-specific checkbox to disable HTTP/2 to avoid fingerprinting (that previously worked for a few feeds I was being blocked from) and allows people to designate proxy URLs (see Miniflux docs).
People wanting to subscribe to sites which do not offer feeds is a more common and universal problem. There are non-Google News solutions here which may indirectly serve as fixes for site-blocking issues in some cases.
Feedle
Feedle is an interesting search engine where every search has an RSS feed. Unlike Google News, searches that do have feeds is a selling point rather than a buried feature. Feedle has two significant limitations compared to Google News. Firstly, Feedle curates its collection of websites, meaning that there is a good chance the site in question is not part of Feedle’s collection (do note however that Feedle has open submissions). Secondly, Feedle does not, to the best of my knowledge, explicitly support domain-specific searches. However, I ran a search for our domain:
https://feedle.world/search/?query=thenewleafjournal.com
And it returned results only from our domain and in chronological order. (Note that there is no reason to use Feedle for The New Leaf Journal because we offer a full selection of full-text feeds.) Thus, if the site you need a feed for is on Feedle, this should work as a good alternative to Google News.
Projects for Creating Feeds For Sites Without Feeds Or Turning Partial Feeds Into Full-Text Feeds
There are two well-known and self-hostable tools called RSS Bridge and RSS Hub which can be used to create feeds for sites which do not offer any or, alternatively, in order to turn partial feeds into full-text feeds. Another interesting tool in this area is morss.it, which is self-hostable and free to use (I have used morss.it to good effect). FiveFilters is a paid solution for this while PolitePol is a freemium option. Open RSS is a free resource which provides full text feeds for a growing collection of websites. Some individual feed readers and services may have built-in tools for handling sites that do not offer feeds. For example, it was possible to use Handy Reading (a local open source Android feed reader I have experience with) to look for links on sites without feeds. (I am sure there are other tools out there – my list is not exhaustive.)
Kill the Newsletter
Finally – if the site in question has a newsletter but no feed, Kill the Newsletter is an excellent free and open source service for turning newsletters into feeds (I use it for a few newsletters).
Conclusion
As a website admin who has recently had issues with “AI” bot traffic, I can understand why more and more sites are using bot protection mechanisms, but these may have a negative impact on legitimate traffic from feed readers and read-it-later tools. However, the trend is nevertheless disconcerting to me as someone who relies on feeds personally and advocates for others to use feeds to take control of their online reading. It would be bleak if bot protection mechanisms incidentally drive some people away from curating their online reading and to the big tech algorithms as the alternative.
Fortunately – there are ways around feed reader challenges such as sites blocking legitimate feed reader traffic or sites not offering feeds at all. Domain-specific Google News feeds may be the simplest and most intuitive solution, although they are not perfect and come with the drawbacks I noted above. I am often critical of Google and there are many articles blaming Google’s decision to axe its former feed reader, Google Reader, for the supposed death of RSS. However, I will take a moment to praise Google for continuing to offer feeds for Google News searches and YouTube, even if Google seems to be wholly unenthusiastic about advertising their existence. It would be great to see some more privacy-focused search engines such as Mojeek also offer search-based feeds. On the off-chance that someone with decision-making authority at Google reads this article, I hope that it inspires the Google employee to provide readily accessible official documentation for Google News and YouTube feeds. If it instead inspires the Google News employee to send these useful feeds to the infamous Google Graveyard, I offer my most sincere apologies.
