I wrote an article in June 2021 about Microsoft’s Bing search engine briefly censoring image results for queries related to the 1989 Tienanmen Square massacre perpetrated by the Chinese Communist Party. In a subsequent article published this year, I wrote about Microsoft’s uncomfortably close relationship with the Chinese Communist Party as it relates to the Bing search engine. It is well established that Microsoft – to a far greater extent than Google – acquiesces to CCP censorship rules for the purpose of maintaining Bing’s presence in China (it is far from alone, however). There have been multiple incidents, including the 2021 Tienanmen Square story, wherein Bing’s China-based censorship escaped China’s borders. I documented in prior articles that Microsoft tends to blame examples of exporting CCP censorship as “bugs,” but has not yet provided any detailed account of what sort of bug causes Bing China’s restricting results for search queries related to the Tienanmen Square massacre to be exported to the United States and Europe.
While Bing’s occasional China lapses are troubling in and of themselves, I have a particular interest in alternatives to Google and Bing in the search sphere. My original article on the Tienanmen Square censorship incident focused on several purportedly alternative search engines which derive their search results from Bing’s index and repackage them with a different user interface and superior privacy policy. DuckDuckGo, Ecosia, Qwant, and Swisscows are among the better-known alternative search solutions which rely on Bing’s index. DuckDuckGo is the most-used of these alternatives in the United States. Bing’s decision to arbitrarily blacklist small sites can make those sites invisible to privacy-conscious searchers.
The CEO of DuckDuckGo, Mr. Gabriel Weinberg, occasionally chimes in on discussions on Hacker News when DuckDuckGo is mentioned. I discussed in brief one such occasion when there was a controversy regarding the DuckDuckGo web browser not blocking certain Microsoft trackers pursuant to DuckDuckGo’s arrangements with Microsoft. Hacker News played host to a lengthy discussion on Google blocking links to some YouTube download sites and projects in its search engine back in May. Hacker News discussions sometimes spin off in many different directions. Any mention of Google is likely to inspire some references to DuckDuckGo. I direct you to the following comment sequence from the discussion.
Hacker News user Tommstein wrote (quoted in part):
I abandoned DuckDuckGo during the Tank Man fiasco, but recently started giving it a chance every now and then.
The “Tank Man fiasco” pertains to the June 2021 incident wherein Bing censored image results for queries related to the Tienanmen Square massacre. That censorship trickled down to DuckDuckGo because DuckDuckGo’s image results come from Bing. Mr. Weinberg was apparently following the thread and took offense to this reason for abandoning DuckDuckGo. I quote the pertinent part of his reply:
First off, we do not remove any results ourselves for political purposes and in fact we have been banned in China for many years for that very reason. What you’re referring to was a temporary bug in our image search results from Bing that they promptly fixed. If they hadn’t fixed it promptly then we would have taken further action. That seems hardly cause to abandon us.
I noted in my article on the greater Bing-China relationship that DuckDuckGo has been banned in China for a long time. Moreover, to the best of my knowledge, Mr. Weinberg is telling the truth when he writes that DuckDuckGo does not remove results on its own for political purposes. He created some ambiguity in 2022 when he suggested that DuckDuckGo would down-rank sources it classified as Russian propaganda (before trying to clarify), but I questioned in my Bing-China article whether that would even be possible for DuckDuckGo given its substantial reliance on Bing.
The next part of Mr. Weinberg’s comment, however, comes off as questionable.
What you’re referring to was a temporary bug in our image search results from Bing that they promptly fixed.
Much like Microsoft, Mr. Weinberg describes the exportation of Bing China’s CCP censorship as a “bug.” To date, Microsoft has provided no evidence that the “Tank Man” incident or any of the other CCP censorship incidents were not volitional but were caused by bugs. Microsoft has never explained what sort of bug could cause the wires to cross between Bing China and the rest of the world, or why it has happened on multiple occasions. Perhaps it is a bug. Perhaps there are many bugs. Maybe Microsoft’s firewall between China Bing and worldwide Bing is less of a firewall than China’s great firewall. But given Microsoft’s financial incentives and its not being particularly forthcoming about the incidents, I see no reason to credit its explanations without evidence.
Now I understand that Mr. Weinberg cannot engage in the sort of speculation and question-raising than I am. Perhaps Microsoft showed him proof of the “bug” and he was satisfied with the explanation. Perhaps because DuckDuckGo’s business depends significantly on maintaining its business relationship with Microsoft, there are certain things he cannot speculate on in public. Whatever the cause, I do not have a business and Microsoft has de-indexed The New Leaf Journal without explanation for more than seven months. I have no aversion to speculating about Bing’s many bugs.
Mr. Weinberg stated that DuckDuckGo would have taken action if the “Tank Man” incident was not promptly resolved:
If they hadn’t fixed it promptly then we would have taken further action. That seems hardly cause to abandon us.
This may well be true. Perhaps DuckDuckGo closely monitors Bing censorship incidents and is prepared to take action when Bing’s upstream decisions affect DuckDuckGo. I know that there is a false perception among some that DuckDuckGo is censorship resistant or a good way to find articles that would be down-ranked by Google. Given DuckDuckGo’s reliance on Bing, this is only true to the extent that Bing’s index may be better for certain kinds of content than Google. However, if you want to find New Leaf Journal articles, Bing and all if its derivatives, including DuckDuckGo, are not the tools for the job. You can however find us with Google, Yandex, Brave Search, Mojeek, Marginalia Search, or just about anything that does not use Bing’s search index.
With all of that being said though – DuckDuckGo does have a clear use-case and I previously used it as my primary search engine for many years. Its reliance on Bing means that it will return the sorts of results that ordinary searchers are familiar with. It comes with a many search tools and features which give it general parity in many respects with Google and Bing (provided you find DuckDuckGo’s results to be acceptable). It has a reasonably strong privacy policy and provides robust JavaScript free search options.
Hacker News user tintedfireglass read Mr. Weinberg’s comment and raised a question:
If Bing censors the results, DDG is automatically going to do it too. I don’t think yall have your own web results. You buy it from Microsoft if I’m not wrong. If bing serves ddg censored results how would ddg try to “un-censor” the results??? is it even possible?
Readers of my Bing-China article will note that I had similar questions about how much leeway DuckDuckGo actually has.
Mr. Weinberg responded:
We actually always had a bunch of our own stuff, and still do, as well as work with other partners. For example, the number one module on mobile is local, and we don’t get any local stuff from Bing at all. Similarly, the number one module desktop is knowledge graph, and we don’t get that from Bing at all either. And yes, you can re-insert results if they are indeed censored, but Bing doesn’t really intentionally censor anything either as far as we can tell or we’d here a lot more about it. Occasionally something drops out of the index for some kind of bug reason that gets fixed.
The first part is technically true but not responsive to the comment. Commenter tintedfireglass is obviously asking about DuckDuckGo’s traditional web and image search results – that is, the list of links returned for a search query. Mr. Weinberg is responding by noting that DuckDuckGo provides separate results modules that are not sourced from Bing. For example, for some queries DuckDuckGo may post an instant answer from Wikipedia, weather information from Dark Sky, or map information from Apple Maps. These are all notable features that are separate from Bing. However, the regular search results returned for a query come from Bing’s index – these modules exist separate from Bing’s index. I think that it is commendable that DuckDuckGo has worked to diversity the information it can provide users within the terms and conditions it has to uphold to use Bing’s index. Moreover, I noted in my 2021 alternative search engine review (hat tip to Mr. Rohan Kumar) that Bing does not allow mixing its search results with other sources. For this reason, I think that DuckDuckGo’s description of where its search results come from is somewhat misleading. For example, many Hacker News discussions feature comments from very tech-literate individuals who seem to not know the extent to which DuckDuckGo’s regular search results rely on Bing (the confusion is perhaps aided aided by the fact that many people never use or have used Bing). DuckDuckGo does acknowledge the Bing reliance – but only in the penultimate sentence of the second paragraph on its sources page:
Of course, we have more traditional links and images in our search results too, which we largely source from Bing. Our focus is synthesizing all these sources to create a superior search experience.
I think this should be a bit more than an aside. As I noted before, DuckDuckGo has a clear purpose and utility as it is, without being deliberately vague about the source of its regular, traditional web, image, news, and video search results.
One comment from Mr. Weinberg struck me as interesting:
And yes, you can re-insert results if they are indeed censored, but Bing doesn’t really intentionally censor anything either as far as we can tell or we’d here a lot more about it. Occasionally something drops out of the index for some kind of bug reason that gets fixed.
I did not know that DuckDuckGo can “re-insert results” that Bing drops or blocks. How would this work in practice? Has it ever happened? Does the site have to be in Bing’s index and blocked from appearing (like The New Leaf Journal was from January to March) or can DuckDuckGo add results that have been entirely removed from Bing’s index (like The New Leaf Journal since March). I assume it is the former case – a situation like the “Tank Man” incident where Bing was merely blocking certain images from appearing in its results instead of removing those images from its index. But I would be curious to learn more about how much latitude DuckDuckGo actually has here. Clarifying what DuckDuckGo can actually do and acknowledging its limitations could go a long way toward increasing user confidence.
After I drafted this article, an interesting story popped up about Bings’ blocking of Techdirt, which is apparently a popular tech news site. I wrote about the incident in a short post here on site while also noting that I found Techdirt’s article to be a bit deficient compared to more detailed Bing ban accounts (while unduly focused on virtue signaling). But that is neither here nor there. For this article, let us turn to the Hacker News discussion. As is generally the case, Bing’s removal of Techdirt from its search resulted in its removal from DuckDuckGo’s search results. Mr. Weinberg chimed in on Hacker News that it was an error and that he was investigating the issue (such is what happens when Bing de-indexes a highly-trafficked site as opposed to small sites, I suppose). Mr. Weinberg’s initial statement did not reference Bing, and several Hacker News commenters jumped in with their questions and critiques. Suffice it to say, I generally agree with many of the comments, especially those which tried to pin Mr. Weinberg down on the source of DuckDuckGo’s traditional links, separate from its modules (special hat tip to commenter thewataccount, whose unanswered question was directly over the target, and to commenter WheatMillington, who noted the obfuscation that I had written in my draft a few hours before the Techdirt story broke).
Here, we will focus on Mr. Weinberg’s temporary solution to the Techdirt drama. In his initial comment, he stated that “the traditional web link [for Techdirt] is back up as well.” What does this mean? I ran a search and saw that Techdirt links were still not appearing in DuckDuckGo’s regular search results as of late Thursday afternoon. However, as Mr. Weinberg suggested, DuckDuckGo placed a link to Techdirt’s homepage above its results for the query Techdirt. I took a Wayback Machine screen capture of a DuckDuckGo search for Techdirt to show what the makeshift solution looked like. Is this what Mr. Weinberg meant a few months ago when he said that DuckDuckGo can “re-insert results” blocked by Bing? If so, DuckDuckGo is quite limited in what it can or will do. Mr. Weinberg clearly cares about the Techdirt issue, but the best solution he could muster on short notice was adding a special Techdirt link above the results for a Techdirt search (note the module on the right side of the screen is a DuckDuckGo instant answer which is not sourced from Bing). So long as Bing is blocking Techdirt results (it is unclear whether Bing is merely blocking the results or if it has de-indexed Techdirt), DuckDuckGo has not demonstrated the capability to re-add them.
Finally, I key in on one final point from Mr. Weinberg’s comment:
[B]ut Bing doesn’t really intentionally censor anything either as far as we can tell or we’d here a lot more about it.
To the extent that Mr. Weinberg says “anything,” this statement is false. Regarding “Tank Man” and the other China censorship incidents I discussed, there is no question – even from Bing’s defenders – that Bing was censoring results under the direction of the Chinese Communist Party. The question is not whether Bing was engaging in intentional censorship, but instead why that intentional censorship made it out of China. The incidents do not receive much publicity because there is not a great deal of interest in the CCP’s practice of exporting censorship, and I have discussed on site how it is welcomed by many people in power in the West. This segues into another point – Bing and all other major search providers certainly censor illegal content. For example, Bing China’s censorship censors content that is illegal in China. The controversy is that other parts of the world do not think that results which may be critical of the CCP should be illegal outside of China. I have documented on GitHub that Bing arbitrarily removes independent writing websites (such as our’s) from its index. Is this censorship? Who knows. While I have noted that I do not believe The New Leaf Journal was targeted for censorship, it is certainly possible that other sites unjustly hit with Bing manual actions have been. Good luck getting through to anyone from Bing to find out.
I conclude with my hope that DuckDuckGo considers making the information about where its traditional search results come from more clear in lieu of minimizing it, and that Mr. Weinberg becomes less defensive about the fact that those results are sourced from Bing (in a point of credit to him, I will note that I am not aware of representatives from other Bing-based search tools such as Ecosia or Qwant publicly addressing upstream Bing problems). If DuckDuckGo makes it clear to regular DuckDuckGo searchers in a way that cannot be easily misunderstood that the web search results come from Bing, Mr. Weinberg will have less explaining to do every time DuckDuckGo is affected by Bing behaving arbitrarily. Moreover, I would encourage Mr. Weinberg to state clearly what DuckDuckGo can and cannot do with respect to sites blocked or limited by Bing and to make it possible for DuckDuckGo searchers and webmasters to contact DuckDuckGo for support in those areas. With this clarity, people will understand what DuckDuckGo is and is not, and we will be able to readily appreciate DuckDuckGo’s strong commitment to user privacy and its efforts to add tools and modules (see my article on DuckDuckGo’s built-in timer) to offer a better experience to people who trust it with their searches.
I may even use DuckDuckGo again some day if The New Leaf Journal is ever back in its search results…