Privacy-focused search engine DuckDuckGo has followed Microsoft and Google to become the latest veteran search player to dip its beak in the generative AI trend — announcing the launch today in beta of an AI-powered summarization feature, called DuckAssist, which can directly answer straightforward search queries for users.
DDG says it’s drawing on natural language technology from ChatGPT-maker OpenAI and Anthropic, an AI startup founded by ex-OpenAI employees, to power the natural language summarization capability, combined with its own active indexing of Wikipedia and other reference sites it’s using to source answers (the encyclopedia Britannia is another source it mentions).
Founder Gabe Weinberg tells TechCrunch the sources it’s using for DuckAssist are — currently — “99%+ Wikipedia”. But he notes the company is “experimenting with how incorporating other sources could work, and when to use them” — which suggests it may seek to adapt sourcing to the context of the query (so, for example, a topical news-related search query might be better responded to by DuckAssist sourcing information from trusted news media). So it remains to be seen how DDG will evolve the feature — and whether it might, for example, seek to ink partnerships with reference sites.
At launch, DuckAssist is only available via DDG’s apps and browser extensions — but the company says it plans to roll it out to all search users in the coming weeks. The beta feature is free to use and does not require the user to be logged in to access it. It’s only available in English for now.
Per Weinberg, the AI models DDG is (“currently”) using to power the natural language summarization are: The Davinci model from OpenAI and the Claude model from Anthropic. He also notes DDG is “experimenting” with the new Turbo model OpenAI recently announced.
It’s worth noting that DDG’s search engine does already have an Instant Answers feature which gets triggered for certain types of queries and also serves answers directly above the usual list of links. (Example scenarios include if you’re asking the search engine to sum basic calculations, display a calendar for the current month or asking for factual snippets of info.)
However DDG says that adding in generative AI summarization has enabled it to expand how many queries can be directly answered in this way — dubbing the addition of generative AI into the mix here a “fully integrated Instant Answer”.
Anthropic begins supplying its text-generating AI models to select startups
“The two main benefits compared to other instant answers are that DuckAssist answers will be more directly responsive to user questions, and that DuckAssist can answer significantly more questions,” Weinberg tells us. “The generative AI behind DuckAssist generates new text for a particular query, where standard instant answers are generally pulling out quotes. In this way, DuckAssist can be more directly responsive to the query, quickly surface information buried in articles, and synthesize information from multiple Wikipedia snippets. As a result, it can answer a wider breadth of questions.”
DuckAssist is intended to help users of the search engine find factual information more quickly — hence it only appears as an option when the technology assesses it can help with a specific query.
“If you search for a question in any DuckDuckGo app or browser extension, and DuckAssist thinks it might be able to find an answer from Wikipedia, you may see a magic wand icon and ‘Ask Me’ button at the top of your search results,” DDG explains.
If an answer has previously been solicited by another DDG user the company says it will be displayed automatically — but it also notes that users can opt to disable Instant Answers (which includes DuckAssist) in the settings if they prefer not to be exposed to AI-generated summaries.
Weinberg says the feature works by using AI to generate new natural language responses “based on specific/relevant sections of Wikipedia articles” that DDG provides via its own scanning of sources. (He specifies DDG is using its own indexing technology “to identify the relevant chunks of text from Wikipedia, and then ask the models to format answers in a way that is directly responsive to the query”.)
Accuracy is one key concern being attached to applications of generative AI — given the technology can be prone to make up information and yet automated output that’s presented in a natural language wrapper can sound highly authoritative despite not being fact-checked.
On this, Weinberg says DuckAssist has been designed to boost the probability that it will give a correct answer while also providing users with information that the answer is automated — and pointing them to the reference sources where they can do their own fact-check (i.e. if it turns out DuckAssist is being more of an ‘ass’ than an assistant).
“A search engine’s job is to surface reliable information quickly. We designed DuckAssist in way that leverages what natural language technology does well while trying to increase the probability it will give a correct answer when it appears in search results. We did that by intentionally limiting the sources DuckAssist is summarizing from,” he says. “For now, DuckAssist is only pulling answers from Wikipedia and a handful of related sources, like Britannica. This greatly limits the probability DuckAssist will generate incorrect information or ‘hallucinate’, where the AI tool makes up random information.
“Nevertheless, we know it won’t be right 100% of the time — If we do not provide the most relevant text [to the AI] to summarize, for example, or if Wikipedia itself has errors. In any case, we label every answer as not independently checked for accuracy and provide a link to the most relevant Wikipedia article for more info.”
On the privacy side, DDG promises this search with AI feature is anonymous — and, in line with its headline privacy pledge, further emphasizes that no data is shared with any of the third parties it’s working with to integrate the generative AI capability into its search engine. (In the blog post Weinberg also specifies that users’ anonymous searches are not being used to train its suppliers’ AI models.)
It’s asking users to give feedback on the quality of the DuckAssist summaries — via a feedback link that’s displayed next to all DuckAssist answers, as part of its approach to tackling the generative AI accuracy issue — and says this feedback is anonymous too, with user reports also only sent to DDG itself, not to any third parties.
While the launch of DuckAssist means there will, inevitably, be more automated answers popping up in response to users’ queries, DDG notes the feature will still only be available for a minority of searches — since it’s only intended to help with relatively straightforward asks. Phrasing a search query as a question makes it more likely the feature will appear in search results, it adds.
“Generative artificial intelligence is hitting the world of search and browsing in a big way,” Weinberg writes in a blog post announcing what he says is “the first in a series of generative AI-assisted features we hope to roll out in the coming months”. “At DuckDuckGo, we’ve been trying to understand the difference between what it could do well in the future and what it can do well right now. But no matter how we decide to use this new technology, we want it to add clear value to our private search and browsing experience.”
More AI powered search and browser features are also in the works from DDG, with additional AI-related news slated for the coming months. (Although he won’t be drawn on what else it has cooking — saying only “stay tuned!”.)
Here’s a clip of the DuckAssist feature in action for a search query that asks “is antarctica a country” — which shows the user being promoted to activate DuckAssist (“ask”) and, on doing that, they receive a summarized natural language answer, displayed above the source (Wikipedia) and a reference to the section of the article it was sourced from:
In its blog post, DDG explains it selected Wikipedia as the main source for DuckAssist since the crowdsourced encyclopedia is already the primary source for its existing Instant Answers feature and, while not foolproof, it assesses it as “relatively reliable across a wide variety of subjects”.
It also points out Wikipedia has the added benefit of being a public resource “with a transparent editorial process that cites all the sources used in an article, you can easily trace exactly where its information is coming from”.
Plus Wikipedia is of course constantly being updated — expanding the queries DuckAssist can meaningfully respond to. That said, there is still a lag in the knowledge graph — as DDG notes that “right now” the DuckAssist Wikipedia index can be up to “a few weeks old”. (But it says it has plans to “make it even more recent”, in addition to adding more sources “soon”.)
It’s worth noting that DDG’s current gen Instant Answers aren’t always right, either.
At the time of writing, a DDG search for “people in space” generated a neat stack of ten cards of astronauts it suggested are currently up in orbit — however it displayed a photo of US astronaut Kayla Barron twice; once on her own card and once (incorrectly) paired with Germany astronaut Matthias Maurer’s card. So error-prone techie shortcuts are nothing new.
Still, the power of generative AI to automate far more interactions — and, in this case, respond to many more types of search queries — could cause a bigger skew in the information landscape by substantially expanding the ability of platforms to apply such shortcuts which boosts the probability of their users running into tech-generated errors.
Google takes on ChatGPT with Bard and shows off AI in search
Comment