AI Chatbot Search: Solving for latest data

Most chatbots today are trained with data which are months or years old. So, they still may not be best placed to answer questions around latest flights, movies and so on. Chatbots may still be lousy in many questions asked in Jeopardy or University Challenge.

While one may argue that these are only a subset of search engine queries or questions relevant in everyday order of things, it is interesting to see that current approach of some chat bots may not be able to address this while others may already have an answer.

In recent edition of ‘Batch’ newsletter by Andrew Ng, there is an interesting discussion on probable way forward:

Here’s one way to think about the problem. ChatGPT’s predecessor GPT-3 has 175 billion parameters. Using 16-bit, floating-point bytes, it would take around 350GB to store its parameters (many reports say 800GB). In comparison, Wikipedia occupies about 150GB (50GB for text, 100GB for images). While the comparison is far from apples to apples, the fact that an LLM has more memory than is needed to store Wikipedia suggests its potential to store knowledge.

But even Wikipedia contains a minuscule fraction of the knowledge available on the internet, which by some estimates amounts to 5 billion GB. Thus search, which can point us to pages from all corners of the web, can answer many questions that an LLM with fixed memory can't.Rather than relying on a fixed LLM to deliver the answer to a query, if we first find relevant documents (online or elsewhere) and then use an LLM to process the query and the documents into an answer, this could provide an alternative to current web search. Executing this efficiently and at scale would be complex, but the effect would be akin to having an LLM do a web search and summarize the results. Examples of this approach include Meta's Atlas and DeepMind's RETRO. 

Source: The Batch