Bing chatbot formulating and testing novel hypotheses in real-time: How slime, chocolate, and Nobel prizes reveal the power and limits of artificial intelligence

Abstract

While the world has been amazed by large language models (LLMs) such as ChatGPT, there remains some debate about whether such generative AI chatbots are capable of synthesizing original ideas and thus serve as engines of discovery. Recently released LLMs that are linked to internet search, such as Microsoft Bing chatbot and Google’s Bard, have even greater potential for discovery since they have access to up-to-date information that extends beyond their original training data sets (in the case of ChatGPT, circa 2021). Thus, the goal of the exploration presented in this article was to test whether Microsoft Bing chatbot (powered by GPT-4), is capable of formulating a novel hypothesis, and then use internet search to collect data to address its hypothesis, and then draw conclusions from that data, i.e., the complete scientific method without human intervention. In three different realizations of this task, it does appear to be possible, although to varying degrees of impact and originality.Micro