{"id":13612,"date":"2025-02-24T10:00:00","date_gmt":"2025-02-24T10:00:00","guid":{"rendered":"https:\/\/modernsciences.org\/staging\/4414\/?p=13612"},"modified":"2025-02-13T17:56:39","modified_gmt":"2025-02-13T17:56:39","slug":"openai-deep-research-agent-ai-tool-limitations-february-2025","status":"publish","type":"post","link":"https:\/\/modernsciences.org\/staging\/4414\/openai-deep-research-agent-ai-tool-limitations-february-2025\/","title":{"rendered":"OpenAI\u2019s new \u2018deep research\u2019 agent is still just a fallible tool \u2013 not a\u00a0human-level\u00a0expert"},"content":{"rendered":"\n
\n
\n \n
\n \n Jan Antonin Kolar\/Unsplash<\/a><\/span>\n <\/figcaption>\n <\/figure>\n\n Raffaele F Ciriello<\/a>, University of Sydney<\/a><\/em><\/span>\n\n

OpenAI\u2019s \u201cdeep research<\/a>\u201d is the latest artificial intelligence (AI) tool making waves<\/a> and promising to do in minutes what would take hours for a human expert to complete.<\/p>\n\n

Bundled as a feature in ChatGPT Pro and marketed<\/a> as a research assistant that can match a trained analyst, it autonomously searches the web, compiles sources and delivers structured reports. It even scored<\/a> 26.6% on Humanity\u2019s Last Exam (HLE), a tough AI benchmark, outperforming<\/a> many models.<\/p>\n\n

But deep research doesn\u2019t quite live up to the hype. While it produces polished reports, it also has serious flaws. According to journalists<\/a> who\u2019ve tried it<\/a>, deep research can miss key details, struggle with recent information and sometimes invents facts.<\/p>\n\n

OpenAI flags this when listing the limitations of its tool. The company also says it<\/a> \u201ccan sometimes hallucinate facts in responses or make incorrect inferences, though at a notably lower rate than existing ChatGPT models, according to internal evaluations\u201d. <\/p>\n\n

It\u2019s no surprise that unreliable data can slip in, since AI models don\u2019t \u201cknow\u201d things in the same way humans do. <\/p>\n\n

The idea of an AI \u201cresearch analyst\u201d also raises a slew of questions. Can a machine \u2013 no matter how powerful \u2013 truly replace a trained expert? What would be the implications for knowledge work? And is AI really helping us think better, or just making it easier to stop thinking altogether?<\/p>\n\n

What is \u2018deep research\u2019 and who is it for?<\/h2>\n\n

Marketed towards professionals in finance, science, policy, law and engineering, as well as academics, journalists and business strategists, deep research is the latest \u201cagentic experience<\/a>\u201d OpenAI has rolled out in ChatGPT. It promises to do the heavy lifting of research in minutes.<\/p>\n\n

Currently, deep research is only available to ChatGPT Pro users in the United States, at a cost of US$200 per month. OpenAI says<\/a> it will roll out to Plus, Team and Enterprise users in the coming months, with a more cost-effective version planned for the future.<\/p>\n\n

<\/p>\n\n

Unlike a standard chatbot that provides quick responses, deep research follows a multi-step process to produce a structured report:<\/p>\n\n

    \n
  1. The user submits a request. This could be anything from a market analysis to a legal case summary.<\/li>\n
  2. The AI clarifies the task. It may ask follow-up questions to refine the research scope.<\/li>\n
  3. The agent searches the web. It autonomously browses hundreds of sources, including news articles, research papers and online databases.<\/li>\n
  4. It synthesises its findings. The AI extracts key points, organises them into a structured report and cites its sources.<\/li>\n
  5. The final report is delivered. Within five to 30 minutes, the user receives a multi-page document \u2013 potentially even a PhD-level thesis<\/a> \u2013 summarising the findings.<\/li>\n<\/ol>\n\n

    At first glance, it sounds like a dream tool for knowledge workers. A closer look reveals significant limitations.<\/p>\n\n

    Many<\/a> early<\/a> tests<\/a> have exposed shortcomings:<\/p>\n\n