Tim Dettmers, a prominent researcher from the Allen Institute and an assistant professor at Carnegie Mellon University, has recently expressed strong skepticism about the feasibility of achieving artificial general intelligence (AGI), a type of AI that could perform any intellectual task that a human being can, including tasks with economic value. In a detailed blog post, Dettmers argues that the pursuit of AGI not only harbors unrealistic expectations but is also grounded on fundamentally flawed premises that overlook the physical and economic limitations of current technologies.
Dettmers points out that conversations about AGI are often rooted in philosophical paradigms without enough consideration for the practical necessities—namely, the physical hardware on which such intelligence would need to operate. Despite the common perception that processors like GPUs are continually improving in capability, Dettmers predicts that these advancements are plateauing. He believes that in as little as one to two years, we may hit a physical limit to how much more these processors can be scaled up due to infrastructural bottlenecks that fail to keep pace with the growing resource demands needed for minor improvements in AI models.
He notes that while GPUs have seen some gains in performance over the last few years through innovations such as lower precision data types and specialized tensor cores, the actual generational increases in computational power have not been as substantial as some manufacturers claim. For instance, while Nvidia’s transition from the Ampere to the Hopper generations saw a tripling in BF16 performance, it required significant increases in power consumption and silicon real estate. Similarly, advancements from Hopper to Nvidia’s newer Blackwell parts showed considerable performance boosts but at the cost of doubled die area and increased power requirements.
Furthermore, Dettmers discusses how even attempts to optimize the integration of multiple GPUs within larger systems only produce temporary relief from the looming performance wall. He cites Nvidia’s GB200 NVL72 as an example, which significantly enhanced performance by intensifying the number of accelerators per compute domain. However, he argues that such rack-level hardware optimizations are a stopgap measure that might only extend viable improvements until around 2026 or 2027.
Despite his criticisms of the current trajectory towards AGI, Dettmers does not view the heavy investments into AI infrastructure as wholly misplaced. He acknowledges the substantial growth in inference applications justifies such financial outlays. Yet, he cautions that if the pace of model enhancements does not keep up, this infrastructure could quickly become an expensive liability.
Dettmers criticizes the dominant mindset within U.S. AI labs, which overly fixates on being the first to develop AGI under the assumption that it would secure a decisive advantage in what is often viewed as an AI arms race. He argues that this focus is not only shortsighted but diverts attention and resources from more immediate, economically productive applications of AI technology. This perspective is contrasted with China’s approach, which he regards as more pragmatic and focused on exploiting the current capabilities of AI to enhance productivity and utility in practical applications.
He also highlights a significant challenge for AGI—transcending the digital and venturing into the physical world through robotics. Here, he notes similar scalability challenges and points out the prohibitive costs and complexity of acquiring and processing physical world data.
Dettmers ultimately describes the pursuit of AGI as a pursuit of a fantasy, which, while compelling as a narrative, offers little in terms of practical economic benefits. He urges a reorientation towards leveraging AI for its current strengths rather than chasing an elusive and potentially unattainable ideal. This pragmatic perspective underscores a broader debate within the field concerning the directions and priorities of AI development.
Read the full post on theregister.com
