Myths and Realities of AI's Rapid Evolution - 2023
Since the 1950s, the journey of artificial intelligence (AI) has mirrored the tale of the boy who cried wolf, with grand claims that ultimately fell short of expectations. Visionaries like Herbert Simon and Marvin Minsky set ambitious goals for achieving Artificial General Intelligence (AGI) by the year 2000. Fast forward to 2021, AI offerings were primarily confined to niche areas (like Siri and Alexa), faced drawn-out developments (such as autonomous vehicles and robotics), or were specialized for specific tasks (including recommender systems, facial recognition, language translation, and customized advertisements).
For many years, AI’s potential appeared to be monopolized by major tech firms and academic circles. Meanwhile, the majority of the population turned their attention and resources towards more immediately lucrative ventures, such as benefitting from the wealth generated by tech giants, investing in the surge of cryptocurrencies, founding Software as a Service (SaaS) startups with high market valuations, or managing growing venture capital funds.
However, the introduction of ChatGPT half a year ago took even industry insiders by surprise with its swift expansion. Despite the tech sector’s growth, the advancement of AI has been driven by a relatively small segment of the industry compared to previous technological revolutions like the internet or software development. The development and deployment of technologies like ChatGPT, Claude, and Bard stem from the transformer architecture introduced in 2017, yet, only a handful of individuals have actively worked on scaling these innovations. Together, OpenAI and Anthropic have less than 800 employees.
This rapid shift, facilitated by a few, has sparked concerns among many tech professionals about being left behind in this new era. Such anxieties give rise to various illusions of hope (believing that the large language model market will evolve in a favorable manner), cope (downplaying one’s market position), and despair (feeling as though the ‘game’ has already been lost).
Currently, these illusions fuel exaggerated stories about foundational model fears, doubts and uncertainties (FUD), the superior performance of open-source models, the invulnerability of established players, infatuation among investors, and pessimistic proclamations. Navigating these narratives demands a healthy dose of skepticism, and even this examination should be approached critically (note, FF has invested in Scale, a key model provider).
Let’s debunk some of the popular yet misleading narratives that are often presented with unwavering confidence despite the uncertain trajectory of the AI industry.
The Myth of Foundational Models Hitting Their Peak
For those not involved in the recent breakthrough in developing cutting-edge large language models (LLMs), there’s a temptation to believe these models represent the pinnacle of what’s achievable.
This has led to many companies initiating substantial funding rounds to develop “superior” foundational models, pitching themselves as necessary alternatives to organizations like OpenAI and Anthropic, but promising lower costs, greater openness, minimal filtering, enhanced architecture, better localization, or improved specificity for certain domains.
Entering the language model sector in 2023 entails reconciling with one’s non-dominant market stance. Critical self-reflection might include accepting current models as a historical endpoint, imagining a future with multiple successful models, or dismissing the idea of an unassailable market advantage.
A common sentiment is skepticism towards significant improvements from GPT-5 over GPT-4, suggesting a saturation point in LLM capabilities. Such views, however, may indicate a reluctance to accept the continuous potential for innovation. Assuming GPT-4 marks the upper limit of current LLM development ignores the possibility of numerous equally successful LLMs and minimal disruption to existing business models.
Yet, even a conservative outlook fails to dim the prospect of more advanced LLMs within the next half-decade. Suppose for a moment that critics are correct, and GPT-5 represents only a modest advancement. Even so, GPT-4 embodies untapped potential. By training on exclusive data, lowering operational costs, and enhancing user experience, we can catalyze significant advancements in model capabilities, unlocking vast economic benefits.
Adoption of new innovations is often gradual. It took the older generation a decade to regularly use Google, and I’m still mastering GPT-4. However, with further advancements in model capabilities and the development of software utilizing current LLM technology, 2023 will not be remembered as a standstill moment.
The Debate on Open Source Models’ Dominance
Recently, open-source initiatives have gained momentum, from adaptations of Whisper to LLaMA developments. There are various stakeholders hopeful for the success of open-source models, including:
- Developers wary of depending on large corporations
- Venture capitalists (VCs) and AI infrastructure startups benefiting from a diversified model landscape
- Individuals feeling sidelined in the 2023 narrative of AI
- Major tech firms lacking a platform to monopolize infrastructure spending
- Advocates for an uncontrollable product for benign or harmful purposes
- Advocates for products free from political bias
Whether this widespread interest translates into open-source models triumphing remains to be seen. Public discourse on open source software’s merits is nuanced, given its protected status and the unseemliness of disparaging comments.
Nevertheless, there’s an underlying belief that open-source models might not become the primary choice for state-of-the-art, commercial models. Clayton Christensen once expressed concern over the iPhone’s closed system, fearing that the inevitable shift towards open modular architecture, illustrated by the personal computer’s evolution and Android’s rapid growth, would diminish Apple’s market hold.
The current discussion around open-source AI models mirrors the initial Apple vs. Android debate, where modular, open-source frameworks unequivocally democratize technology access, as seen in emerging markets predominantly using Android. However, cohesive products often secure the most value. Despite holding only a 21% share in smartphone shipments, the iPhone captures 50% of the revenue and 82% of profits.
Examining the list of operating systems reveals a plethora of options, though few have had significant impact. The dominance of operating systems followed an even more pronounced power-law distribution than smartphones did. Within the realm of open-source systems, Linux emerged as the singular power-law champion. Nearly every tech market follows this pattern: smartphones, social networks, cloud infrastructure, and even SaaS.
Regulatory actions could potentially restrict open-source models for safety reasons, but market dynamics also place open-source in a challenging position for most applications. While open-source models introduce necessary competition, keeping the price and neutrality of closed models in check, the case for open-source would be stronger if large foundational model enterprises were engaging in egregious pricing or censoring practices.
Open-source makes sense in specific contexts:
- For large organizations, open-source models could be crucial: they allow companies to maintain control and privacy. In the data warehouse sector, some choose Databricks over Snowflake for its data interoperability, versus Snowflake’s more restrictive architecture.
- For applications where immediate response time trumps model quality (such as real-time speech-to-text), local deployment of open-source models can offer speed at the expense of accuracy and scope.
Yet, when it comes to consumer-facing LLM applications and SaaS, surpassing the quality of closed-source, meticulously maintained, integrated solutions is challenging. Based on rigorous evaluations, leading-edge models still outperform open-source alternatives. As expectations for LLM reliability, versatility, and robustness continue to climb, open-source models are unlikely to become the predominant solution for applications, capturing an even smaller share of the overall value.
The Fallacy of Incumbent Supremacy in AI
The notion that only existing market leaders will triumph in AI fosters a defeatist mindset, assuming the race is already decided. Recent developments have indeed tilted in favor of incumbents, as seen with Microsoft’s GitHub Copilot, OpenAI’s buzz, NVIDIA’s surge due to increased GPU demand, and Google’s cloud platform benefiting from Midjourney.
While the shift to cloud computing posed significant challenges for 20th-century software firms, allowing newcomers like Salesforce and Atlassian to thrive, integrating LLMs seems remarkably straightforward for incumbents. Many have quickly leveraged obvious opportunities, with Notion, Epic Health, Intercom, and Zendesk incorporating AI into their services.
The adaptability of LLM APIs and incumbents’ extensive reach have resulted in immediate successes. In the short term, AI appears more as a progressive innovation rather than a disruptive force. However, this view may lack creative foresight at the product concept stage, where it’s simpler to adapt existing solutions than to envision entirely new opportunities.
Even with GPT-4 and upcoming models’ capabilities, completely novel product categories could emerge, diverging from the traditional formula of adding AI to incumbent workflows. These new ventures will demand distinct user experiences to attract customers. This is a golden opportunity for startups to position themselves against incumbents by designing radically different product experiences.
The burst of high-quality LLMs has already spurred promising niches, including enterprise search (Glean), legal automation (Harvey, Casetext), and marketing automation (Typeface, Jasper). These applications have the potential to flourish. Pioneering new categories takes time—during the productivity boom of the 2010s, platforms like Figma and Notion required years to develop market-ready products.
Claiming that only incumbents will emerge victorious is an oversimplified perspective, justifying inaction with the belief that all opportunities are already seized. The idea of a major technological shift without room for new entrants is implausible.
The Reality of VC-Backed AI Ventures
Conversely, some view the AI landscape with undue optimism, particularly venture capitalists (VCs) dreaming of lucrative start-up investments. Historically, VC involvement has been closely associated with technological breakthroughs. Rapid advancements typically spell success for VCs.
However, VCs hold little stake in leading AI companies today. Few VCs have investments in giants like NVIDIA or Microsoft. Even among scalable startups, Midjourney progressed without VC backing, and OpenAI has only a minimal portion of its funding from this sector.
Venture capitalists, with their focus on early-stage investments, strive to craft narratives around opportunities for budding startups in new foundation models, vector databases, and LLM-driven applications. While some of these stories may hold merit, there is a natural bias among VCs to envision favorable market dynamics that promise AI returns.
Quickly assembled LLM infrastructure landscapes by various firms insert AI into a simplistic SaaS model framework of the 2010s, suggesting a proliferation of niche solutions comprising an “AI stack,” with each capturing a slice of value.
This approximation appears fragile. Much of today’s infrastructure is designed to complement LLM shortcomings: additional modalities, enhanced monitoring, linked commands, expanded memory. However, the API-based construction of foundation models abstracts away the complexities of infrastructure from developers. Incremental improvements to models gradually render the surrounding infrastructure ecosystem obsolete. If a few dominant LLMs prevail, a robust infrastructure ecosystem becomes less critical.
On the application level, VCs may find more fertile ground, with new entrants requiring venture capital to launch. LLMs will unveil new software categories across various industries, from legal to healthcare to service sectors. Nevertheless, since software incumbents can also leverage LLM APIs, the application-level prospects for startups will be confined to innovative product concepts.
If the number of VC-investable openings is limited, VCs will play a minimal role in the significant platform transformation of recent decades. At the infrastructure level, VCs fervently wish for a fragmented AI stack narrative to hold true, serving both as a coping strategy and a demonstration of their grasp on the industry to limited partners and founders.
Certainly, some VC-supported businesses will grow into substantial, independent entities. However, they should be evaluated cautiously given the underlying motives. The pace at which VCs deploy capital may outstrip the realization of successful venture-backed endeavors.
The Call to Decelerate AI Advancements
Numerous individuals have reasons to advocate for slowing down LLM development: competing foundation model creators not leading the pack, the Chinese Communist Party (CCP), employees facing potential job losses, and those fearing AI’s implications.
Underdog LLM developers: The “Pause Giant AI Experiments” open letter appeared to be thinly disguised competitive FUD, motivated by self-interest. Many signatories were AI researchers trailing behind in advancements, seeking time to catch up.
The CCP: China’s emphasis on the quantity of LLM releases, as opposed to their quality or prevalence, is a strong indicator of them lagging behind the US. Like the letter’s signatories, China would benefit from a slowdown in US LLM advancements, helping them close the competitive gap.
Employees at risk of displacement: Individuals in positions potentially replaceable by AI, such as copywriters or call center staff, have a professional interest in delaying AI’s progress.
Highlighting GPT-4’s failures in tasks involving mathematics and logic has gained popularity. There’s a sense of reassurance in acknowledging that AI has not yet surpassed human capabilities in every area.
AI pessimists: While the potential dangers of AI should be taken seriously, the argument for an imminent AI catastrophe is unfalsifiable — a doomed forecast cannot be disproven easily. The underlying value systems of these commentators also introduce biases, warranting a critical analysis remaining an exercise for the reader.
The argument for slowing down should be approached with caution, especially in the absence of concrete evidence suggesting an imminent AI escalation. Premature alarm could undermine the credibility of AI safety discussions, with premature doomsday predictions potentially desensitizing people to genuine risks. Meanwhile, the potential benefits of AI for humanity are too significant to postpone.
In Closing
The initial years of the internet in the mid-1990s and the dawn of cryptocurrencies in the early 2010s saw a select few participants, most of whom reaped financial rewards.
In contrast, the launch of ChatGPT has drawn a vast audience keen to engage in the AI arena, including researchers, developers, VCs, and startups. However, unlike previous technological shifts, the scenario with LLMs suggests that relatively few newcomers may succeed in this deployment phase. The immediate and evident value proposition of LLMs has spurred rapid demand, unlike the delayed acceptance of the internet or crypto. The leading players in the AI sector are advancing at an unprecedented pace.
Given the disproportionate influence of a few individuals defining AI’s frontier, the broader tech community is left grappling with biases and misinformation. The prevailing discourse is often a blend of despair (mope), denial (cope), and opportunism (hope).
While this analysis has not covered every prevailing AI narrative, caution should be exercised whenever statements are presented with absolute certainty. Given the early stage of this technological cycle, the truth remains elusive and requires proactive engagement through dialogue and firsthand experiences. Trust must be placed in direct observation and involvement.