#3 - What drives the moat in AGI? Input teardown
Summary:
Capital unlikely to be the barrier
Research labor & data will contribute to moat
Given the prize at stake (and market cap at risk) I suspect $10B will begin showing up from multiple players. I have two questions:
#1 - Who can bring that kind of money to bear / how painful is it for them to do so?
#2 -What are the other inputs and what is their relative scarcity (and value toward moat)
See first post re: the overall Map. The main inputs are Capital, Compute, Labor, and Data.
Second post here on MSFT
#1 - Financing
$10B isn’t a lot, given the prize.
table footnotes1
The premise you have to agree with is that the upside of AGI is there and monetizable over a 10 year period. This is a topic I will cover in a future post. From a CAPM2 there is higher beta vs. other platform investments. That said, break down the drivers of beta. You have market risk, technology risk, and execution risk. I suspect the technology and market risk are much lower than Meta’s ~$200B3 investment in Metaverse.
I’m applying a liberal use of by-and-large here. Exceptions and outlier drivers exist but aren’t a great starting point for building a mental map.
Conclusion: Low Moat Driver. Multiple players can and likely will bring significant resources to bear.
#2 - Compute
Outputs: Higher volume of existing chipsets and more SKUs4
Inputs: Design, Fab capacity
In the short term, I assume OpenAI / Alphabet’s DeepMind have an advantage. Alphabet via its own TPUs and OpenAI via Microsoft Azure7.
In the short term, NVIDIA is the clear winner. Given the prize, I suspect more will compete.
In the medium term I suspect that advantage declines. This is a topic worthy of a follow on post. The core assumption I have here is twofold. #1 - rational actors will increase the number of chips capable of handling large models (design). #2 - fabs (production) that will increase supply. I am aware that many limitations currently exist with #1 and #2.
The absence of different vendors does not in itself signal a trend given that in past rounds of MLPerf, individual vendors have decided to skip the competition only to return in a subsequent round. - ZDNet
Conclusion:
In the short term moderate moat.
In the medium term less so.
Google appears to have a face value advantage due to vertical integration.
What could make this null?#1 - Azure may have a head start if it secured a large part of NVIDIA’s output to be captive (for OpenAI or other)
#2 - GPUs and TPUs are not 1:1 (a topic for a later deep dive)
#3 - The bottleneck is at the Fab and the competition for run time is zero sum between all the players
#3 - Research & Engineering Labor
This is harder to break down but here is how I think about it. This framework is not perfect nor is it intended to be. Its purpose is to create probabilistic understanding.
Output: Research & Engineering talent
Input: Labor incentives:
Controllable: Financial, impact (positive impact + negative impact mitigation), enjoyment of environment, training
Non Controllable: raw ability
note: reinforcement learning used as a proxy
People will get caught up on the number and miss the conclusion. Whether you believe there are 100 valuable engineers or 5k is irrelevant. The point is that is a very small resource surface area for $50b to be funneling through.
My main model analog here is Lockheed Skunk Works8. MSFT was right to keep this out of the mothership and allow OpenAI to run free. It will be interesting to see if Google keeps its mothership at bay and how fast it can move. There is no market share awarded for having a lab doing R&D but not shipping.
Conclusion: Large moat driver in the medium term
#4 - Data
Outputs: Large Language Model Data set
Inputs: Public, Semi-private, and private data sets
OpenAI as a case study:
OpenAI GPT 3 started with public data sets. These were refined from 45TB down to 570GB9.
In summary
Conclusion: Feedback loops likely to be a valuable driver of moat (as it has been for Google)
Google has incentives I will cover. Regardless, despite risk to its bread and butter, game theory would state if it believes its economic model’s demise is inevitable, it would rather own the future outcome than cede it.
Huge Thanks to Xander Dunn for reviewing this
$10B used as a proxy given the press releases from various companies in the space
https://www.investopedia.com/terms/c/capm.asp
This assumes the $30B annual spend continues
https://www.investopedia.com/terms/s/stock-keeping-unit-sku.asp
https://resources.nvidia.com/en-us-tensor-core
https://cloud.google.com/tpu/docs/system-architecture-tpu-vm
https://www.theinformation.com/articles/why-openai-spent-barely-a-dime-on-microsofts-cloud-after-1-billion-deal?rc=5zfb38
US was behind Nazi Germany on Jet engine propulsion and US incumbents had missed the boat
https://arxiv.org/pdf/2005.14165.pdf