Listen to
Podcast

The Shrinking Addressable Market of Purpose-Built AI Chips

By Karu Sankaralingam

Caveat: This analysis is based on observed market trends, and uses a mathematical model to capture some quantitative insights. The goal is to get some insights from the model. To quote my fellow-Badger, George Box, "all models are wrong, but some are useful"

It seems clear that AI algorithms are evolving and changing reasonably dramatically almost every successive year. Resnet-50 surpassing and matching the accuracy of human image classification was a few years ago. Since then we have seen an explosion of uses of things like LSTMs and RNNs powering conversational AI, Transformer and Attention powering natural language processing, GANs for content creation, and graph neural networks for various forms of anomaly detection and processing of relationships. Graph neural networks seem novel and exotic, but are in production deployment for spam detection at Xainyu, for example, for detecting spam reviews. Individual businesses seem to be rapidly adopting these techniques in a multiplicative fashion — that is, their workflows are adopting many of these techniques and not just one of them. There seems to be roughly one new usage scenario every year.

This adoption can be modeled using two variables:

  1. Alpha: percentage of new AI business in any given year that is based on a single technique.
  2. Beta: percentage of workflows from the previous year that remains single-purpose.

Consider a fixed function AI chip. An aggressive scenario is a chip is conceived the year an algorithm advance is made and comes into production deployment two years after that AI use case has been in the field (production lag). Even this aggressive timeline is too long. It implies that by the time the chip is available, the many workflows would already be multi-use case. Exacerbating the issue, even before the chip’s deployment, some of the AI market will become unaddressable by the chip, because of the transition of some workflows into multi-use case, and because some percentage of the new market created will be on a new algorithm un-addressable by this chip that is merely one year old! When considering this effect over a period of five years, it becomes clear that a purpose-built AI chip’s market is shrinking even as the overall AI market grows exponentially.

It is tempting to think that some of the hyperscalars (Amazon AWS, Google GCP, etc.) have a good predictive understanding of their use cases and have a long window into the future. However, there seems to be an arms race underway between their respective AI groups, startups, and academic research, which means that the rate of change is hard to predict as is — and what kinds of algorithms will be deployed even in two years or three years. Further, the AI use case is driven by the inward need of the hyperscalars and their outward customer need. The Xianyu example is an outward need, while Amazon doing something with Alexa is an inward need, where ostensibly the hyperscalar can predict usage many years in advance. We believe the prior trends, and just sheer practicality indicate that use cases will evolve rapidly. It is therefore going to be impossible to predict what is going to stick and attempt to design an AI-chip for a specific algorithm. Some recent advances in GPT-3 point to this. With much fanfare, GPT-3 was announced as a phenomenal break-through by Open-AI, and then made exclusively accessible to Microsoft. In a few short weeks, a model that was 99.99% smaller was developed by an academic group and it outperformed GPT-3 on one task. This portends a trend that will continue and reinforces the model-diversity argument.

Below we develop a mathematical model to understand the business implications. The key findings from the model are:

  • The market for single-purpose AI chip could be as small as only 3% of the total AI chip market in another 4 years.
  • Chips that are muli-purpose and algorithm-adaptive will be required for more than 60% of the AI chip market in 4 years

Details of the model follow.

Assuming the aforementioned production lag can be dealt with, let us assume a somewhat complex deployment scenario where a system comprises multiple fixed function chips (either integrated as an IP block or as multiple different chips on a card/board). We map out below the addressable market of a single-chip solution and a multi-chip solution for different ratios of alpha and beta.

Definitions:

  • Single purpose market means there is a business use case that is using only one AI algorithm. We are further assuming that the single-purpose solutions are equally distributed between all available solutions in any given year.
  • Single purpose market per AI solution means the total business use cases all cumulatively are using only one AI algorithm.
  • Multi-purpose market means a business use case that is using all the AI algorithms including the one that appeared in this year. Either because they have different applications that use different ones, or have one application that uses multiple.
  • Single custom-chip market means the subset of the total market that can be addressed by ONE company that has a custom chip targeted at one of the AI solutions with custom-silicon for it. While multiple companies can each have their own custom chip going after different applications, we are defining a market that ONE company can capture, with the underlying assumption that ONE company can have only ONE custom chip.
  • Multiple custom-chip market means the subset of the total market that can be addressed by ONE company that has a solution with multiple custom-chips that are each targeting different solutions.

Mathematical terms

Total market in year i: Mi

Ratio of new market that is single-purpose: α

Ratio of old market that remains single-purpose: β

Number of AI solutions in year i: ni

Number of AI solutions in year i for which chips exist: ci

Size of market total single purpose in year i : MTSPi

Size of market single purpose per AI solution in year i : MSPSi

Size of market multi purpose in year i : MMPi

Size of market single chip in year i : MSCi

Size of market multi chip in year i : MMCi


MTSPi = ( Mi- Mi-1)* α + MTSPi-1* β

MSPSi = MTSPi / ni

MMPi = Mi-MTSPi

MSCi = MSPSi

MMCi = MSCi* ci

Assumptions

  1. It takes two years from specification of a chip based on an algorithm to its product deployment.
  2. AI algorithm is defined as “large” changes to primitives of what is done requiring a new chip hardware.
  3. Regardless of single-chip or multi-chip, we are assuming there is always opportunity to sell into the market (optimistic deployment assumption).
  4. We are assuming that the distribution of single-purpose solutions will remain equal between new techniques and old techniques, regardless of how old the techniques are.
  5. 50% yearly growth in the AI market.
  6. In the table below we consider an ad-hoc ratio of what constitutes the single-purpose market every successive year.

Scenario: Aggressive use of multiple solutions

MARKET YEARLY GROWTH 1.5
Total market (USD) $10.0 B $15.0 B $22.5 B $33.8 B $50.6 B $75.9 B
Ratio of new market that is single-purpose 1.0 0.7 0.5 0.4 0.3 0.2
Ratio of old markets that remains single-purpose 1.0 0.8 0.8 0.6 0.5 0.4
Number of AI solutions/techniques 1.0 2.0 3.0 4.0 5.0 6.0
Availability of custom-solution 1.0 1.0 1.0 2.0 3.0 4.0
Total Single purpose market $10.0 B $11.5 B $13.0 B $12.3 B $11.2 B $9.5 B
Single purpose market per AI solution $10.0 B $5.8 B $4.3 B $3.1 B $2.2 B $1.6 B
Multi-purpose market 0 $3.5 B $9.6 B $21.5 B $39.4 B $66.4 B
Single custom-chip market $10.0 B $5.8 B $4.3 B $3.1 B $2.2 B $1.6 B
Multiple custom-chip market $10.0 B $5.8 B $4.3 B $6.1 B $6.7 B $4.8 B

4 Key takeaways

  1. It will be extremely difficult to justify the costs of building single purpose chips in terms of the costs and risks relative to the market opportunity.
  2. At the very least, anyone pursuing the market opportunity with single-purpose chips would be wise to hedge their investments and diversify their risk with multi-purpose solutions.
  3. Given the aggressive assumptions used in the model, achieving even the 3% number for single-purpose chips will be challenging.
  4. It seems quite obvious that the best risk-adjusted approach to capture the market will be to utilize algorithm-adaptive chips.