While Nvidia leads the AI chip market, competition is intensifying.

• Nvidia's AI accelerators hold between 70% and 95% of the market share for artificial intelligence chips.
• The level of competition has intensified with the increase in startups, cloud companies, and other chipmakers working on development.
• In May, Nvidia's market cap reached $2.7 trillion after its shares increased by 27%.

The chipmaker's market cap reached $2.7 trillion in May, making it one of the most valuable public companies in the world, after only two other companies. The company's year-over-year sales tripled in the third straight quarter, fueled by the growing demand for its artificial intelligence processors.

Nvidia has a dominant market position in the AI chip industry, with an estimated market share of between 70% and 95%, according to Mizuho Securities. This is reflected in Nvidia's impressive 78% gross margin, which is significantly higher than the average for hardware companies that must manufacture and ship physical products.

In the latest quarter, chipmakers reported gross margins of 41% and 47%, respectively.

Some experts have characterized Nvidia's dominance in the AI chip market as a moat, with its advanced AI graphics processing units (GPUs) and CUDA software providing a significant advantage over competitors.

Jensen Huang, Nvidia CEO, whose net worth has increased from $3 billion to approximately $90 billion in the past five years, has expressed concern about his 31-year-old company losing its competitive edge. He acknowledged at a conference in late 2020 that there are numerous powerful competitors emerging.

"Huang stated in November that he believed people were not attempting to drive him out of business. However, he acknowledged that he likely knew they were trying to do so, which changed the situation."

Nvidia has pledged to release a new AI chip architecture annually, instead of every other year, and to develop new software to further integrate its chips into AI applications.

Nvidia's GPU is not the only one capable of running the complex math required for generative AI. If less powerful chips can perform the same task, Huang may have reason to be paranoid.

Inference, or deploying AI models, presents an opportunity for companies to replace Nvidia's GPUs, especially if cheaper alternatives are available. Nvidia's flagship chip costs around $30,000 or more, motivating customers to explore other options.

"D-Matrix co-founder Sid Sheth stated that while Nvidia would like to have 100% of it, customers would not want Nvidia to have complete control over it. He explained that it is too big of an opportunity and would be unhealthy for any one company to dominate the market."

D-Matrix, founded in 2019, aims to release a semiconductor card for servers this year that will decrease the cost and latency of running AI models. The company raised $110 million in September.

The AI chip market is expected to reach $400 billion in annual sales in the next five years, and companies ranging from multinational corporations to nascent startups are competing for a share of this lucrative market. Nvidia has already generated about $80 billion in revenue over the past four quarters, and it is estimated that the company sold $34.5 billion in AI chips last year.

Nvidia's GPUs are being utilized by many companies with the hope that a different architecture or specific trade-offs could result in a superior chip for specific tasks. Additionally, device manufacturers are developing technology that may eventually perform a significant amount of computing for AI currently taking place in large GPU-based clusters in the cloud.

"Fernando Vidal, co-founder of 3Fourteen Research, stated on CNBC that Nvidia is the go-to hardware for training and running AI models. However, there has been incremental progress in leveling the playing field, with hyperscalers developing their own chips and startups designing their own silicon."

Lisa Su, CEO of AMD, aims to convey to investors that there is ample opportunity for numerous successful businesses in the industry.

"Su stated in December, during her company's launch of its latest AI chip, that "the secret lies in the abundance of choices available." She predicted that there would be no single solution, but rather multiple options."

Other big chipmakers

AMD produces GPUs for gaming and is modifying them for AI use in data centers, just like Nvidia. Its top-of-the-line chip is the Instinct MI300X. Microsoft has already purchased AMD processors and is providing access to them through its Azure cloud.

At launch, Su emphasized the chip's strengths in inference rather than competing with Nvidia for training. This week, Microsoft announced it was using AMD Instinct GPUs to serve its Copilot models. Morgan Stanley analysts viewed this news as a sign that AMD's AI chip sales could exceed $4 billion this year, the company's public target.

Intel, which was surpassed by Nvidia in revenue last year, is also attempting to establish a presence in AI. The company recently unveiled the third version of its AI accelerator, Gaudi 3, and compared it directly to the competition. Intel described Gaudi 3 as a more cost-effective alternative and better than Nvidia's H100 in terms of running inference, while also being faster at training models.

Recently, Bank of America analysts predicted that Intel will have less than 1% of the AI chip market this year, despite having a $2 billion order of backlogs for the chip.

The primary obstacle to broader acceptance may be software. AMD and Intel are both members of the UXL foundation, which aims to develop open-source alternatives to Nvidia's CUDA for controlling hardware in AI applications.

Nvidia's top customers

Nvidia faces a challenge in competing against its major customers, who are also its biggest competitors, as they develop processors for their own internal use. These Big Tech companies, including Google, Microsoft, and Amazon, account for over 40% of Nvidia's revenue.

In 2018, Amazon launched its own AI-focused chips under the Inferentia brand name. The second version of Inferentia is now available. In 2021, Amazon Web Services (AWS) introduced Tranium, a chip designed for training. While customers cannot purchase the chips, they can rent systems through AWS, which claims that Tranium is more cost-efficient than Nvidia's.

Since 2015, Google has been using Tensor Processing Units (TPUs) to train and deploy AI models. In May, the company announced the sixth version of its chip, Trillium, which was used to develop its models, including Gemini and Imagen.

Google also uses Nvidia chips and offers them through its cloud.

Microsoft isn't as advanced as it claimed last year when it announced it was developing its own AI accelerator and processor, named Maia and Cobalt.

While Facebook is not a cloud provider, it needs a lot of computing power to run its software, website, and ads. In April, the company announced that it had already implemented some of its own chips in data centers, which improved efficiency compared to GPUs.

In May, JPMorgan analysts predicted that the market for creating custom chips for major cloud providers could be worth up to $30 billion, with an estimated annual growth rate of 20%.

Startups

In 2023, venture capitalists invested $6 billion in AI semiconductor companies, a slight increase from the previous year's $5.7 billion, as seen in data from PitchBook.

Designing, developing, and manufacturing semiconductors can be costly for startups, but there are chances for differentiation.

Cerebras Systems, a Silicon Valley-based AI chipmaker, concentrates on optimizing AI operations and addressing performance limitations, rather than the broader functionality of a GPU. The company was established in 2015 and was valued at $4 billion during its most recent funding round, as reported by Bloomberg.

The WSE-2 chip, which is a Cerebras product, integrates GPU capabilities, central processing, and extra memory into a single device, making it ideal for training large models, according to CEO Andrew Feldman.

"We utilize a large chip, while they rely on numerous small chips," Feldman stated. "Their data transfer challenges are significant, unlike ours."

Despite facing competition from Nvidia, Feldman's company is still gaining clients, including the Mayo Clinic, the U.S. Military, and others, through its supercomputing systems.

Feldman believes that the presence of significant competition is beneficial for the ecosystem.

D-Matrix's Sheth announced that his company plans to launch a card with a chiplet this year, which will enable more computation in memory instead of on a chip like a GPU. This product can be easily integrated into an AI server with existing GPUs, reducing the workload on Nvidia chips and lowering the cost of generative AI.

Sheth stated that customers are highly receptive and motivated to facilitate the launch of a new solution.

Apple and Qualcomm

The potential danger to Nvidia's data center business could come from a shift in processing location.

AI work is increasingly being moved from server farms to personal devices like laptops, PCs, and phones.

While big models like those created by OpenAI need large clusters of powerful GPUs for inference, companies such as Apple and Microsoft are developing "small models" that require less power and data and can run on a battery-powered device. These models may not be as skilled as the latest version of ChatGPT, but they have other applications, such as summarizing text or visual search.

Neural processors, which offer privacy and speed benefits, are being added to Apple and Qualcomm's chips to improve AI efficiency.

Qualcomm recently unveiled a PC chip that enables laptops to utilize Microsoft's AI services directly on the device. Additionally, the company has invested in several chip manufacturers to produce low-power processors for running AI algorithms outside of smartphones or laptops.

Apple has been advertising its latest laptops and tablets as AI-optimized due to the neural engine on its chips. During its upcoming developer conference, Apple is expected to reveal a range of new AI capabilities, which will likely be powered by the company's iPhone-based silicon.

WATCH: Nvidia and Microsoft overshadowed weakness in tech stocks