Samsung's failure to keep up with the AI industry resulted in a $126 billion loss.

• SK Hynix has surpassed Samsung Electronics in the development of high-bandwidth memory technology.
• Nvidia's chip architecture utilizes HBM as a crucial element for training large AI models.
• Nvidia's approval of Samsung's HBM is still pending, which could help Samsung regain its competitive edge, according to analysts.

Samsung Electronics held a dominant position in the memory semiconductor market, which placed it well to take advantage of the AI boom.

Samsung, the South Korea electronics giant, has fallen behind its long-time rival SK Hynix in next-generation chips that have been a key component for AI silicon leader. As a result, Samsung's profit has plummeted, wiping off around $126 billion from its market value, according to data from S&P Capital IQ. An executive from Samsung issued a rare public apology about the company's recent financial performance.

A crucial type of chip used to store data is memory, which is present in a wide range of devices, including smartphones and laptops. For many years, Samsung has been the dominant force in this technology, surpassing South Korean competitor SK Hynix and U.S. rival in terms of performance.

As AI applications like OpenAI's ChatGPT gained popularity, the focus shifted to the infrastructure needed to train the large models they rely on. Nvidia has emerged as the leader in this area with its GPUs, which are widely used by tech giants for AI training.

High-bandwidth memory, or HBM, is a vital component of the next generation of semiconductor architecture. Although it initially had a limited market, its demand has significantly increased with the rise of AI.

That's where Samsung got caught out and failed to invest.

For a long time, HBM has been a niche product and Samsung has not dedicated its resources to its development, according to Kazunori Ito, director of equity research at Morningstar, who shared this information via email to CNBC.

"The belief was that the high development costs of stacking DRAMs were not justified due to the small size of the addressable market and the complexity of the technology involved."

HBM chips approved for use in Nvidia architecture helped SK Hynix establish a close relationship with the U.S. giant. Nvidia's CEO even requested the company to expedite the supply of its next generation chip, highlighting the significance of HBM in its products.

SK Hynix posted record quarterly operating profit in the September quarter.

SK Hynix's strong R&D investments and industry partnerships enable it to maintain a competitive edge in HBM innovation and market penetration, according to Brady Wang, associate director at Counterpoint Research, who shared this information via email to CNBC.

Samsung reported that HBM sales grew more than 70% in the third quarter compared to the previous quarter. The company stated that its current product, HBM3E, is in mass production and generating sales.

The company is currently working on its next-generation HBM4 and aims to begin mass production in the second half of 2025.

Can Samsung make a comeback?

According to analysts, Samsung is falling behind competitors due to several factors, including insufficient investment in HBM and not being a first-mover.

According to Morningstar's Ito, Samsung has not been able to close the gap with SK Hynix on the HBM development roadmap.

Nvidia's role in Samsung's short-term comeback is evident.

Nvidia has not yet approved Samsung as a HBM supplier due to a strict qualification process. However, if Samsung passes this verification, it could lead to growth and increased competition with SK Hynix, according to analysts.

NVIDIA's approval is crucial for Samsung to capitalize on the growing demand for AI servers, given that NVIDIA dominates the AI chip market and most HBMs are used in this sector, according to Ito.

Samsung has made "significant advancements" in HBM3E and has "completed a crucial stage in the certification process."

The spokesperson stated that sales expansion is anticipated to begin in the fourth quarter.

Samsung's research and development capabilities and semiconductor manufacturing capacity could help it close the gap with SK Hynix, according to Wang.