How can Amazon use customized AWS chips to catch up with Microsoft and Google in the AI field?
According to CNBC, the Amazon team is designing two chips, codenamed Inferentia and Training, for training and accelerating generative artificial intelligence. These chips provide an alternative option for Amazon Cloud Services AWS customers to train large language models instead of Nvidia graphics processors. Currently, the procurement of GPUs has become increasingly difficult and expensive, and these customized chips can provide AWS customers with alternative solutions for training large language models on NVIDIA chips.
Amazon Cloud Technology CEO Adam Selipsky said in June this year, "The world hopes to have more chips for generative artificial intelligence, whether it's GPUs or Amazon's self-developed chips we're designing
However, other companies have taken faster action and invested more funds to gain business from the wave of generative artificial intelligence. When OpenAI launched ChatGPT in November last year, Microsoft gained widespread attention for its chatbots and $13 billion investment in OpenAI. The company quickly added generative artificial intelligence models to its products and merged them into Bing in February of this year.
In the same month, Google launched its own large-scale language model, Bard, and subsequently invested $300 million in OpenAI competitor Anthropic.
It wasn't until April that Amazon announced the launch of its own large language model series (called Titan) and a Bedlock service to help developers use generative AI enhancement software.
Amazon is not used to chasing the market. Amazon is used to creating the market. I think for the first time in a long time, they have found themselves at a disadvantage and are working hard to catch up, "said Chirag Dekate, vice president analyst at Gartner.
Meta has recently released its own Llama 2 major models. This open source ChatGPT competitor can now be tested on Microsoft's Azure public cloud.
Chips are 'true differentiation'
Dekate stated that in the long run, Amazon's customized chips can give it an advantage in the field of generative artificial intelligence,
I think the true differentiation lies in the technical capabilities they exert, as Microsoft does not have customized chips for Training or Inferentia
AWS quietly began producing custom chips as early as 2013, using a specialized hardware called Nitro, which is now the best-selling AWS chip. Amazon claims that each AWS server has at least one, with a total usage of over 20 million.
In 2015, Amazon acquired Israeli chip startup Annapurna Labs. In 2018, Amazon launched the ARM based server chip Graviton, which is a competitor to x86 CPUs from giants such as AMD and Intel.
Arm may account for a high single digit or even 10% of total server sales, with a large portion coming from Amazon. Therefore, they are doing quite well in terms of CPU, "said Stacy Rasgon, senior analyst at Bernstein Research.
Also in 2018, Amazon launched chips focused on artificial intelligence. Two years ago, Google announced the launch of its first Tensor Processor Unit (TPU). Microsoft has not yet announced that it is developing the Athena AI chip, which is reportedly developed in collaboration with AMD.
Matt Wood, Vice President of Amazon Products, explained the uses of the Training and Inferentia chips.
Machine learning is divided into these two different stages. Therefore, you train machine learning models and then infer these trained models, "Wood said. Compared to any other method of training machine learning models on AWS, Training has improved its cost-effectiveness by approximately 50%
Following the release of Inferentia in 2019 (currently in its second generation), Training was first launched in 2021.
Inferentia "provides very low-cost, high throughput, and low latency machine learning inference, meaning that when you input a prompt in a generative artificial intelligence model, all predictions are processed here to give a response," Wood said.
However, currently, Nvidia's GPU is still the king in terms of training models. In July of this year, AWS launched a new AI acceleration hardware supported by NVIDIA H100.
Rasgon said, "Nvidia chips have a huge software ecosystem, and over the past 15 years, the software ecosystem built around them has been unparalleled by other companies. Currently, the biggest winner in artificial intelligence is Nvidia
Leveraging Cloud Advantages
However, AWS's cloud dominance is a major differentiation factor for Amazon.
Amazon doesn't need any extra attention. Amazon already has a very strong cloud installation foundation. All they need to do is figure out how to enable existing customers to leverage generative artificial intelligence to extend to value creation activities, "Dekate said.
When choosing generative artificial intelligence between Amazon, Google, and Microsoft, millions of AWS customers may be attracted to Amazon as they are already familiar with Amazon and run other applications and store data there.
Mai Lan Tomsen Bukovec, Vice President of AWS Technology, explained, "This is a question of speed. How quickly these companies can develop these generative artificial intelligence applications depends on how they start with data in AWS and use the computing and machine learning tools we provide to drive it
According to Gartner data, AWS is the world's largest cloud computing provider, with a market share of 40% in 2022. Although Amazon's operating profit has declined year-on-year for three consecutive quarters, AWS still accounted for 70% of Amazon's $7.7 billion operating profit in the second quarter. From a historical perspective, AWS's operating profit margin is much higher than Google Cloud.
In addition, AWS also has an increasing portfolio of developer tools focused on generative artificial intelligence. Swami Sivasubramanian, Vice President of Database, Analysis, and Machine Learning at AWS, said, "Let's turn the clock back, even before ChatGPT. It's not like after that, we suddenly rushed out with a plan because you can't design a new chip in that fast, let alone establish a basic service in 2 to 3 months
Bedlock can provide AWS customers with access to large language models developed by Antiopic, Stability AI, AI21 Labs, and Amazon Titan. Sivasubramanian said, "We don't believe that one model will rule the world. We want our customers to have state-of-the-art models from multiple suppliers because they will choose the right tools for the right job
One of Amazon's latest artificial intelligence products is AWS HealthScribe, which was launched in July and aims to help doctors draft patient visit summaries using generative artificial intelligence. Amazon also has a machine learning center called SageMaker, which provides services such as algorithms and models.
Another important tool is CodeWhisperer, which Amazon says has increased developers' task completion speed by an average of 57%. Last year, Microsoft also reported that its coding tool GitHub Copilot has improved work efficiency.
In June of this year, AWS announced a $100 million investment to establish a generative artificial intelligence innovation center. Adam Selipsky, CEO of AWS, said, "We have many clients who want generative artificial intelligence technology, but they may not necessarily know what it means to them in their own business context. Therefore, we will introduce solution architects, engineers, strategists, and data scientists to work with them one-on-one
Although AWS has so far mainly focused on developing tools rather than building competitors for ChatGPT, a recently leaked internal email shows that Amazon CEO Andy Jassy is directly overseeing a new central team that is also building scalable large language models.
During the second quarter financial report conference call, Jia Xi stated that a "significant portion" of AWS's business is now driven by artificial intelligence and its support for more than 20 machine learning services, with customers including Philips, 3M, Old Mutual and HSBC.
The explosive growth of artificial intelligence has brought about a series of security concerns, as companies are concerned that employees will incorporate proprietary information into training data used in the common big language model.
Adam Selipsky said, "I cannot tell you how many Fortune 500 companies I have contacted have disabled ChatGPT. Therefore, through our generative artificial intelligence methods and Bedlock services, anything you do and use through Bedlock will be in your own independent virtual private cloud environment. It will be encrypted and have the same AWS access control
At present, Amazon is only accelerating the promotion of generative artificial intelligence, claiming that "over 100000" customers are currently using machine learning on AWS. Although this only accounts for a small portion of AWS's millions of customers, analysts say this situation may change.
We don't see companies saying, 'Oh, wait a minute, Microsoft is already leading in generative artificial intelligence. Let's go out and change our infrastructure strategy, moving everything to microsoftware. If you're already a customer of Amazon, you're likely to explore the Amazon ecosystem more widely.'
Amazon Cloud Technology CEO Adam Selipsky said in June this year, "The world hopes to have more chips for generative artificial intelligence, whether it's GPUs or Amazon's self-developed chips we're designing
However, other companies have taken faster action and invested more funds to gain business from the wave of generative artificial intelligence. When OpenAI launched ChatGPT in November last year, Microsoft gained widespread attention for its chatbots and $13 billion investment in OpenAI. The company quickly added generative artificial intelligence models to its products and merged them into Bing in February of this year.
In the same month, Google launched its own large-scale language model, Bard, and subsequently invested $300 million in OpenAI competitor Anthropic.
It wasn't until April that Amazon announced the launch of its own large language model series (called Titan) and a Bedlock service to help developers use generative AI enhancement software.
Amazon is not used to chasing the market. Amazon is used to creating the market. I think for the first time in a long time, they have found themselves at a disadvantage and are working hard to catch up, "said Chirag Dekate, vice president analyst at Gartner.
Meta has recently released its own Llama 2 major models. This open source ChatGPT competitor can now be tested on Microsoft's Azure public cloud.
Chips are 'true differentiation'
Dekate stated that in the long run, Amazon's customized chips can give it an advantage in the field of generative artificial intelligence,
I think the true differentiation lies in the technical capabilities they exert, as Microsoft does not have customized chips for Training or Inferentia
AWS quietly began producing custom chips as early as 2013, using a specialized hardware called Nitro, which is now the best-selling AWS chip. Amazon claims that each AWS server has at least one, with a total usage of over 20 million.
In 2015, Amazon acquired Israeli chip startup Annapurna Labs. In 2018, Amazon launched the ARM based server chip Graviton, which is a competitor to x86 CPUs from giants such as AMD and Intel.
Arm may account for a high single digit or even 10% of total server sales, with a large portion coming from Amazon. Therefore, they are doing quite well in terms of CPU, "said Stacy Rasgon, senior analyst at Bernstein Research.
Also in 2018, Amazon launched chips focused on artificial intelligence. Two years ago, Google announced the launch of its first Tensor Processor Unit (TPU). Microsoft has not yet announced that it is developing the Athena AI chip, which is reportedly developed in collaboration with AMD.
Matt Wood, Vice President of Amazon Products, explained the uses of the Training and Inferentia chips.
Machine learning is divided into these two different stages. Therefore, you train machine learning models and then infer these trained models, "Wood said. Compared to any other method of training machine learning models on AWS, Training has improved its cost-effectiveness by approximately 50%
Following the release of Inferentia in 2019 (currently in its second generation), Training was first launched in 2021.
Inferentia "provides very low-cost, high throughput, and low latency machine learning inference, meaning that when you input a prompt in a generative artificial intelligence model, all predictions are processed here to give a response," Wood said.
However, currently, Nvidia's GPU is still the king in terms of training models. In July of this year, AWS launched a new AI acceleration hardware supported by NVIDIA H100.
Rasgon said, "Nvidia chips have a huge software ecosystem, and over the past 15 years, the software ecosystem built around them has been unparalleled by other companies. Currently, the biggest winner in artificial intelligence is Nvidia
Leveraging Cloud Advantages
However, AWS's cloud dominance is a major differentiation factor for Amazon.
Amazon doesn't need any extra attention. Amazon already has a very strong cloud installation foundation. All they need to do is figure out how to enable existing customers to leverage generative artificial intelligence to extend to value creation activities, "Dekate said.
When choosing generative artificial intelligence between Amazon, Google, and Microsoft, millions of AWS customers may be attracted to Amazon as they are already familiar with Amazon and run other applications and store data there.
Mai Lan Tomsen Bukovec, Vice President of AWS Technology, explained, "This is a question of speed. How quickly these companies can develop these generative artificial intelligence applications depends on how they start with data in AWS and use the computing and machine learning tools we provide to drive it
According to Gartner data, AWS is the world's largest cloud computing provider, with a market share of 40% in 2022. Although Amazon's operating profit has declined year-on-year for three consecutive quarters, AWS still accounted for 70% of Amazon's $7.7 billion operating profit in the second quarter. From a historical perspective, AWS's operating profit margin is much higher than Google Cloud.
In addition, AWS also has an increasing portfolio of developer tools focused on generative artificial intelligence. Swami Sivasubramanian, Vice President of Database, Analysis, and Machine Learning at AWS, said, "Let's turn the clock back, even before ChatGPT. It's not like after that, we suddenly rushed out with a plan because you can't design a new chip in that fast, let alone establish a basic service in 2 to 3 months
Bedlock can provide AWS customers with access to large language models developed by Antiopic, Stability AI, AI21 Labs, and Amazon Titan. Sivasubramanian said, "We don't believe that one model will rule the world. We want our customers to have state-of-the-art models from multiple suppliers because they will choose the right tools for the right job
One of Amazon's latest artificial intelligence products is AWS HealthScribe, which was launched in July and aims to help doctors draft patient visit summaries using generative artificial intelligence. Amazon also has a machine learning center called SageMaker, which provides services such as algorithms and models.
Another important tool is CodeWhisperer, which Amazon says has increased developers' task completion speed by an average of 57%. Last year, Microsoft also reported that its coding tool GitHub Copilot has improved work efficiency.
In June of this year, AWS announced a $100 million investment to establish a generative artificial intelligence innovation center. Adam Selipsky, CEO of AWS, said, "We have many clients who want generative artificial intelligence technology, but they may not necessarily know what it means to them in their own business context. Therefore, we will introduce solution architects, engineers, strategists, and data scientists to work with them one-on-one
Although AWS has so far mainly focused on developing tools rather than building competitors for ChatGPT, a recently leaked internal email shows that Amazon CEO Andy Jassy is directly overseeing a new central team that is also building scalable large language models.
During the second quarter financial report conference call, Jia Xi stated that a "significant portion" of AWS's business is now driven by artificial intelligence and its support for more than 20 machine learning services, with customers including Philips, 3M, Old Mutual and HSBC.
The explosive growth of artificial intelligence has brought about a series of security concerns, as companies are concerned that employees will incorporate proprietary information into training data used in the common big language model.
Adam Selipsky said, "I cannot tell you how many Fortune 500 companies I have contacted have disabled ChatGPT. Therefore, through our generative artificial intelligence methods and Bedlock services, anything you do and use through Bedlock will be in your own independent virtual private cloud environment. It will be encrypted and have the same AWS access control
At present, Amazon is only accelerating the promotion of generative artificial intelligence, claiming that "over 100000" customers are currently using machine learning on AWS. Although this only accounts for a small portion of AWS's millions of customers, analysts say this situation may change.
We don't see companies saying, 'Oh, wait a minute, Microsoft is already leading in generative artificial intelligence. Let's go out and change our infrastructure strategy, moving everything to microsoftware. If you're already a customer of Amazon, you're likely to explore the Amazon ecosystem more widely.'