{"id":11457,"date":"2024-03-21T22:00:00","date_gmt":"2024-03-21T22:00:00","guid":{"rendered":"https:\/\/modernsciences.org\/staging\/4414\/?p=11457"},"modified":"2024-03-07T09:58:47","modified_gmt":"2024-03-07T09:58:47","slug":"what-is-a-gpu-an-expert-explains-the-chips-powering-the-ai-boom-and-why-theyre-worth-trillions","status":"publish","type":"post","link":"https:\/\/modernsciences.org\/staging\/4414\/what-is-a-gpu-an-expert-explains-the-chips-powering-the-ai-boom-and-why-theyre-worth-trillions\/","title":{"rendered":"What is a GPU? An expert explains the chips powering the AI boom, and why they\u2019re worth\u00a0trillions"},"content":{"rendered":"\n
\n \n
\n \n AMD<\/span><\/span>\n <\/figcaption>\n <\/figure>\n\nConrad Sanderson<\/a>, CSIRO<\/a><\/em><\/span>\n\n

As the world rushes to make use of the latest wave of AI technologies, one piece of high-tech hardware has become a surprisingly hot commodity: the graphics processing unit, or GPU. <\/p>\n\n

A top-of-the-line GPU can sell for tens of thousands of dollars<\/a>, and leading manufacturer NVIDIA has seen its market valuation soar past US$2 trillion<\/a> as demand for its products surges.<\/p>\n\n

GPUs aren\u2019t just high-end AI products, either. There are less powerful GPUs in phones, laptops and gaming consoles, too.<\/p>\n\n

By now you\u2019re probably wondering: what is a GPU, really? And what makes them so special?<\/p>\n\n

What is a GPU?<\/h2>\n\n

GPUs were originally designed primarily to quickly generate and display complex 3D scenes and objects, such as those involved in video games and computer-aided design<\/a> software. Modern GPUs also handle tasks such as decompressing<\/a> video streams. <\/p>\n\n

The \u201cbrain\u201d of most computers is a chip called a central processing unit (CPU). CPUs can be used to generate graphical scenes and decompress videos, but they are typically far slower and less efficient on these tasks compared to GPUs. CPUs are better suited for general computation tasks, such as word processing and browsing web pages.<\/p>\n\n

How are GPUs different from CPUs?<\/h2>\n\n

A typical modern CPU is made up of between 8 and 16 \u201ccores<\/a>\u201d, each of which can process complex tasks in a sequential manner.<\/p>\n\n

GPUs, on the other hand, have thousands of relatively small cores, which are designed to all work at the same time (\u201cin parallel\u201d) to achieve fast overall processing. This makes them well suited for tasks that require a large number of simple operations which can be done at the same time, rather than one after another. <\/p>\n\n\n\n

Traditional GPUs come in two main flavours. <\/p>\n\n

First, there are standalone chips, which often come in add-on cards for large desktop computers. Second are GPUs combined with a CPU in the same chip package, which are often found in laptops and game consoles such as the PlayStation 5. In both cases, the CPU controls what the GPU does.<\/p>\n\n

Why are GPUs so useful for AI?<\/h2>\n\n

It turns out GPUs can be repurposed to do more than generate graphical scenes. <\/p>\n\n

Many of the machine learning techniques behind artificial intelligence (AI), such as deep neural networks<\/a>, rely heavily on various forms of \u201cmatrix multiplication\u201d. <\/p>\n\n

This is a mathematical operation where very large sets of numbers are multiplied and summed together. These operations are well suited to parallel processing, and hence can be performed very quickly by GPUs.<\/p>\n\n

What\u2019s next for GPUs?<\/h2>\n\n

The number-crunching prowess of GPUs is steadily increasing, due to the rise in the number of cores and their operating speeds. These improvements are primarily driven by improvements in chip manufacturing by companies such as TSMC<\/a> in Taiwan. <\/p>\n\n

The size of individual transistors \u2013 the basic components of any computer chip \u2013 is decreasing, allowing more transistors to be placed in the same amount of physical space. <\/p>\n\n

However, that is not the entire story. While traditional GPUs are useful for AI-related computation tasks, they are not optimal.<\/p>\n\n

Just as GPUs were originally designed to accelerate computers by providing specialised processing for graphics, there are accelerators that are designed to speed up machine learning tasks. These accelerators are often referred to as \u201cdata centre GPUs\u201d. <\/p>\n\n

Some of the most popular accelerators, made by companies such as AMD and NVIDIA, started out as traditional GPUs. Over time, their designs evolved to better handle various machine learning tasks, for example by supporting the more efficient \u201cbrain float<\/a>\u201d number format. <\/p>\n\n

\n \"A<\/a>\n
\n NVIDIA\u2019s latest GPUs have specialised functions to speed up the \u2018transformer\u2019 software used in many modern AI applications.<\/span>\n NVIDIA<\/a><\/span>\n <\/figcaption>\n <\/figure>\n\n

Other accelerators, such as Google\u2019s Tensor Processing Units<\/a> and Tenstorrent\u2019s Tensix Cores<\/a>, were designed from the ground up for speeding up deep neural networks.<\/p>\n\n

Data centre GPUs and other AI accelerators typically come with significantly more memory than traditional GPU add-on cards, which is crucial for training large AI models. The larger the AI model, the more capable and accurate it is.<\/p>\n\n

To further speed up training and handle even larger AI models, such as ChatGPT, many data centre GPUs can be pooled together to form a supercomputer. This requires more complex software in order to properly harness the available number crunching power. Another approach is to create a single very large accelerator, such as the \u201cwafer-scale processor<\/a>\u201d produced by Cerebras.<\/p>\n\n

Are specialised chips the future?<\/h2>\n\n

CPUs have not been standing still either. Recent CPUs from AMD and Intel have built-in low-level instructions that speed up the number-crunching required by deep neural networks. This additional functionality mainly helps with \u201cinference\u201d tasks \u2013 that is, using AI models that have already been developed elsewhere. <\/p>\n\n

To train the AI models in the first place, large GPU-like accelerators are still needed.<\/p>\n\n\n\n

It is possible to create ever more specialised accelerators for specific machine learning algorithms. Recently, for example, a company called Groq has produced a \u201clanguage processing unit<\/a>\u201d (LPU) specifically designed for running large language models along the lines of ChatGPT. <\/p>\n\n

However, creating these specialised processors takes considerable engineering resources. History shows the usage and popularity of any given machine learning algorithm tends to peak and then wane \u2013 so expensive specialised hardware may become quickly outdated. <\/p>\n\n

For the average consumer, however, that\u2019s unlikely to be a problem. The GPUs and other chips in the products you use are likely to keep quietly getting faster.\"The<\/p>\n\n

Conrad Sanderson<\/a>, Research Scientist & Team Leader, CSIRO<\/a><\/em><\/span><\/p>\n\n

This article is republished from The Conversation<\/a> under a Creative Commons license. Read the original article<\/a>.<\/p>\n\n","protected":false},"excerpt":{"rendered":"AMD Conrad Sanderson, CSIRO As the world rushes to make use of the latest wave of AI technologies,…\n","protected":false},"author":760,"featured_media":11436,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"nf_dc_page":"","fifu_image_url":"","fifu_image_alt":"","footnotes":""},"categories":[16],"tags":[334,254,1237,1288,474],"class_list":{"0":"post-11457","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-tech","8":"tag-artificial-intelligence","9":"tag-computer","10":"tag-computer-engineering","11":"tag-gpu","12":"tag-the-conversation","13":"cs-entry","14":"cs-video-wrap"},"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/modernsciences.org\/staging\/4414\/wp-json\/wp\/v2\/posts\/11457","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/modernsciences.org\/staging\/4414\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/modernsciences.org\/staging\/4414\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/modernsciences.org\/staging\/4414\/wp-json\/wp\/v2\/users\/760"}],"replies":[{"embeddable":true,"href":"https:\/\/modernsciences.org\/staging\/4414\/wp-json\/wp\/v2\/comments?post=11457"}],"version-history":[{"count":1,"href":"https:\/\/modernsciences.org\/staging\/4414\/wp-json\/wp\/v2\/posts\/11457\/revisions"}],"predecessor-version":[{"id":11458,"href":"https:\/\/modernsciences.org\/staging\/4414\/wp-json\/wp\/v2\/posts\/11457\/revisions\/11458"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/modernsciences.org\/staging\/4414\/wp-json\/wp\/v2\/media\/11436"}],"wp:attachment":[{"href":"https:\/\/modernsciences.org\/staging\/4414\/wp-json\/wp\/v2\/media?parent=11457"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/modernsciences.org\/staging\/4414\/wp-json\/wp\/v2\/categories?post=11457"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/modernsciences.org\/staging\/4414\/wp-json\/wp\/v2\/tags?post=11457"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}