Keller and Koduri headline the Beyond CUDA Summit today — AI leaders rally to challenge Nvidia's dominance

News
By published

Key leaders, researchers, and engineers will discuss how to bridge the 'CUDA moat.'

Beyond CUDA Speakers
(Image credit: TensorWave)

TensorWave, a cloud-platform for AI workloads powered by AMD's MI Instinct accelerators, is kicking off the Beyond CUDA Summit starting today. The event focuses on the concept of the 'CUDA moat,' and how developers can optimize their AI-centric workloads using other alternatives. Attendees can expect to see demonstrations, hot takes, panels, and expert opinions from influential leaders in the AI field including computer architect icons like Jim Keller and Raja Koduri.

It's no secret that Nvidia-built GPUs constitute the majority of hardware in the AI space. Although AMD's Instinct accelerators offer performance comparable to Nvidia hardware, the already-established and mature CUDA ecosystem is indispensable to some users / organizations. Nvidia realized the potential of parallel computing on its GPUs early on and developed a proprietary platform dubbed CUDA, which is now the de facto standard for GPU-accelerated computing.

Through continuous efforts, optimizations, and the sudden rise of AI which coincidentally is powered by GPUs, Nvidia has positioned itself as a leading solution provider. In fact, 90% of Nvidia's revenue is now driven by its data-center offerings, with CUDA being a central selling point. This creates a vendor lock-in situation, where CUDA (software) effectively confines the industry to Nvidia's hardware, limiting innovation and competition.

The industry is shifting gears to a more open-source and hardware-agnostic future, but that's easier said than done. We have OpenCL, ROCm, oneAPI, and Vulkan as alternatives, however, each trails Nvidia in one or many aspects. Enter Beyond CUDA, where key figures in the AI field have rallied up to congregate and develop a more diverse and heterogeneous future. Hosted by TensorWave, the Beyond CUDA Summit will address the many challenges the AI computing industry faces, such as hardware flexibility, cost efficiency, and exploring the available alternatives to CUDA.

Platforms like ROCm require significant developments to achieve parity with CUDA. Even now, ROCm only supports a small selection of modern GPUs while CUDA maintains compatibility with hardware dating back to 2006. AMD's latest RDNA 4 GPUs are still not officially supported by ROCm. Developers have long bemoaned AMD's slow adoption of new features and support on new hardware. On the positive side, Strix Halo is now ROCm-compatible, though only on Windows.

If you live in San Jose, buckle up as the summit takes place at The Guildhouse, which is with notable irony just three blocks away from the McEnery Convention Center, the site of Nvidia's GTC, which also commences today. Participants have the opportunity to win an AMD Instinct MI210 GPU with 64GB of HBM2e memory. The event runs from 12 PM to 10 PM PDT, with four time slots for various sessions. You can learn more details about the summit here.

See more GPUs News
TOPICS
Hassam Nasir
Hassam Nasir
Contributing Writer

Hassam Nasir is a die-hard hardware enthusiast with years of experience as a tech editor and writer, focusing on detailed CPU comparisons and general hardware news. When he’s not working, you’ll find him bending tubes for his ever-evolving custom water-loop gaming rig or benchmarking the latest CPUs and GPUs just for fun.

14 Comments Comment from the forums
  • ezst036
    We have 5 standards, each competing with the other and no clear winner. Here's an idea, what if everybody could rally behind just one standard?

    *creates new standard*

    Afterwards:

    There are 6 competing standards.
    Reply
  • bit_user
    The article said:
    ROCm only supports a small selection of modern GPUs while CUDA maintains compatibility with hardware dating back to 2006.
    This needs two big footnotes.
    AMD might finally be getting to grips with the technical problems that have limited the ROCm support window. We'll see.
    While CUDA works on older hardware, the latest CUDA versions do not. IIRC, each CUDA version has a minimum driver requirement and once a Nvidia GPU moves into legacy support status, it stops getting newer drivers. They still provide some bugfixes to the legacy support drivers, but you won't be able to run newer CUDA versions on it.
    Also, I think CUDA is a little bit of a trap that AMD (and, to some extent, Intel) walked right into. You can't build a better CUDA. The best CUDA-like thing will always be Nvidia's CUDA.

    There was a brief window of time where it looked like OpenCL might be able to gather enough momentum, but that fell apart as Apple and Google withdrew support for it, and then AMD got distracted by HSA and then HIP. At the time, Intel wasn't really a GPU player. More recently, we have Rusticle, which extends OpenCL to most GPUs supported by the open source Mesa userspace graphics API framework. While Rusticle offers the tantalizing promise of near-ubiquitous out-of-the-box OpenCL support, it comes very late in the game.

    Finally, CUDA is not the ideal API for AI. The CUDA programming model is suited for SIMD + SMT, which is great for fairly general-purpose parallel programming, but too wasteful for AI. I expect Nvidia to move beyond CUDA, for their future training chips! Maybe Rubin is that thing, maybe not.
    Reply
  • Giroro
    Koduri is directly responsible for putting both AMD and Intel into terrible positions to compete against Nvidia. Not just in AI, but in everything.
    Why would anybody expect him to have a strategy to fight Nvidia? He's the one who cleared a path for Nvidia to build a monopoly so easily. He's incompetent at best, maybe even a saboteur.
    I don't know if he's secretly on Nvidia's side, but he definitely has the poison touch - which real leaders at this summit would be wise to avoid.
    Reply
  • bit_user
    Giroro said:
    Koduri is directly responsible for putting both AMD and Intel into terrible positions to compete against Nvidia. Not just in AI, but in everything.
    The poor showing of Vega and Alchemist both happened under him, which does look very bad. Not to mention the excessive delays and horrible yields of Ponte Vecchio (i.e. the Intel Datacenter Max GPUs). Plus, Intel canceled a mid-range datacenter GPU, before it reached the market - what were they thinking, even starting with 3 simultaneous architectures, in their plan - 4, if you count the iGPUs??

    So, yeah. I tend to think he had a lot to do with bad decisions behind both of those product lines. However, to be fair, we should recognize that RDNA got started under Koduri, as well. So, I might still listen to what he has to say, but also take it with a grain of salt.

    Giroro said:
    Why would anybody expect him to have a strategy to fight Nvidia? He's the one who cleared a path for Nvidia to build a monopoly so easily.
    I wonder who actually signed off on AMD's software strategy. Because I think ROCm's rough track record had a lot more to do with why AMD hasn't been able to effectively serve the AI training market than AMD's hardware. MI300X is actually good hardware, but ROCm is what's been holding it back.

    Giroro said:
    He's incompetent at best, maybe even a saboteur.
    I don't know if he's secretly on Nvidia's side, but he definitely has the poison touch - which real leaders at this summit would be wise to avoid.
    I think you're really stepping over the line. Nvidia is a ruthless competitor and this is a tough business. To beat them, you need to be exceptional. I think Koduri just wasn't quite up to the challenge.

    Specifically, I think one of his main failings was that he was too ambitious - he didn't properly account for the limitations of his teams, so he over-promised and under-delivered. The smarter thing would've been to try to do more with less. At Intel, he should've never agreed to try and leap-frog Nvidia in a single generation. If you go back and look at the eye-watering complexity of Ponte Vecchio, it looks like it was designed by a mad man!
    Reply
  • blppt
    Any time Jim Keller is involved in anything, the tech world takes notice. Forget Koduri.
    Reply
  • hotaru251
    its really simple...just don't use/support cuda in future stuff. make a standard agreed by the ones involved that bars using cuda same way nvidia bans others using cuda.
    Reply
  • ekio
    Vulkan is an amazing API, available on all machines easily.
    I would try to improve Vulkan Compute instead of creating a new standard again nobody will use.
    Reply
  • bit_user
    hotaru251 said:
    its really simple...just don't use/support cuda in future stuff. make a standard agreed by the ones involved that bars using cuda same way nvidia bans others using cuda.
    Which standard? Intel has been pushing oneAPI; AMD has been pushing HIP.

    If the problem we're really trying to solve is AI, I've argued that you don't actually want something CUDA-like, at all. Then again, there are already AI standards like ONNX, though I don't know if they have any support for custom layers (that's where CUDA gets its nose under the tent).
    Reply
  • bit_user
    ekio said:
    Vulkan is an amazing API, available on all machines easily.
    It is not available on many purpose-built AI chips or FPGAs (which do offer support for OpenCL). The machines that support it are only GPUs. On a CPU, you can run Lavapipe, but I'm not even sure if that scales very well.

    ekio said:
    I would try to improve Vulkan Compute instead of creating a new standard again nobody will use.
    Vulkan is an absolute mess of different versions and extensions though. It's also a major pain to use.

    It doesn't even have a standard shading language, which is a big part of the problem CUDA solved, although Microsoft has recently made HLSL free to use with Vulkan.

    Next, I think SPIR-V is too GPU-centric. It doesn't address the concerns I have about CUDA-type solutions being a poor fit for AI.

    Finally, Vulkan is not a good solution for scientific or financial processing, due to significantly lower precision guarantees than OpenCL provides.

    If the way forward is through Vulkan, it will still need a lot of work. That much, I can say for sure.
    Reply
  • wwenze1
    You know you've won when you have people getting together to talk about "beyond you" and never invited you.

    Also isn't CUDA a trademarked term? Why are they using this term for this event?
    Reply