Developer creates a basic first person shooter game using Gaussian splats, and you can play it for free in your browser
It's not much of a game, but it's an impressive proof of concept for the novel technology.
It's true: London-based developer Iakov Sumygin has created a minimalistic playable video game using Gaussian splats for the environment data, and you can load it up and try it right now in your browser. There's not much to do, but we're a tech site, not a dedicated gaming site, and so, as you could imagine, the more interesting part of this story is the technology behind it rather than the extremely basic FPS gameplay.
To understand this post, you're going to have to understand what Gaussian splats are. If you're familiar with the concept of volume pixels, or "voxels," Gaussian splats can be thought of as "voxels of variable size and density." They're not really like voxels; they don't exist in a fixed grid, and they aren't physicalized the same way voxels are. In essence, Gaussian splats are a data type that you can use for representing a 3D world with extremely high visual fidelity, legitimately photo-realistic quality, yet with a modest rendering cost.
Splat scenes, sometimes called "3DGS" scenes, are created quite simply by taking many photos or even videos of a real space and then performing a process called "Structure-from-Motion," which essentially interprets the video data as a 3D scene and creates the sparse point cloud. Then, scene optimization is performed (often using AI) to adjust each Gaussian's position, shape, opacity, and color to match the input images or video.
Gaussian splats are a fairly new technology; while the core idea dates back to Lee Westover's work in the early 1990s, the modern version mostly stems from a paper published by French researchers in 2023 that refined the concept. Since then, the tech has been developed and used quite a bit; last year's Superman movie famously used Gaussian splats for complex visual effects, while musician A$AP Rocky has released music videos with the technique, including one ("HELICOPTER") based on a newer "4D Gaussian Splatting" method that bakes a temporal element into the capture, allowing for moving objects.
Now, you'll notice I said "visual" fidelity above, and therein lies the rub: Gaussian splats can't really be used for interactive media because they don't capture anything about the world except how it looks. Or can they? That's the real innovation of Sumygin's work: taking a pre-made scan of an abandoned building and running it through specialized tools (which he created) to voxelize the splats and create a collision mesh. Boom: workable 3D geometry for a video game.
After that, he baked a lighting grid to apply lighting to the object and character models he imported to the project, vibe-coded a basic AI and pathfinding method for the enemy soldiers, and dropped the whole thing on PlayCanvas, which is a browser-based game engine platform owned by Snap, the company that Sumygin works for. (Yes, that's the SnapChat guys.)
Speaking as a gamer, the demo isn't particularly interesting as a video game. There's one clumsy-looking gun, one enemy model with very static animations, many bugs (my gun was eating more ammo per shot every time I reloaded it), and it's very easy to break, such as by falling through the world. I'm also not as taken with the visuals as some people seem to be. While it does indeed look convincingly photorealistic at points, fine detail is basically nil; in parts, it reminds me of Id Software's Rage, which had gorgeous graphics until you looked too closely at things. Frankly, Unreal Engine 5 projects like Unrecord and Bodycam do the same thing better.
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.
It's really what Sumygin's little game represents that is interesting, because Sumygin's project is completely fascinating from a technical perspective. He doesn't say how long it took him to make, but now that the tooling is done, it should be relatively straightforward and quick for anyone to 3D scan an environment and create a game with impressive visuals based in that environment. It's quite fascinating as a concept. Oh, and the whole project is under 100 megabytes, which is also quite impressive for the level of detail.
If you're a developer interested in the technique, Sumygin's project is completely open-source over on PlayCanvas, including the assets that he used. Frankly, the project is mostly an advertisement for his company's SuperSplat product, but it's an effective one, and it gives a tantalizing glimpse into what could be a very handy tool in the game developers' toolbox.
Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.
Zak is a freelance contributor to Tom's Hardware with decades of PC benchmarking experience who has also written for HotHardware and The Tech Report. A modern-day Renaissance man, he may not be an expert on anything, but he knows just a little about nearly everything.
-
Findecanor The locale of the game (as far I could get) consisted of mostly flat surfaces though, so it did not really showcase much of what the technology is capable of compared to texture-mapped triangles.Reply -
Jabberwocky79 Is this the same technology that I was reading about 10 years ago? A quick Google search yielded this description:Reply
Euclideon Unlimited Detail: A 3D graphics engine developed by the Australian company Euclideon Pty Ltd. It used a point cloud search engine indexing system to render "unlimited detail" without polygon meshes, aiming to replace traditional rasterization. While it garnered significant attention and government grants, it faced skepticism from industry experts like John Carmack and was ultimately deemed unsuitable for real-time gaming at the time due to massive data storage and I/O limitations.
I remember it required significantly less GPU power to run, and at the time, it was theorized that Nvidia was undermining the value of the concept for fear it would eat into their profits. -
klayrity Reply
Euclidian unlimited detail is a voxel mesh renderer, which gets more detailed the closer you get. Unreal Engine 5 uses similar technology with the version of nanite they've implemented for their foliage.Jabberwocky79 said:Is this the same technology that I was reading about 10 years ago? A quick Google search yielded this description:
Euclideon Unlimited Detail: A 3D graphics engine developed by the Australian company Euclideon Pty Ltd. It used a point cloud search engine indexing system to render "unlimited detail" without polygon meshes, aiming to replace traditional rasterization. While it garnered significant attention and government grants, it faced skepticism from industry experts like John Carmack and was ultimately deemed unsuitable for real-time gaming at the time due to massive data storage and I/O limitations.
I remember it required significantly less GPU power to run, and at the time, it was theorized that Nvidia was undermining the value of the concept for fear it would eat into their profits.
Gaussian splats are rendering a point cloud of millions of blobs of color on flat planes. Those coloured blobs change color depending on view angle, what is referred to as a radiance field, allowing each splat to represent the object differently when viewed from different points in space. Instead of using a lighting model, all of the lighting information is baked into the cloud of splats, freeing up rendering overhead, at the cost of memory, making it a cheaper rendering option for higher quality results.
Problem being that it's basically impossible to edit geometry captured by splats, it's just as difficult to animate it, it's difficult to relight the capture (though not impossible with some AI tools that are being released), and is overall rather unwieldy and hard to implement. -
edzieba Reply
Euclideon was a scam, or at the absolute best, someone who had little idea of the limitations of voxel storage who though they could blag investment until they - ultimately failed to - solve the fundamental issues. It was a basic point-cloud, and relied on unrotated tiling of the same point cluster over and over to achieve even vaguely adequate performance. No animation or rotation was possible, and even non-integer shifting was a challenge.Jabberwocky79 said:Is this the same technology that I was reading about 10 years ago? A quick Google search yielded this description:
Euclideon Unlimited Detail: A 3D graphics engine developed by the Australian company Euclideon Pty Ltd. It used a point cloud search engine indexing system to render "unlimited detail" without polygon meshes, aiming to replace traditional rasterization. While it garnered significant attention and government grants, it faced skepticism from industry experts like John Carmack and was ultimately deemed unsuitable for real-time gaming at the time due to massive data storage and I/O limitations.
I remember it required significantly less GPU power to run, and at the time, it was theorized that Nvidia was undermining the value of the concept for fear it would eat into their profits.
Gaussian Splats sidestep the issue by using far, far sparser point-clouds and achieving ;density; by replacing each point with a gaussian. The gaussian is a fuzzy blob with non-zero width, length, and orientation, so a few more values sorted per point covers a massively greater volume.
As for splats for gaming: probably not much good for direct environments for the moment - you need geometry surfaces 'behind'; the splats anyway for collisions/pathing/lighting/etc so may as well render the geometry. But it could be exceptionally for replacing mid/far geometry. For example, skyboxes, distance cityscapes, huge forests (organics like foliage work very well with gaussian splats), etc.
Whilst a lot of the wow-demos for gaussian splatting is based on photogrammetry of real scenes, there is nothing to stop you capturing the raw imagery to train the model using a rendering engine instead. This means you could create a splat of your entire level offline at very high render quality, stream in the distant areas in the real-time engine, and swap out the splat for real-time rendered geometry near the player for interaction. That lets you replace low-detail LOD models with a pre-rendered splat, without the noninteractivity issues. -
bit_user Good discussion, here! I appreciate hearing both of your thoughts on this, @klayrity and @edzieba!Reply
My gut feeling is that you're onto something, here. I think LoD-based approaches break down as objects (e.g. trees, boulders, buildings, etc.) shrink below the size of a pixel. This might be where Gaussian Splatting could really come into its own. Perhaps it needed be derived from photogrammetry, either. Maybe you could even generate a distant model of a locale that has a standard geometrical representation?edzieba said:But it could be exceptionally for replacing mid/far geometry. For example, skyboxes, distance cityscapes, huge forests (organics like foliage work very well with gaussian splats), etc. -
SmokyBarnable Interesting new technology used to showcase kill thrills with simulated guns. I weep for humans most days now.Reply -
bit_user Reply
It's just a rendering technology. It can be used for any sort of simulation, game, or virtual experience as other kinds of interactive 3D rendering techniques.SmokyBarnable said:Interesting new technology used to showcase kill thrills with simulated guns. I weep for humans most days now. -
TheEldritchVoid If used with other tools that'd definitely be neat, especially with the implications: since homie works at snap they can take the scanning tech and spin it into its own $5-$10 app for mapping real space to virtual space (could be neat the work out something with Google maps and pogo for VR quality street view) that would be huge for indie devs since they can scan in somewhere they think would make a cool map then add details and textures with whichever engine best suits their needs! Yea the big players are gonna have it and use it but I'm much more excited about stuff like this helping to level the playing field between massive AAA corps and solo indie dev working on their first project.Reply -
SmokyBarnable Reply
Yes it could be. But I’m talking about what it was actually used for in this early demo, and why.bit_user said:It's just a rendering technology. It can be used for any sort of simulation, game, or virtual experience as other kinds of interactive 3D rendering techniques.