Granite Texture Streaming 101: An Overview From Graphine’s CEO

Graphine recently released the fourth generation of its texture streaming middleware, Granite. We’ve covered Granite before, so we know it improves graphics performance and enables the use of higher resolution textures for 3D objects and environments. We’ve just never truly understood how Graphine’s magic works. Following the launch of Granite SDK 4.0, we decided it was high time we found out.

Shortly after the launch of Granite SDK 4.0, we had the chance to speak with Aljosha Demeulemeester, Graphine’s CEO, to get a better understanding of how Granite works and what it’s used for. If you’re a 3D content developer, especially if you make VR games, you may want to pay attention. Granite has the potential to significantly reduce the GPU workload while improving visual fidelity.

Demeulemeester started off with a quick summary of his company’s software: “Granite SDK is a middleware that can be integrated into any real-time 3D engine that is entirely focused on texture streaming,” he remarked. 

Clever Techniques Bring Out Better Performane

At its core, Graphine is designed to help curtail the hardware limitations that restrict the visual fidelity of 3D objects and environments, which is limited to the quality of the texture covering its surface. The highest polygon model will still look terrible if you cover it with a low-resolution texture. Higher resolution textures improve the visual quality, but as Demeulemeester put it, “[...] at a certain point, you just use too much video memory.”

He went on to paint a clearer picture of what he was talking about.

“To give you an example, if you have an 8K x 8K PBR (physically-based rendered) texture, which has a couple of different channels, then that might be 256MB of data for that one single texture, which could be just a character or a table or something like that. So, if you have 10 of these in your scene close to the camera, they already would need more than 2GB of video memory to render that correctly.”

That’s an awful lot of video memory for just “10 objects.” You can see how this scenario would get out of hand, even for today’s top-of-the-line graphics cards, which routinely carry 6GB, 8GB, and even 12GB of graphics memory.

If you want to use higher-than-8K resolution textures, the problem gets even worse. With 16K resolution textures, for example, you’re limited by more than memory constraints—at that point, you’re dealing with software limitations. “And if you want to go even higher in resolution, then the graphics drivers will just stop,” said Demeulemeester. “Graphics APIs like Direct X and OpenGL kind of limit the developers of using more than 16K texture resolution.”

Granite works around the texture resolution limitations by segmenting texture data into tiny pieces. The middleware splits each texture into 128 x 128-pixel tiles to create a “texture tile database” called a tileset. Graphine calls upon the tiles as necessary in real-time using prefetch algorithms. The middleware sends only the necessary texture segments to render the visible scene. In a way, the Granite SDK is built on the same principle as foveated rendering: It focuses only on what impacts the the parts you can see.

Demeulemeester continued:

“What [Granite] actually does is in the data loading side of things. It only loads the data into the GPU memory that is actually visible—the data that will actually be needed to render the screen pixels. We take the entire digital image and divide them into very small pieces. And then we're going to figure out, while the game is running—while the player is running around in the environment— which of those tiles are actually visible. The great thing about Granite is that all these extra pieces that are not visible; they’re not needed in memory. And so, we actually need much less memory, and you can add much more texture content to your environment, and have a more unique, and more much interesting environment.”

Of course, there’s a practical limitation to everything, and Granite can scale beyond practicality. Graphine’s middleware supports unfathomably large 256,000 x 256,000-pixel textures. And a single Granite project can use up to 64 individual tilesets that can each theoretically contain 128GB of texture data, for a potential total of 8TB of texture information.

You would never be able to ship a game with that much texture data, but Graphine isn’t limited to game development. It could also be used for visualization projects that demand incredibly high-resolution textures but don’t necessarily require the same rendering performance (framerate) that video games demand.

It’s not likely that a workstation would have access to enough hard drive storage for 8TB of texture data, but a project of that size wouldn’t rely on local storage. Granite is designed to work with local storage, but Graphine is working on network storage solutions. “We have prototypes of network streaming,” said Demeulemeester. “Right now, we store the texture data on disk, but the next step is streaming data from a cloud server.”

Streaming data from a network storage solution presents a new set of problems that Graphine must solve. Graphine relies on the predictable time it takes to retrieve texture data from a hard drive to fill the visible screen area at a consistent framerate. Network transfer is far less reliable and is often much slower than local hard drive access. Graphine is countering the network latency by “prefetching more content.” Graphine has a patent in prediction to help solve this issue.

The streaming option isn’t actually part of Granite, but Graphine is willing to entertain the idea of sharing the tech with the right developer.

“I can’t give all the details,” said Demeulemeester. “But if a developer would be looking for that, we can work with them, but it's not an off-the-shelf product. Granite is a product that streams from the local drive into video memory.”

Past releases of the Granite SDK supported only tile-based texture streaming, but that solution doesn’t work for all types of textures. Granite “needed a generic streaming system, instead of just one for super high-resolution textures” because the tile-based streaming system “doesn’t make sense,” Demeulemeester said, for 10-30% of textures.

“So, one of the interesting things that we added is a completely new streaming system to Granite,” said Demeulemeester. “We have this tile-based virtual texturing streaming system, but then now we also have a more old-school MIP map shading system, which is what Unreal is still using.”

Implications For Virtual Reality

Technology like Graphine’s Granite texture streaming middleware is invaluable for virtual reality development. The more interactive and experience, the bigger the need for high-resolution textures. Survios saw the need for high-resolution textures for Raw Data and recently partnered with Graphine for that very reason.

Demeulemeester continued:

“In virtual reality, you have a tendency to get really close to objects and really inspect them. So, once you get super close to them, you need super high-resolution textures to have some detail still. And also, you have these high-resolution screens, meaning you have more screen pixels, and there's a very strong desire to go even much higher resolution. And so, for every screen pixel, you actually need to have some texture data. So, more screen pixels mean you need more texture data; getting closer to objects means you also need more texture. So, there’s a very strong demand there to use super high-resolution textures. But on the other hand, there’s a very critical demand of having a solid performance on Vive and Oculus; solid 90fps. And on Daydream; solid 60[fps]. And so, you cannot have a streaming system that messes with your framerate—that suddenly starts to lock the rendering pipeline because it’s uploading a very big chunk of data.”

Granite Is Perfect For Photogrammetry

If you think Granite would help improve performance in photogrammetry environments, you’re not alone. In fact, Graphine is way ahead of you:

“Photogrammetry is a very good use case because Granite is very good at streaming very large resolution textures,” said Demeulemeester. “With photogrammetry, it’s a completely different approach where you actually scan an existing environment. If you’re scanning, you end up with every pixel in your environment being unique, so, you end up with a lot of texture content, and that is where Granite really shines.”

Flexible Licensing Plans

Graphine doesn’t advertise the price for a Graphine SDK license because the cost varies for every customer. Granite SDK is sold as a project license, which is to say developers negotiate the license for a specific development project and keep it for the duration of the project. Because every development team is different, and projects have variable development lengths, Graphine negotiates the price with every licensee. Demeulemeester said the company tries to cater to low-volume projects, too, but he said those projects usually opt for the less expensive plug-in licenses.

Graphine offers Unity and Unreal Engine Granite plug-ins, which it sells on a per-seat basis. Developers can opt for annual subscriptions, which give them access to the Granite plug-in and all updates for one year. There’s also a perpetual license option which includes lifetime access to Granite plug-in, and two years of updates.

"It’s a per case basis. We try to work something out for smaller projects," said Demeulemeester. "A lot of VR projects are smaller than larger game projects, so we want to be a good match with these VR projects, and we work something out. But in most cases, they are using Unity or Unreal, and for those people, we have a plug-in based on the SDK."

Graphine offers Indie and Pro versions of the Unity and Unreal Engine plug-ins. The Indie Unity license is $65 and lasts forever. The Pro version costs $199 per year, or you can throw down $349 for a lifetime license. The Unreal Engine plugins are much more expensive. The lifetime Indie version sells for $249, and the lifetime Pro version sells for $890. If you’d prefer an annual subscription, it will set you back $490 per year.

You can purchase a seat license for the Granite plug-ins directly from the Graphine website. You must fill out a request form if you're interested in an SDK license.