Apple MetalFX Upscaling Technology for Mac Gaming Unveiled

Apple Metal is the fruity Cupertino outfit's equivalent of DirectX. It's an API that lets games developers tap into the hardware inside Apple platforms, for faster and more responsive gaming. Alongside the introduction of the first M2 Macs and new Metal 3 on Monday, we learned about MetalFX Upscaling for the first time.

YouTube

Watch On

Watch the Apple WWDC2022 keynote video segment, where MetalFX Upscaling is addressed here, or you can play the embedded video above and mouse forward to about 1 hour and 22 minutes. The gaming segment lasts about three or four minutes and includes a talk from a senior member of Capcom's Resident Evil Village development team.

Apple's technology reveal starts with an explanation that MetalFX Upscaling uses a mix of "high quality spatial upscaling techniques and temporal anti-aliasing." The video of No Man's Sky running as part of the presentation isn't very meaningful as we aren't told what hardware is being used to run this game, nor do we have any performance statistics.

We get a deeper explanation of MetalFX Upscaling and its potential from the Capcom executive in the video. He talks us through the graphical features and performance of Apple hardware in Resident Evil Village. The exec claims that Apple Silicon is becoming a great platform for gaming, and in this segment we see some gameplay footage, which is said to be "screaming" fast on new Macs thanks to Metal 3 advances. The video shows fluid movement through complex scenes with purported high quality textures, geometry, and complex shaders used.

Some rough performance estimates are provided by the Capcom executive. He says that using the latest MacBook Air, Resident Evil Village was "running effortlessly at 1080p." For higher resolution gaming, it was stated that the Mac Studio provided a "breathtaking 4K experience." Overall, a good experience could be had on "every Mac with Apple Silicon" according to the game developers. We'd assume that means at least 30 fps, but no hard figures were provided.

Both No Man's Sky and Resident Evil Village for Mac will be released later this year.

MetalFX Upscaling vs the PC Rivals

That's all we have on Apple's MetalFX Upscaling at this time, but the lay of the upscaled land is becoming increasingly packed with options. Graphics upscaling techniques are becoming increasingly important for reaching high framerates at high resolutions.

Nvidia was first out of the gate with DLSS in August 2019, which was partly an attempt to convince gamers of the value of its new (at the time) Tensor Cores, as well as an effort to address the poor real-time raytracing performance of the first generation of GeForce RTX GPUs with ray tracing acceleration cores (i.e. the RTX 20-series). AMD released FidelityFX Super Resolution (FSR) as a performance boosting alternative to DLSS in June 2021, and then followed up with FSR 2.0 in May 2022. Intel's XeSS is still under development, with performance optimized for Arc Alchemist's Xe Matrix Extensions (Intel XMX), though it should also work in some fashion on non-Intel hardware.

Nvidia DLSS 1.x and AMD's FSR 1 were based on spatial upscaling, but the second generation versions of these technologies included superior and more compute intensive temporal upscaling in their code. DLSS 2.0 was officially introduced in March 2020 (though a variation of it was used in Control before that time), and FSR 2.0 only recently became available in games, starting with Deathloop, and then Farming Simulator and God of War added the tech.

In brief, spatial upscaling means the only data used for upscaling a frame comes from the frame itself. Temporal upscaling uses data from the current frame as well as previous frames for better results, and it also uses the depth buffer (z-buffer) and a motion vector buffer to better track changes between frames. Also of note is that FSR 1.0 doesn't attempt to do anti-aliasing, while FSR 2.0, DLSS, and XeSS handle both upscaling and anti-aliasing.

Where does Apple's MetalFX Upscaling fit into the mix? It's not entirely clear how it will perform yet, but the claim that it uses both spatial and temporal algorithms seems a bit odd. Temporal upscaling uses the current frame (spatial data) combined with previous frames and other data, so perhaps that's what Apple is doing. Of course, MetalFX Upscaling will only run on Apple's M1 and M2 chips, using AI Neural Engine for processing, so direct comparisons with the other upscaling solutions may prove difficult.

Apple itself as well as Mac gamers will definitely benefit from having a good upscaling solution, due to Apple's use of high resolution "Retina" displays. Even the fastest GPUs can choke when attempting to render complex games at 4K, and upscaling can provide a major boost to performance. With the M2 GPU only rated at 3.6 teraflops (that's less than a Radeon RX 6500 XT), Apple needs all the help it can get.