Sign in with
Sign up | Sign in
DICE's Johan Andersson Talks BF4, Frostbite, Mantle, The Future
By ,
1. Chris Angelini And Johan Andersson Talk Battlefield 4 And Frostbite 3

Last week I sat down with Johan Andersson, who you know as a rendering architect in the Frostbite engine team at DICE, to talk about Battlefield 4, Frostbite 3, next-gen consoles, and AMD's Mantle initiative. Of course, if you've been enjoying Battlefield 4 in the weeks since its launch, then you're already familiar with his work.

Back when Battlefield 3 was first introduced, I spent several days playing through the single-player campaign and tooling around in the multi-player component, benchmarking the latest graphics cards and CPUs. Johan helped provide feedback on bottlenecks, graphics features, and testing. Two years later, it was good to catch up with him again.

Chris Angelini: Great talking to you again Johan, and congratulations on Battlefield 4. Played through the entire campaign on launch night to come up with a benchmark for my GTX 780 Ti review.

Johan Andersson: Yeah, it's good talking to you, too.

Chris: So, I've been thinking about the new game, Frostbite 3, consoles, and Mantle. To start, you've clearly doing a lot with AMD. Is DICE using middleware for Battlefield 4's audio, or it an in-house solution? Are there any plans to incorporate TrueAudio support?

Johan: We’re using our own audio solutions essentially that we have here in-house. Currently there's no plans on using TrueAudio. All of that came quite late for us and we were pretty happy with what we had. In the future, we're probably going to investigate this. Not for BF4 specifically, but for Frostbite in general. We haven’t really had time to fully evaluate it and what it could mean.

Chris: Alright. So, what feature of Battlefield 4 excited you the most to see implemented successfully? 

Johan: That's a good question. I'm not really sure. I guess it may sound like marketing thing, but I'll say Levolution. We've had dynamic radiosity in our engine for quite some time. And we've extended the engine to be quite a bit more flexible now with Frostbite 3, but just seeing artists playing around with the entire Frostbite toolbox of various types of lighting changes and weather changes and effects, and using that in practice. Like Paracel Storm for example in BF4 when you see that the entire level becomes stormy and you see the water changing and it starts to rain. The sky is changing. Actually the lighting change is quite significant on the level, and also we added a lot of dynamic wind simulation on trees and various types of foliage. Just seeing that all sort of come together, and then of course we added the marketing name Levolution for it. I think that's really cool and actually impacts gameplay quite a bit, which is fun.

Chris: Yeah and I'd agree. Levolution and what it means for destruction is one of my favorite things about the game as well. What technically needed to be done to make that happen? Why haven’t more games gone that route of adding more destructible terrain? 

Johan: It's a little bit difficult to say. All games have different constraints, right? If you're building a single-player campaign, it may not really make sense to do that. Though, for Battlefield and its multi-player game, it's always been quite large-scale and quite a lot of different things going on in the levels and just getting more and more dynamic things in there made sense for us. It's also something that sort of signifies a little bit that its become more of a mature engine. When you're not just working on your low-level systems or you're focusing on just building your renderer you're actually having the content creators knowing how to utilize the engine and extending that requires quite a lot of work. And then to have the artists that are very used to working with your engine; it requires a lot of flexibility to be honest, inside the engine also, being able to control all of these systems and making sure they play nicely together. It can also be quite heavy from a performance point of view, just simulating wind and doing all of the water simulation that we’re doing. That requires massive amounts of parallelism and compute, especially trying to do that at 60 FPS on the consoles.

Chris: Right.

Johan: It requires a lot of effort for us there. I don’t think there's probably not that many other studios or game teams out there that have all of those things together to be able to accomplish something like that.

Chris: Yeah, sure, okay. I believe you're already using a compute shader for tile-based deferred shading in Frostbite 2 right? 

Johan: Yes.     

Chris: How does Frostbite 3 build on what you're doing with DirectCompute? 

Johan: We have support for the compute shader; that's essentially our most important part. We’ve significantly optimized the lighting compute shader since BF3, also of doing more optimizations on that which is quite nice. We've also done some work of doing some of our blurring and post-processes inside it. Oh, and our sprite depth of field actually uses a compute shader. We use this on Ultra detail settings for the PC to get sort of the bokeh shape for our depth of field. The compute shader sort of orchestrates of all that to make sure it routes a really good performance. Then we render it using the graphics pipeline. We've sort of gotten our feet wet in quite a few different places with DirectCompute and going forward I think it will be really important for us to optimize performance further with it.     

2. The Future Of Hardware And Game Realism

Chris: Exciting, okay cool. Obviously you have a direct line to these hardware vendors. What do you want to see in the next generation of GPUs that’ll make your job easier? 

Johan: That's a fun question. We have a pretty long list of all the stuff we typically go through and then talk to them with, but one very concrete thing we’d like to see, and actually Intel has already done this on their hardware, they call it PixelSync, which is their method of synchronizing the graphics pipeline in a very efficient way on a per-pixel basis.You can do a lot of cool techniques with it such as order-independent transparency for hair rendering or for foliage rendering. And they can do programmable blending where you want to have full control over the blending instead of using the fixed-function units in the GPU. There’s a lot of cool components that can be enabled by such a programability primitive there and I would like to see AMD and Nvidia implement something similar to that as well. It's also very power-efficient and efficient overall on Intel's hardware, so I guess the challenge for Nvidia and AMD would be if they were able to do that efficiently because they have a lot of the different architectures there. So that's one thing. What other components do we have? Usually when the architects are over we have these meetings of just sitting and talking for 14, 15 hours, or an entire day about everything.

Chris: That'd be a fun conversation to sit in on.

Johan: Yeah, it's really fun. We want to enable, and I mentioned a bit about it last week during my talk last week about Mantle, was enabling the GPU to execute in a little bit more of a heterogeneous fashion of being able to run multiple compute shaders in parallel with your graphics work and ideally having more collaboration between the CPU and GPU. We can do things like that on the console because they're integrated machines, so the CPU and GPU are on the same die. On the PC you are seeing it more and more with the APUs and Intel's Ultrabooks that also have integrated graphics.

I want to see more of this type of collaboration between CPU and GPU to drive many more advanced rendering techniques. For example once we've rendered the Z-buffer for a scene then we know the depth of every single pixel that we have in that our frustum and based on that information we can actually do things like shadow maps that are adapted specifically only to cover the area that they actually need to. Typically you don’t really have that knowledge and on the CPU you prepare that data that the GPU will render a few frames later, so you have to brute force a lot of things. You have to send out a lot of work and you can't really be reactive. With many of the things that we can do with Mantle and I think going forward also with closer CPU and GPU interaction in general we can do a lot more clever techniques and less brute force type of techniques as well. That's a pretty frequent topic the when we talk with architects.

Chris: Sure, so I also want to know about features that'll make the biggest difference to realism, but in my previous question I was talking about features that'd make your job easier. So as a follow-up to that one, are there different features you want to see that'll improve the experience an end-user has when they play your games from the perspective of realism?

Johan: Yeah, so realism. I think that there's a few things, well I guess it goes in both categories.  Another thing that I haven’t mentioned yet is that Nvidia has been doing a lot of good work with nested data parallelism or I think they call it dynamic parallelism in their big Kepler cores where you actually run compute work that is sort of nested and can interact in very interesting ways. That enables a lot of other programability mechanisms with it and nice performance.

For realism specifically, we are having some challenges in general going forward because there are so many rendering techniques that we implement through just standard rasterization and post-processes. These things will sort of start to break down more and more as we have more and more complex scenes, and we want to have more transparent surfaces in those. Doing just the standard rasterization and then trying to do depth of field and motion blur correctly on those, but doing them as post-processes is very, very limited. Let’s say you have a transparency and you may have two or three layers of windows together with some particle effects there, and then you want to do depth of field on that scene afterwards, but you only have your depth buffer. It doesn’t know about these transparent surfaces, or if it knows about them it doesn’t know what's behind them.

I think a challenge there is figuring out what is a good, efficient future graphics pipeline both for mobile, which maybe have different constraints, but also for depth stuff, because the rasterization pipeline is really quite efficient, but has its limitations there. There’s various other types of alternatives such as micro triangle or micro polygon rasterizers or stochastic rasterization where you sort of can start in with depth of field and motion blur, and they can be more of an integral part of your rendering. This of course has a lot of other potential drawbacks or difficulties in how these things interact, but you sort of get to the point where more of these techniques can more freely interact. I think that can really bring on a lot more extra realism. At least I'm talking from purely what the GPU vendors and we can do together on that.

There's a lot stuff that we can do just with our engine also, and that we are doing going forward. Things like more physically based rendering and shading where we try and use more real life measurements of light sources and materials and try to represent those accurately within the game. Typically, in previous games and engines, we sort of look at the reference and then you try to recreate that, but you don’t really measure, you don’t know the ranges. There's nothing really to truly compare it with, so the type of games we're doing now with very big, complex content and levels of game play, it gets more important to be able to have that reference; the frame of reference of something that's real and trying to recreate that.

3. Preparing For A World Of VR And HMDs

Chris: Right, very cool. In that same forward-looking thread, is there any work that you’ve had to do to prepare or enable Frostbite games for this world that we’re coming up on that's eager to explore head-mounted displays and virtual reality. 

Johan: We’re sort of only started getting our feet wet there a little bit and trying out various types of scenarios within our games. There's a lot of people in our multiple game teams here at EA that are excited with VR and HMD type of devices. Also, a little bit, it's difficult to say exactly how our games would be playable with them. Some games fit very well and some you may have a very different type of game play experiences, but usually the experience overall of just exploring a battlefield world with a VR device is actually really quite compelling.

Chris: Yeah.

Johan: We have found a lot of parallelism on the CPU side, which helps a lot because you need to render two views and you need to render the absolute highest frame rate, but most importantly, at very, very low latency. We haven’t done too much work on the latency side, but we've done a lot of work on the performance side. Both in general in the engine, so it's not specific for VR, but it still tremendously benefits VR. I think that's sort of the main area we've been focusing on and we've been running tests with the various APIs and systems out there, just to sort of enable our content creators our game teams to be able to play with it and not be bottlenecked by, okay the engine does not support this so we can't try it, but now we actually have more components working with it. The game creators can try things out and think of the problems from there. There's a lot of difficult challenges, like what do you do with your first-person animation and things like that. We haven’t really thought about it too much yet. That's a lot of work just with your ordinary rendering, but VR it can be from any perspective.

Chris: Right.

Johan: We've been focusing on the engine side mostly just to enable the rest of the game teams to use these things and for focusing on optimization.

Chris: And so knowing the dependencies of those types of environments do you anticipate other developers that don’t have access to as scalable of an engine might encounter difficulty or compatibility issues with them?

Johan: Yeah, I think it will be difficult to just take a standard game that you have and then just bring it over. If you don’t have the performance sure you may be able to prove out like some of the core animation things, but perhaps not the full gameplay. There could be a little bit of a challenge there.

On the other hand, it can actually be a benefit if you're like a small indie developer and you're just starting from scratch and building something and targeting VR...I'm not sure if there are any games like that, there probably are. Just not having the sort of legacy of this big established ways of making games can probably be really quite refreshing there. Sure you could do things like that with Frostbite also, having just a big engine and then you do a very different type of game. We’ll see what we do in the future there. I think it will be quite exciting regardless.

4. The Future Of Frostbite: More Streamlined Development

Chris: Yeah, excellent.I believe Frostbite is in its seventh year or so right? Can you tease a little maybe of what we might see in the next evolution of the engine?

Johan: Yeah, sure. One of the things that we’re focusing quite a lot of effort on is actually not something that the consumers will see directly. It's more of an internal thing of how we sort of work with content, how we work with the engine, and we have so many game developers now. It's like 15 different game teams out there. There’s hundreds and hundreds and hundreds of game creators, artists, level designers, audio designers, and everything. One of the main things we’re focusing on is just streamlining work flows and making sure that they have really, really good iteration times. This has been a focus with Frostbite traditionally, and especially when we did Frostbite 2 we did a really big push on it. 

We’re sort of doing that again and that will hopefully make it a lot more efficient and in some cases even more fun for us to work because they will come to work and directly see the results. It's a big effort to improve in these areas. And I think end users, the way they will see this is that hopefully they'll play richer games and have more content and more variability. The stuff I mentioned with Levolution, that's something that sort of springs from this also. If you have content creators that can be really creative with the engine and work very well with it and you have a powerful core engine, well, then they can create some amazing scenarios there. I think you will see that in quite a few games going forward now we’re improving things even further here. 

Another thing is we’re doing quite a lot of work on rendering at its core also, things like physically based rendering and more realistic materials in our overall rendering pipeline we're working with, and just pure performance there also with Mantle will lead to some very cool games in the future. We’re not really constrained by what we can do today, but we sort of have a lot more to play with and enable new types of rendering techniques and solutions for some of our different games out there. That will be really interesting to see. 

The first games we launched were cross-generational games focusing both on current-generation consoles with Xbox 360 and PS3, but also next-generation consoles and PC. That was quite difficult to do, to sort of build a game that works on all of those different platforms.   

Chris: Yeah.

Johan: Some of our games going forward will only focus on the next-generation consoles and PC, and with the same engine, there's so much you can achieve if you can not only target that type of performance level. I'm really happy what we’re able to do now with the scaling in Battlefield, it's really quite great I think, but having the game designers think only about the next-generation consoles is really interesting and really awesome.   

5. Parallelism And Graphics On Next-Gen Consoles

Chris: Excellent, okay. A couple of times now you’ve talked about the scalability of Frostbite when it comes to threading in multi-core CPUs. Both of these next-gen consoles are heavily threaded, but the cores themselves are lighter than what we’re typically used to on the PC side. What are you doing to utilize those available x86 resources and how does that compare to the past development work that you’ve done?

Johan: We've spent a lot of time...we've been working on the next-gen consoles for around two years, and this last year we've sort of been focusing a lot on just bringing up everything and optimizing it also. It's been a bit of a challenge, the performance on the next-gen CPUs. But as our engine was quite parallel, we parallelized it even further. That was sort of a must for us to run with 64 players at 60 FPS on PS4 and Xbox One. There was a lot of work to parallelize everything. So now we've got around 90 to 95% of the CPU utilization and have some screen shots I can show you and it looks quite cool when you’ve got that type of utilization in multi-player.

Chris: Yeah.

Johan: So that's pretty cool, and the difference before was that, well, the PS3 was the bottleneck back in the day. Even though the Cell architecture was really powerful, just the work it took to move things off the SPU...and we went through, we spent a lot of time moving things to the SPU and saw some really great performance was there, but we also had to compensate a lot on GPU performance by moving things to the SPU. This took us a lot of time, and we ended up with quite good performance, but that sort of architecture was not very balanced back then.

Now, with the new generation consoles, the architecture is a lot more balanced and we got quite good performance from the get-go out of these CPUs. Then, we're able to spend a lot more time really fully utilizing them, as in parallelizing, and also doing other types of optimization on the CPU. We don’t have to optimize away every load-hit-store or cache miss because these are out-of-order processors so they handle more code in more of a general manner, which was really good. Although, they run at quite low clock frequencies, so it's still been a a challenge to make sure that we utilize all of them and that we essentially do now.

Going forward, I think we’ll start moving more code over to compute on the GPUs. Think like doing cloth simulation or parts of your rendering, you can do it in a little bit less brute-force way on the CPU and instead have the GPU handle that in a very efficient manner. So, that will be a bit of a challenge for us going forward to do that, but that's sort of the trajectory we're on more and more.

Chris: Okay, excellent. Simultaneously we’re dealing with these graphics engines that are not as capable really as the high-end PC hardware.  How close can you get to the PC experience at 1080p quality-wise on a console, and if you have to compromise what's the first thing to go?

Johan: Yeah, this is sort of a decision for each game team going forward that we work with, but before, getting to having 64 players and getting to 60 FPS was the most important thing because we wanted to bring this PC game experience that you have. And if you play Battlefield I think you'll agree that the actual game experience of playing on the 64-player server and everyone is interacting, I mean, and having a great frame rate is actually a significant difference from what we had on the current generation of consoles with only 24 players. This sort of enables more types of gameplay, and that was the most important thing for us. That sort of set the bar that we need to get there, and we did get there. Sure we did still have to do a little bit of a compromise on the solution. We're not running at the full native 1080p; we’re running a little bit lower resolution than that. But I think it was well worth those tradeoffs in order to make sure that we can actually have the full sort of PC game experience overall being on there. And you're playing the games in a little bit different way. You’re playing with the controller on a TV; you're not playing on a PC with a monitor. And it's very low-latency in that way where it's even more twitchy for example on a PC or a monitor that's even more sharp.

Chris: Right.

Johan: I think it worked out quite well for us, but it's sort of a decision for each game going forward also like what makes more sense. 

6. Fully Utilizing Next-Gen Console Hardware

Chris: Right. Are you rendering it to a different resolution natively on Xbox versus PlayStation 4?

Johan: Yeah, we’re rendering essentially to a separate render target and then up-sampling that to the native 1080p and then we’re rendering the UI on top of that in 1080p, which actually helps quite a lot because there's a lot of text and small details in the UI on BF.

Chris: Got you. I know that developers took a while before they were really able to utilize PlayStation 3’s architecture. Given the composition of these next-gen consoles is it less challenging for you to fully utilize them and is there a lot of untapped potential still, or are you guys taking advantage of most of that right out of the gate?

Johan: Definitely it was a lot easier this generation and a lot quicker. I think you're going to see that on the games that we and other people launch that they actually are, for launch titles usually so early in the generation, the first title that you have they typically are not great titles, or perhaps the actual games are not that great but they look okay. Here we got off from an extremely good start I think and we've spent a lot of time working on that to make sure that we got to that point, but there is still a lot of things about next-gen consoles that we can specifically optimize and utilize. Things like I mentioned with the async compute or things with how we tweak for the CPUs or what are the exact shaders and settings that we’re using for our graphics rendering. There are still some, quite a lot of untapped potential in the consoles there that we’ll be utilizing for the upcoming games. 

Chris: Okay, cool. I know you mentioned that we were going to start getting to the point where we could talk about performance benefits of Mantle in Battlefield 4. Are we there yet or do we need to keep waiting a little bit longer? 

Johan: (Laughs) Yeah, we’re not there yet.

Chris: Okay.

Johan: It's a lot of work just putting all of the pieces together and performance is the thing you get last there. I don’t want to give any numbers that are not fully representative yet either.

Chris: Sure. 

Johan: Yeah, we’re not there yet.

7. What Does A World With Many Low-Level APIs Look Like?

Chris: Let’s do a quick hypothetical. Nvidia sees that developers are starting to pick up on Mantle, and they announced their own close-to-metal API. And you know, Intel sees that it's got a lot of market share on the integrated side especially and it follows suit. What happens and who wins out of that?

Johan: Yeah, that's an interesting question. Of course, I think one of the most important things is that, and we have talked a bit on it at the conference last week, and that is that Mantle, even though it's work being done by AMD to expose the stuff on their specific architecture, and it's worked together with us also on the stuff we've learned from the next-gen consoles and just in general with our engine architecture. But it's not an API that is designed only for their specific GPUs. This actually is a thin and low-level abstraction over the hardware there and it has to play nicely with everything that you have in Windows also. I would like to see other vendors supporting that. It’s a bit of a tall order and of course everything still comes from AMD here now initially.

But I think it's important to say that there's really no technical reason that you need multiple sort of low-level graphics APIs, one for each vendor. If that would happen, that would be for political reasons that you would have multiple ones, that they can't all play nicely together or they're not able to find a way of collaborating on this. That would be unfortunate I think. Although, that said, at least in the desktop space, there are only three players out there, and we already support...I don't even know how many rendering back-ends we do have. It's five, six, seven back-ends or something like that now?

Chris: Mm-hmm.

Johan: For us supporting additional low-level graphic APIs, well, once you drop to that lower level it's not as difficult to support additional ones especially if you already have some. That's not the way I would like to see it going forward. I would like to see that, first of all, that we prove out that Mantle actually works and that we deliver great improvements with it and we have a good trajectory with that. That’s the plan that I would be for and upcoming titles and do that together with AMD and get it out there. Then, go wider with that, and have these potentially very tough discussions with the various IHVs and see if we can find a way of essentially standardizing, whether that is just Mantle or whether that is Mantle as part of some, I don’t know, foundation or standards body or if there are actually other APIs in the future that sort of bring on the same benefits and improvements. That I don’t know yet. Although Mantle is a really good design I think. It is very well executed for all of these things. But there will be some tough discussions that we’ll have going forward, all of us together.

Chris: Yeah and I think we’re so cynical because of how long we've been doing this, but it's so rare to see these companies collaborate on a cooperative basis. Can Mantle’s same goals be achieved through an industry-standard API, and could DX be made to achieve similar things?

Johan: I think they can go more and more in that direction. I'm not fully sure; there's also the question of time, when you're able to actually, even if you want to go in that direction, when can you deliver that and get everyone on-board and things like that. That’s a pretty important component. Let’s say if it was only DX that went sort of closer to Mantle, that would definitely be a good thing. But the rest of ecosystem is based on OpenGL, so ideally you want OpenGL to move in that direction as well, or OpenGL to become something more like Mantle on the lower level.

I think that's really important because the future is not just DX. Sure, it's pretty dominating on the desktop, and we use a DX variant on the Xbox One, but we don’t use it on PS4, we don’t use it on PS3, we don’t use it on iOS, we don’t use it on Android, we don’t use it on Mac, we don’t use it on Linux. Many of these other platforms are really important also, and that's where, I think, ideally I really want to have a cross-platform API, so it's not tied to DX, even though DX is today the most important API out there and the one that most of our users are using by far.

Chris: Right.

Johan: Ideally I would like to see a solution that can work on Mac and Linux, and in the future also on the mobile platforms that are moving forward a lot, and are even more constraining about CPU performance and efficient rendering and power efficiency and all that.

Chris: For all those other platforms you're looking to OpenGL and OpenGL ES then?

Johan: Yes, exactly. Microsoft also, of course, they have DirectX working on Windows 8 tablets and Windows Phone, but they have a tiny, tiny market share there. Microsoft definitely does not have a history of sharing APIs with the wider industry and things like that, even less than the IHVs have.

Chris: Right.

Johan: IHVs have a better chance of working together with us developers and seeing if we can have an overall cross-platform solution. A little bit difficult challenge overall, but we have to push for what we would like to see out there. 

8. What Are The Chances That AMD Shares Mantle With Anyone?

Chris: In the multi-vendor slide from APU13, you mention that Mantle isn’t tied to GCN. Right now, the API is GCN-specific. What would it take to expose its functionality on other GPUs then? Can this be done now, or would AMD need to make a decision to open it up? There’s a bullet that says forward-compatible, but can it also be made backward-compatible to prior/existing architectures?

Johan: Mantle requires a certain set of key functionality of the GPU, so it can’t be supported on older architectures before AMD’s GCN architecture. The focus right now for Mantle is to finish the implementation of the first version of it, and get it out together with a Mantle-based rendering back-end in Battlefield 4. This back-end will only be able to run on AMD’s GCN-based GPUs. Again, going forward I would like to see a future version of Mantle support GPUs from more vendors, and that would be a discussion that would have to happen between AMD, the other vendors, and us developers to figure out a way to make it happen.

Chris: Very cool. Okay, so I’ll just take us back to beginning; now that we've got Battlefield 4, and of course we want to do the best job possible of reflecting real-world performance on the hardware platforms we're testing. There's so much we can do with that on the single-player. I know you were always a big proponent of testing multi-player for the experience that's more persistent and probably more popular once everybody gets done. Is there a better way to do that yet in Battlefield 4? Do we have a solution for consistent multi-player testing?

Johan: Unfortunately not. We do have some internal tools that sort of records parts of the network stream that we use and then play that back and so you sort of play back multi-player footage, which is quite good. But it's not really packaged together in a sense that we can use it in the retail game unfortunately. It's also not fully true either because it's skipping quite a few things, actually, that you're doing in a full multi-player scenario there. Multi-player, unfortunately, as I say it’s rather chaotic, but it is the truer thing to see the real performance when you especially when it comes to CPU performance. GPU performance you can see pretty well with the single-player. But CPU performance is primarily multi-player. We don’t really have a great solution there unfortunately.

I think one of the best things one can do for benchmarking, but it's very tiresome for you guys, but that is to essentially play multi-player on the same level and play it similar ways. 

Chris: Right.

Johan: Then spend quite a lot of time playing it actually, but it becomes, yeah, not a very exact science unfortunately.

Chris: Yeah, difficult to quantify for sure. 

Johan: Yeah.

Chris: Johan, I again appreciate your time.

Johan:  Yeah, no problem.