Definitely go for 4GB.
At this point, a vast majority of games go past 2GB in VRAM usage.
With 4GB, it is difficult to tell. At 1080p, it really should not be an issue. With AC Unity, the VRAM usage caps off at around 3.8 GB. This cuts it fairly close, but for whatever reason (lack of optimisation as usual), the VRAM usage does not seem to increase when running at 1440p or 4K.
At 4K, a few games do seem to go over it. That is extremely undesirable, but a single 980 will not perform optimally at 4K anyways. With DirectX12, VRAM stacking with SLI will become possible, albeit at the whim of developers.
I'm no CoD fan, but Advanced Warfare does use an unusual amount of VRAM for the quality. I believe with High/Ultra textures, it can use upwards of 3GB at 1080p.
Shadow of Mordor with the downloadable "Ultra" texture pack *is* recommended for Graphics Cards with 6GB or more of VRAM. I assume this was meant at 4K, because I've heard conflicting reports about actual VRAM usage. According to some (I've yet to buy and test the game out), it is manageable with 3GB of VRAM.