If you're only running a single 1080 monitor and have no future plans on upgrading then 2 GB should be fine. Only situation I've heard a few people claim is playing a Heavily modded Skyrim game using a lot of custom textures at max settings was too much for 2 GB of VRAM, but worked fine with 4 GB.
As a side note I do agree with mcnumpty23's comment that a GTX 670 is very close to a GTX 680 in performance. I'd save the extra money and run with the GTX 670.
Good luck with whatever you decide.