"It's not how much and how fast your memory is, it's what you do with it."
HBM2 is supposed to be able to support up to 16GB of memory, I think. So that would top most of Nvidia's Geforce cards. Though they already have HBM2 equipped cards of their own (Tesla models).
Since Nvidia seems to be pursuing the cheaper path of even faster GDDR5X 11 and 12 Ghz stuff, it may be a wash, with their cards already around 500 GB/s of bandwidth, and a potential HBM2 at 720 GB/s it isn't as huge an increase as it would seem.
AMD's cards are likely to have the same smaller footprint of the Fury line though, and I wouldn't be surprised to see more OEM water cooling.