Yes, it is the application (game) that will spell out its minimum video card memory requirements.
Often a card will work fairly well with less that the recommended memory on board, but at the price of the game experience, usually with a drop in frame rates or with lower video quality or resolution settings.
That said, there are very few titles that want more than 2GB.. One of the notable exceptions being BF4 on ultra. I'm happy with BF4 on my 1GB video card.