i didn't like how it lacks 1 thing media player had already, it lacked a toolbar option which just seemed odd. If you reduced Groove it also stopped playing - i believe that has been fixed since i stopped using it.
I think right click doesn't keep it off, its a built in app and if you remove them, they just come back. The Powershell option might work, never tried it.
I too use Foobar, nice if it had some skins or maybe i haven't looked hard enough.