As you found QoS is mostly a worthless feature.
Yes you can really only control the UPLOAD traffic. In most cases your upload stream is not the one that is overloaded it is your download. The problem with the download is that when the ISP detects the connection is full it randomly throws away traffic. Not much your router can do the traffic is gone way before it even gets close to your house.
Now some routers have the ability to put hard limits on bandwidth both up and down. Which is really nice if you actually have a problem with your upload bandwidth and need to limit some traffic. The download limitation on its face seems stupid. My link is overloaded so the ISP is dropping random data...so I get mad and drop even more of the youtube traffic that ISP does not drop. That does not magically undrop the traffic the ISP dropped. The only reason this appears to work is some application like youtube are smart enough to detect the traffic loss and slow down. Because they are now transmitting less data to try to avoid the dropping the total utilization should less and the ISP can stop dropping traffic.
The only flaw with this concept is the application must be able to detect the traffic loss and take action to reduce it bandwidth. If the application is designed to send 2m/sec no matter what you do then QoS inbound limits will not solve the issue.