Theoretically yes, any system that is network-connected to your PC can interface with it.
Practically, that would be hard to do if you have the appropriate measures in place, have not installed software that allows remote access, have a firewall in place etc. Also, a properly configured system should give you a warning when a remote desktop application is running.
Finally, if the ISP would try to do that, there would be more than one PC affected, impossible for them to pull such a scheme for long without being discovered. Needless to say, that would be the end for them.