The device (I won't say PC for now) that is providing the display and keyboard requires CPU, RAM, GPU and ROM. The ROM takes the place of a HDD/SSD to hold the code that allows the device to access the remote service. If you look at HP or Dell thin clients they have enough ROM to hold a basic OS which can connect to a network and knows specific network protocols. You can take a conventional PC and use the PXE boot capability to read an OS from a remote server and have it function similar to a thin client. It will function without local storage.