Hi storage nerds,
I'm getting deeper and deeper into the rabbit hole of storage, searching for the ultimate solution.
My problem:
*small* 500GB SSD in notebook
4TB ext. drive
8TB NAS
desktop PC
2nd old notebook
smartphone
camera
cloud storage with onedrive (1TB), dropbox, gdrive, ...
and lots of data of different sizes (2TB personal videos, 3TB photos, work projects, personal documents etc).
It all started with a simple thing: Synchronize some photos in my 1TB OneDrive to my SSD, but all to the exHDD. Therefore, I would need to run two instances of OneDrive with different settings. Further thinking got me deeper into the mud...
... now I want to access this data always in the same location (say drive letter H: on windows), which aggregates all my data onto one virtual "user documents" drive. New photos usually reside on the SSD, while all the old bloat might be only on NAS, exHDD, cloud, web backup due to storage limitations.
I still want to open my video editing project from 2010 without having to search for all the linked video and audio files, open my photo library and it has all the ratings, soft edits etc. saved for the pictures in their original location (drive letter H: ).
This means that there has to be an abstraction layer that aggregates the data from all locations onto one new location. Without much practical experience, afaik this could be done with mklink. This would load "H:\photos\2018 blub" from the SSD, while "H:\photos\2012" would load from the NAS or the ex.HDD, depending on what's available, most recent and fastest. Fresh from the press, you might load the photos from today straight from your smartphone (requires at least a synced file directory).
Next, the locations should be selectively synchronized whenever they are accessible. ex.HDD and SSD sync directly via USB, NAS via LAN/WiFi, cloud via internet, etc. Syncthing and git come to mind, but this seems to be a bit bigger task. Nextcloud or seafile might be a good starting point, but are centralized.
TL;DR
aggregate all available storage (SSD, exHDD, NAS, cloud, smartphone, ...) to one virtual folder (symlink, mklink)
synchronize decentralized between locations (syncthing, git)
Has anyone solved such a problem before? I don't really want to frickle around myself, but have not found a professional solution that would do the job. Any ideas where to start?
I'm getting deeper and deeper into the rabbit hole of storage, searching for the ultimate solution.
My problem:
*small* 500GB SSD in notebook
4TB ext. drive
8TB NAS
desktop PC
2nd old notebook
smartphone
camera
cloud storage with onedrive (1TB), dropbox, gdrive, ...
and lots of data of different sizes (2TB personal videos, 3TB photos, work projects, personal documents etc).
It all started with a simple thing: Synchronize some photos in my 1TB OneDrive to my SSD, but all to the exHDD. Therefore, I would need to run two instances of OneDrive with different settings. Further thinking got me deeper into the mud...
... now I want to access this data always in the same location (say drive letter H: on windows), which aggregates all my data onto one virtual "user documents" drive. New photos usually reside on the SSD, while all the old bloat might be only on NAS, exHDD, cloud, web backup due to storage limitations.
I still want to open my video editing project from 2010 without having to search for all the linked video and audio files, open my photo library and it has all the ratings, soft edits etc. saved for the pictures in their original location (drive letter H: ).
This means that there has to be an abstraction layer that aggregates the data from all locations onto one new location. Without much practical experience, afaik this could be done with mklink. This would load "H:\photos\2018 blub" from the SSD, while "H:\photos\2012" would load from the NAS or the ex.HDD, depending on what's available, most recent and fastest. Fresh from the press, you might load the photos from today straight from your smartphone (requires at least a synced file directory).
Next, the locations should be selectively synchronized whenever they are accessible. ex.HDD and SSD sync directly via USB, NAS via LAN/WiFi, cloud via internet, etc. Syncthing and git come to mind, but this seems to be a bit bigger task. Nextcloud or seafile might be a good starting point, but are centralized.
TL;DR
aggregate all available storage (SSD, exHDD, NAS, cloud, smartphone, ...) to one virtual folder (symlink, mklink)
synchronize decentralized between locations (syncthing, git)
Has anyone solved such a problem before? I don't really want to frickle around myself, but have not found a professional solution that would do the job. Any ideas where to start?
Facebook
Twitter
Instagram