Sign in with
Sign up | Sign in
Your question
Closed

RAMCloud: The Idea of Storing All Data in RAM

Tags:
Last response: in News comments
Share
November 11, 2011 6:15:34 PM

No really? This can be done for a consumer, as well. Imagine having your entire OS in the RAM? An X79 board has what, 8 RAM slots? Throw 8GB into each one and you get 64GB, that's more than enough for a full Windows installation and some programs. It will be blazing fast.
Score
22
November 11, 2011 6:22:25 PM

i really don't see an use for this type of things...although it will be nice if you are a Minecraft maniac...but in real life task even more than 8gb for now is very unnecessary and stupidly excessive.
Score
-21
Related resources
November 11, 2011 6:26:47 PM

And how are you planning to get around the volatility problem on your home rig sir amk-aka-Pantom?
Score
-11
November 11, 2011 6:29:48 PM

Sounds awesome - until the power goes out.

;(
Score
24
November 11, 2011 6:33:46 PM

ltdementhiali really don't see an use for this type of things...although it will be nice if you are a Minecraft maniac...but in real life task even more than 8gb for now is very unnecessary and stupidly excessive.


Prices keep going down. RAM is dirt cheap; 8GB costs $50 (2x4GB), the new 8GB chips are costly but prices WILL fall, and having your stuff in the RAM is awesome. And for servers, it's even better. Did you read this:

Quote:
The researchers estimate that a single multi-core RAM server could support at least 1,000,000 small requests per second, while disk based systems are typically maxed out at 1000 to 10,000 requests.


That is, bye-bye to DDOS.
Score
12
November 11, 2011 6:35:50 PM

NetherscourgeSounds awesome - until the power goes out.;(


UPS for you :D  Mine can keep my gaming rig, my lights, fans (no, not the case cooling, big ceiling fans) and so on running for 2 days straight.

blood_dewAnd how are you planning to get around the volatility problem on your home rig sir amk-aka-Phantom?


Elaborate, please!
Score
2
November 11, 2011 6:39:22 PM

ltdementhiali really don't see an use for this type of things...although it will be nice if you are a Minecraft maniac...but in real life task even more than 8gb for now is very unnecessary and stupidly excessive.

more than 8GB is recommended if you do some CS5.5 or 3D uses... 8GB is need for gaming system... Your all about the games instead of what's out there in the internet that you needa experience...
Score
2
November 11, 2011 6:45:02 PM

blood_dewAnd how are you planning to get around the volatility problem on your home rig sir amk-aka-Pantom?

There is non-volatile ram in development. Not sure if it will ever become a real product, but it's there.
Score
1
November 11, 2011 6:46:31 PM

amk-aka-Phantom - after you use all that 64gb of ram for your os and some programs, what ram do you have left to actually run them? sheesh
Score
-4
November 11, 2011 7:02:15 PM

ok so what happens if the computer turns off? screw all of you!!! your information is gone lol
Score
-6
November 11, 2011 7:03:19 PM

HaserathThere is non-volatile ram in development. Not sure if it will ever become a real product, but it's there.

I thought SSDs use non-volatile NAND RAM? or is that NAND flash? :\

Why do they call it a "could" system? Isn't this just a ram-based server? Cloud is like, for storing data and things right? "Run your shitz of the cloud". I can't see the point of it for cloud storage, would be too expensive. What's the point of the super-low latency if your ping to the RamCloud will be 200ms? A movie streams a ms late, big deal...

If it's used for regular server hosting (like game servers, etc) then i can see the point, because computation time would reduce.
Score
2
November 11, 2011 7:04:47 PM

LegendKillermore than 8GB is recommended if you do some CS5.5 or 3D uses... 8GB is need for gaming system... Your all about the games instead of what's out there in the internet that you needa experience...



Also nice if you have a couple VMs running. My board is maxed out at 16.
Score
1
November 11, 2011 7:07:08 PM

BTW does anyone else notice a CPU usage spike when you scroll from the article to the comments? Happens somewhere at the end of the article...been noticing this for a while.

Use chrome, 32-bit, 3.2GB RAM and a Core 2 Quad. Tried on a C2D laptop and happens there too, had used chrome again.

EDIT: Just tried with IE, spike happens but you cant notice it. Chrome actually stutters a bit...
Score
6
November 11, 2011 7:08:15 PM

ojas said:
I thought SSDs use non-volatile NAND RAM? or is that NAND flash? :\

Why do they call it a "could" system? Isn't this just a ram-based server? Cloud is like, for storing data and things right? "Run your shitz of the cloud". I can't see the point of it for cloud storage, would be too expensive. What's the point of the super-low latency if your ping to the RamCloud will be 200ms? A movie streams a ms late, big deal...

If it's used for regular server hosting (like game servers, etc) then i can see the point, because computation time would reduce.


yeah if you are the ONLY one using it then the timing difference is almost nothing compared to the ping, but if you have hundreds of thousands of users accessing the data HDD's and even SSD's wouldn't be able to keep up, that is where the Ram Cloud would come in
Score
2
November 11, 2011 7:09:08 PM

viometrix said:
amk-aka-Phantom - after you use all that 64gb of ram for your os and some programs, what ram do you have left to actually run them? sheesh


About 29GB. My current OS/programs partition takes 35GB. U mad?
Score
11
November 11, 2011 7:10:21 PM

mindless728yeah if you are the ONLY one using it then the timing difference is almost nothing compared to the ping, but if you have hundreds of thousands of users accessing the data HDD's and even SSD's wouldn't be able to keep up, that is where the Ram Cloud would come in

Ah...didn't think of that!
Score
1
Anonymous
November 11, 2011 7:39:34 PM

It's called a RAM-drive, and this is nothing new. You can do this on any desktop machine if you want, it's just usually not worth it due to power interruptions. Still fun nonetheless.
Score
7
November 11, 2011 8:09:59 PM

You lost me at cloud
Score
8
November 11, 2011 8:12:54 PM

Not sure if the researchers have taken into consideration the added cost from having to use only buffered ECC RAM. The unbuffered non-ECC RAM some of the commenters here are referring to is considered volatile for more than just what happens when the power turns off. After some time the data stored in this RAM becomes corrupted and would no longer be the same data. Ever wonder why some programs after being open and running for several days can sometimes start to act funny after a while?

For the past 20 years or so OS's have used redundancy to help with this issue via the swap file on the HDD. Using buffered ECC RAM can eliminate about 99% of this problem, but can cost 2-3 times as much in some cases. Although, I suppose you could offset the cost by using twice as much "regular" RAM and using half for redundancy. Still, the best option would be to store the data on a typical storage device and have a newer better buffering system which keeps and accurate and updated copy in the RAM "cloud" which could keep the data integrity in check. This way there would only be the older latency issue initially when the data is being uploaded or altered, or when it needs to be rebuffered because of corruption. Plus that thing about the power going out would be a non issue.
Score
3
Anonymous
November 11, 2011 9:21:38 PM

This makes zero sense for a home user. But makes a lot of sense in some situations. Master server that writes results to disk, with many slaves that only have ram for instance.

I can see this ram cloud service now . "Tech support: how can i help you", "you: all my files are missing", "tech: ya we had a power failure, sorry everything is stored in ram, all data is gone", "you: excuse me?"
Score
0
Anonymous
November 11, 2011 9:39:05 PM

If you went for a massive RAM drive like this why wouldn't you couple it with tiers of other storrage? SSD's and a large pool of harddrives? I don't know the optimal combination, having some parts work as cache for others.
Score
0
November 11, 2011 9:53:04 PM

My second computer was a Leading Edge 286 with a VGA color monitor, about $2 grand. I don't remember the specs except for the gigantic 65MB HD. My impossible dream was having a 640MB HD and a 640MB ram drive and leave the machine turned on all the time and keep the hard drive mirrored on the Ram drive. I still miss the Leading Edge clicker keyboard.
Score
1
November 11, 2011 10:02:04 PM

there are ways to power ram, to get over the volatile problem, expensive as hell last time i looked into them though.
Score
-1
November 11, 2011 10:47:31 PM

ltdementhiali really don't see an use for this type of things...although it will be nice if you are a Minecraft maniac...but in real life task even more than 8gb for now is very unnecessary and stupidly excessive.

You don't even understand what this article is about and I don't care to enlighten you, just one clue it's not about consumer level use.
Score
5
November 11, 2011 11:01:18 PM

ojasI thought SSDs use non-volatile NAND RAM? or is that NAND flash? :\Why do they call it a "could" system? Isn't this just a ram-based server? Cloud is like, for storing data and things right? "Run your shitz of the cloud". I can't see the point of it for cloud storage, would be too expensive. What's the point of the super-low latency if your ping to the RamCloud will be 200ms? A movie streams a ms late, big deal...If it's used for regular server hosting (like game servers, etc) then i can see the point, because computation time would reduce.

NAND RAM is much slower than DRAM and can wear out after certain number of writes and erase cycles.

Yes someone's ping could be 200 due to internet connection, but usage of RAM for storage could allow that 100-1000X people have 200ms ping where HDD/SSD allows for X people to have.
Score
1
November 11, 2011 11:04:33 PM

ThisIsMeNot sure if the researchers have taken into consideration the added cost from having to use only buffered ECC RAM. The unbuffered non-ECC RAM some of the commenters here are referring to is considered volatile for more than just what happens when the power turns off. After some time the data stored in this RAM becomes corrupted and would no longer be the same data. Ever wonder why some programs after being open and running for several days can sometimes start to act funny after a while?For the past 20 years or so OS's have used redundancy to help with this issue via the swap file on the HDD. Using buffered ECC RAM can eliminate about 99% of this problem, but can cost 2-3 times as much in some cases. Although, I suppose you could offset the cost by using twice as much "regular" RAM and using half for redundancy. Still, the best option would be to store the data on a typical storage device and have a newer better buffering system which keeps and accurate and updated copy in the RAM "cloud" which could keep the data integrity in check. This way there would only be the older latency issue initially when the data is being uploaded or altered, or when it needs to be rebuffered because of corruption. Plus that thing about the power going out would be a non issue.

Or you can, as researches did, use regular HDDs for backup and redundancy.
Score
0
November 11, 2011 11:14:15 PM

ojasBTW does anyone else notice a CPU usage spike when you scroll from the article to the comments? Happens somewhere at the end of the article...been noticing this for a while.Use chrome, 32-bit, 3.2GB RAM and a Core 2 Quad. Tried on a C2D laptop and happens there too, had used chrome again.EDIT: Just tried with IE, spike happens but you cant notice it. Chrome actually stutters a bit...


Tested it out.. you're right.
Score
0
November 11, 2011 11:14:40 PM

NetherscourgeSounds awesome - until the power goes out.;(

You are 100% correct. No UPS lasts forever. Even with generators all it takes is a single datacenter to have an outage just a little greater than their backup power. Just ask RIM.

I'ts a horrible Idea if you not constantly replicating to disks, wich will probably not keep up.
Score
-1
November 11, 2011 11:15:17 PM

Purely in-memory datastores, like memcached and redis, have been around for awhile. MemcacheDB and RedisCluster (still in progress) are projects which manage large clusters running these services. Redis and memcached were both benchmarked handling in excess of 100k reads/writes per second on consumer grade laptops a few years ago. Redis can even snapshot to disk fairly frequently so that it isn't as volatile.

It's great to see more people talking about this as a viable technology, but these researchers are a few years behind the times. The algorithms for making large clusters handle failing nodes are in wide use for disk based storage systems, the software for memory-only storage services are in common use, combinations there of, as mentioned above, are already being built and have been talked about for years.
Score
2
November 11, 2011 11:24:09 PM

ojasBTW does anyone else notice a CPU usage spike when you scroll from the article to the comments? Happens somewhere at the end of the article...been noticing this for a while.Use chrome, 32-bit, 3.2GB RAM and a Core 2 Quad. Tried on a C2D laptop and happens there too, had used chrome again.EDIT: Just tried with IE, spike happens but you cant notice it. Chrome actually stutters a bit...

Nope. Running the latest version of Chrome, an i5 750, 8GB RAM, Win7 64-bit, and a geforce 550ti. I looked while scrolling down and the CPU usage was only at 3-5%.
Score
0
Anonymous
November 11, 2011 11:25:47 PM

Gawd, this is 'tarded. Smart server applications cache everything they need in memory anyways this is like something a 14 y/o would think of, I'd hate to think of the "research" that went into this idea.
Score
2
November 11, 2011 11:30:13 PM

My Clan is using Ram Disks already for our Minecraft Server. World Saves and Map generations are quick. Though I have no problems already with current cloud services.
Score
-1
November 12, 2011 12:33:14 AM

This sounds great I wonder why it's never been done before .... oh wait, it has. Flashback to the RamDisks of the 90s (sans LIM card).
Score
0
November 12, 2011 2:55:18 AM

Yeah somone correct me if im wrong, but i beleive you can already partion some of your ram as a useable drive. I'm not 100% on the specifics (ie does it simply load pre defined programs into an alloted space or does it automatically load the things causing most dely intell SSD boost style) but honestly, realising the throughput of ram is higher than that of SSD's is all theyve done here. WOW. Go Stanford. . . . .
As for volatility issue the obvious is to have an SSD storing the data when powers off and transfer that data to the RAM drive on boot.
So yeah. Not impressed.
Score
0
November 12, 2011 3:40:59 AM

*Power goes out* - That's It boys - It was nice working with you, but the company is now over.
Score
2
November 12, 2011 7:17:12 AM

Not much different from a regular server with large amounts of RAM!
Score
0
November 12, 2011 8:34:23 AM

ram cloud! me like.
why not use something like capacitors in the ram modules or high speed flash memory to store data in case of outage and use ssd's to cache the data and then store it on a disk drive as back up. only problem(latency/lag) with that would be to bring the data back from hdd to the ram. from fastest(main) to slowest(storage/backup): ram > flash cache > ssd cache > hdd(storage/backup).
non volatile ram would be awesome too.
Score
0
November 12, 2011 9:26:05 AM

livebriandNope. Running the latest version of Chrome, an i5 750, 8GB RAM, Win7 64-bit, and a geforce 550ti. I looked while scrolling down and the CPU usage was only at 3-5%.


Not saying it's a chrome prob, i think it's a web page prob.

And your PC is pretty good, maybe it can take it? I dunno...try scrolling up and down across the entire page repeatedly, very fast.
Score
2
November 12, 2011 9:34:43 AM

Cool stuff.
Score
0
November 12, 2011 3:03:42 PM

used to setup a Ramdrive back in the old DOS days to speed things up by putting the temp dir and swap file there. i assume windows is doing this automatically in the background with the system cache. and hard drives have their buffer caches too. databases use a lot of ram to as data buffers too. but sometimes i wonder if there is more we could do to manage all the ram we have on most new systems.
Score
0
November 12, 2011 3:11:59 PM

ojas said:
BTW does anyone else notice a CPU usage spike when you scroll from the article to the comments? Happens somewhere at the end of the article...been noticing this for a while.

Use chrome, 32-bit, 3.2GB RAM and a Core 2 Quad. Tried on a C2D laptop and happens there too, had used chrome again.

EDIT: Just tried with IE, spike happens but you cant notice it. Chrome actually stutters a bit...


Well, I thought that was normal for most of the pages with heavy content, like facebook, and even hotmail (but I think that one has real problems),
My processor is a Phenom II X4 running in XP limited to 4GB RAM (limited to 3.3) in dual channel at 1600 cas 7 , and I experience this stuttering while using CnQ almost in any page, but when disabled or using Phenom MSR Tweaker or K10Stat for custom CnQ, it disappear.

Now my chrome version is an old developer 12.0.712.0 unknown (with 120% zoom, at 1080p, with a lot of extensions including Smoothscroll, and AdBlock), my processor is overclocked to 3.7 but it just jump to 2.7 when scrolling TH pages with a usage of no more than 20%.

The page have experienced some changes recently so maybe that is why has become a little heavier, but still think is maybe due to your processor.
Score
0
November 12, 2011 4:40:11 PM

They could also title this article "The powers that be REALLY REALLY want to get your data away from you!" ...so lets create this near sounding scenario that cures all the ills of storage on your PC.

SSD has latency problems? . LOL
Score
0
November 12, 2011 5:10:48 PM

Why this would be creepily awesome for the consumer, performance-wise, the cost of a 128GB RAM drive would seemingly not be worth even the spectacular performance. I want one.
Score
0
November 13, 2011 1:58:02 AM

In contrast to many of the above posts i think the article was more about the size of the RAM drive then that they were using a RAM drive as a RAM drive is nothing new but a 64TB RAM drive is. As stated above many people already use RAM drives for their severs and thats good and all just they made a storage sever RAM drive which i hope they are keeping backed up as a single power failure to it would be a massive loss of information.
Score
0
November 13, 2011 3:10:18 AM

Not exactly new news. :)  I remember being able to store a small Mac OS install in memory on an old Powerbook. I also remember RAMdisks for DOS. I seem to recall one that would let you take any memory DOS couldn't address at the time and use it for blazing fast program storage.
Score
0
November 13, 2011 4:26:53 PM

KrntWell, I thought that was normal for most of the pages with heavy content, like facebook, and even hotmail (but I think that one has real problems),My processor is a Phenom II X4 running in XP limited to 4GB RAM (limited to 3.3) in dual channel at 1600 cas 7 , and I experience this stuttering while using CnQ almost in any page, but when disabled or using Phenom MSR Tweaker or K10Stat for custom CnQ, it disappear.Now my chrome version is an old developer 12.0.712.0 unknown (with 120% zoom, at 1080p, with a lot of extensions including Smoothscroll, and AdBlock), my processor is overclocked to 3.7 but it just jump to 2.7 when scrolling TH pages with a usage of no more than 20%.The page have experienced some changes recently so maybe that is why has become a little heavier, but still think is maybe due to your processor.


well, i've the latest version of chrome, 1024x768, ABP running too....and as i said, usage spikes happen with IE too, but there's no stuttering. It's unlikely to be a browser issue.

This has been happening for quite some time, before the changes. Yes, usage hits 22% when i rapidly scroll up and down, but that shouldn't happen i suppose? Compared to Wikipedia, i dont see any spike till i start scrolling rapidly and repeatedly, and even then the usage varies from 8% to 20% at the most. On chrome. Plus my mouse contributes a bit to the usage as well. (800 dpi, USB, i use Logitech's SetPoint). But that still doesn't account for that spike during normal scrolling, as i use the side buttons on the mouse.

May be my proc, or maybe CnQ and SpeedStep... But it doesn't make sense, why does it happen just at a particular point? There's something up with the code after the article ends, imo.

I was hoping that someone from the site could check it out, that's all...which is why i put it up here, wanted to confirm that it happens at all...

BTW, WHAT jumps from 3.7 to 2.7? Clock speeds? Shouldn't it be the other way around? i'm unable to follow you there...it's overclocked, but then it underclocks itself on TH pages, and CPU usage jumps to 20% (which is the problem)?
Score
0
November 13, 2011 10:05:57 PM

ojas said:
well, i've the latest version of chrome, 1024x768, ABP running too....and as i said, usage spikes happen with IE too, but there's no stuttering. It's unlikely to be a browser issue.

This has been happening for quite some time, before the changes. Yes, usage hits 22% when i rapidly scroll up and down, but that shouldn't happen i suppose? Compared to Wikipedia, i dont see any spike till i start scrolling rapidly and repeatedly, and even then the usage varies from 8% to 20% at the most. On chrome. Plus my mouse contributes a bit to the usage as well. (800 dpi, USB, i use Logitech's SetPoint). But that still doesn't account for that spike during normal scrolling, as i use the side buttons on the mouse.

May be my proc, or maybe CnQ and SpeedStep... But it doesn't make sense, why does it happen just at a particular point? There's something up with the code after the article ends, imo.

I was hoping that someone from the site could check it out, that's all...which is why i put it up here, wanted to confirm that it happens at all...

BTW, WHAT jumps from 3.7 to 2.7? Clock speeds? Shouldn't it be the other way around? i'm unable to follow you there...it's overclocked, but then it underclocks itself on TH pages, and CPU usage jumps to 20% (which is the problem)?


Hmm... I tried to notice the stuttering in some heavier pages (newegg, modDB, devianart, TechpowerUP) with no success, then I returned to TH and now I notice it, maybe because now I opened a newer article that wasn't in the cache, I'm not sure why I didn't notice it before if it was present (maybe because I am a little accustomed to some slower machines lately), it happens just before showing the comments section (maybe the comments are loaded while scrolling).

Also about the processor clock thing I was talking before, It idles at 800 MHz, jumps to 2.2 then to 2.7 and then to 3.7 depending on the usage, and for scrolling it just need to jump to 2.7 with a usage of 20% in that current state, about that, maybe I was't able to notice the stuttering because the CPU needs like 8 seconds to return to 800 from 3.7 GHz and if I make the test when the CPU is not in its lowest C-state the issue will not be noticeable, and maybe that is why in some Sandy CPUs it will not be noticeable.

Score
0
November 14, 2011 3:28:51 AM

ojasNot saying it's a chrome prob, i think it's a web page prob.And your PC is pretty good, maybe it can take it? I dunno...try scrolling up and down across the entire page repeatedly, very fast.

I'm not on that machine right now, but on my AMD E350 netbook (also 4GB RAM@1066MHz, 320GB HD) with Windows 7 SP1 64-bit and Firefox 8, I tried scrolling up and down really fast and didn't notice anything. Actually, wait, there is a slight delay when I go by the top of the comments section. I have noticed that a little sometimes when scrolling through some webpages on here. But on my i5 machine, I don't notice it at all.
Score
0
November 14, 2011 3:42:19 AM

Didn't they come out with an extension on a RAM stick to a USB just last year....?? I don't remember the company that did it, but it's something like the same, just with a lot more options and a lot better.....
Score
0
!