What happened to RAM compression?

August 16, 2005 9:00 PM

Are there any Windows RAM compression programs still out there?

I know that virtual memory made RAM compression into a fairly expensive (in both money and CPU cycles) way to give your system more RAM, but what if you've maxxed out physical RAM and are still requiring huge amounts of swap? I looked up MagnaRAM, but it looks like Quarterdeck got swallowed by Symantec and they didn't continue the product.

posted by krisjohn to computers & internet (13 comments total)

more details please. OS? amount of memory? have you stopped all unnecessary services and removed unneeded TSRs?
posted by monkeyman at 9:15 PM on August 16, 2005

Freeram XP helps to free up some RAM when it runs low.

It helped me fore quite a while until I quadrupled the amount of RAM I had in my system.
posted by tomble at 9:27 PM on August 16, 2005

well... since ram accesses can happen a lot, any ram compression would be pretty expensive. i would question the utility of it, even if there were a decent product for it out there.

a high-end consumer pc should be able to handle 4G with no trouble. if you're still paging memory, maybe you should look into scsi or some of the higher-end memory access channels.

which leads me to wonder...what are you doing that's so memory hungry?
posted by moz at 9:35 PM on August 16, 2005

monkeyman: It's mostly an abstract question, but if you want specifics you can use my laptop; Windows XP SP2, 768MB of RAM, typically 512MB swap minimum with easily over 1.5Gig when busy. I know what I'm running and there aren't any big background things I don't need.

A friend was recently complaining that one of their servers, not Windows, had 2Gig of RAM and was still using up all of the allocated 4Gig of swap. Though he implied the problem might have been a mis-configured database that will remain nameless.

tomble: It's been my experience that programs like that just swap everything out all the time and actually slow down the system when you're trying to move between lots of things quickly.
posted by krisjohn at 9:41 PM on August 16, 2005

moz: A new high-end PC would probably be able to handle 4Gig, but we have, for example, some older PCs at work that maxxed out back at 384MB of RAM. And we have a batch of slightly less old PCs that use PC133 RAM, which is a bugger to find in 512MB sticks at a reasonable price.

Then there are laptops.

Mostly though, I'm a compression junkie and I was wondering if anyone still produced a real RAM compression program for Windows XP. Lord knows I've implemented stuff far more twisted than a bit of software compression over main memory.
posted by krisjohn at 9:47 PM on August 16, 2005

I think the speed hit of having to decompress / recompress everything on a page hit would really kill performance. Plus there are lots of situations where compression would be impossible - e.g. executables and DLLs are memory-mapped from their images on disk.

On a related note: The Memory-Optimization Hoax is an article by Mark Russinovich of sysinternals.com where he debunks the idea that RAM somehow needs "optimizing". Few people know as much about windows as this guy, and he has access to the source code.
posted by Rhomboid at 9:57 PM on August 16, 2005

s/page hit/page fault/
posted by Rhomboid at 9:57 PM on August 16, 2005

Rhomboid, you mean this article?
posted by davy at 10:32 PM on August 16, 2005

*sigh* Yes. C+P error.
posted by Rhomboid at 1:06 AM on August 17, 2005

It really is not worth your time to compress your ram. See, the point of you compressing it in the first place would be to make more room, thereby speeding up the apps you want to run.

As you know, it is very expensive in CPU cycles to A: do it and B: get data back from it. There is a very good reason why ram compression software is getting harder to find. Probably your best best if you can't sleep without it is to pull out your 256 stick and replace it with a 512, giving you another 256. Otherwise, you would be better off optimizing windows to siginificantly increasing your performance.

Another option that is in the air is getting a USB 2 external hard drive, something like a 7200 rpm 8 or 16 mb cache drive and telling windows to use that as the swap drive, considering your laptop drive is not very fast.

If you want to be totally abstract, the cost of the software would be much better invested in more physical ram.
posted by Dean Keaton at 1:14 AM on August 17, 2005

Dean Keaton: My laptop has 256 soldered to the motherboard and a single slot with a 512MB PC133 SODIMM. I think it's maxxed out, but even if it's not, the only way to upgrade is to replace the (mildly expensive) 512MB PC133 SODIMM with a (very expensive) 1Gig PC133 SODIMM.

The USB 2 hard drive idea would be cool if the laptop had USB 2 built into it. It doesn't. I have a PCMCIA USB2 adapter, but I think the latency it introduces would counter any attempt to speed up swapping. Also, using the interface for swapping would kill the framerate on my USB2VGA adapter.

The laptop is only an example system for the purposes of anyone that wants to play the man not the ball. I know when a system's hardware is effectively maxxed out.

Rest assured that I won't be buying any "memory optimisers", but I would be interested in exploring any real RAM compressors to see if the increased overhead required to manage compressed RAM is paid back by reduced swapping.
posted by krisjohn at 1:48 AM on August 17, 2005

I'm honestly surprised that RAM compression hasn't made a comeback. Lots of people use computers that don't have enough RAM for whatever reason. When RAM compression was popular the last time around, typical clock speeds were 50mhz, and the average access time on a hard drive was around 18ms. Now, typical clock speeds are 3Ghz, and access times are about 9ms. In fact, since processors do so much more work per clock these days, your processor probably does about 150-500x as much work per second as it did back then, and yet hard drives are only about 2x faster in access time, which is generally the limiting factor for paging. To some extent, DMA hard drives help, because one program can be running while the other one is paging in. However, we still have the situation where the processor can execute over 20 million instructions in the time it takes to load in a single 4k page. You should be able to compress and decompress data in less than 5% of the time it takes to pull it from disk. Thus, if you're really swapping much at all, it would seem to make sense to have a portion of your RAM kept in compressed form.
posted by cameldrv at 3:49 AM on August 17, 2005

Exactly, I cannot belive that this great idea has died!

BtW: When I first tried a RAM Compressor with Windows ME
6 years ago I was rather sceptical - I was convinced that this was absurd nonsense. (unsound like creating a Swapfile in a RamDisk or unsound like cacheing files in Memory while caching Memory on files at the same time)

But The Ram Compressor worked fine with Windows ME and so I read the technotes on it.

And now I know that some strategies may seem to be
unsound at first glance while in fact these are genious Improvements !

This is what I remember having read, at least it sonds similar:

The sophisticated MMU can ensure that often used Mempages will be available
in fastest RAM and the MMU is also capable of swapping out those rarely used Mempages
either to a permanent swapfile on an USB Stick or to a ermanent swapfile on an Flashdrive or to a permanent swapfile on a IDE/RAID Disk.
But more than this it may also swap rarely used Mempages
either to unused, slow Memory or to a permanent swapfile on a Ramdisk.

This is all implemented in our computers ever since the 80386 Processor (I386) and this is what happens dayly in our computers.

Of couse should we increase the overall performance by improving the configuration in case the system shows a huge numbers of PageMisses for a special process.

----

But the Method of swapping out a no longer active MEMPage to anywhere and the Method of re-getting this MEMPage back in case of a PageMiss -
these 2 Methods do in no way interfere with the current Memory Management Achitecture - These Methods may implement whatever transport or encryption seems apropriate (simply copying the data is the default implementation here)

Wheather we apply Compresion when a page goes outside or wheather we use Decompression when the page is reactivated . these two options do in no way influence the current Memory Memagement Architecture -
these two options can be seen as additional and independent goodies!!!

Dont get fooled by the Time Consumption a Compresion Algorithm would need:

The MMU compresses very quicly with high - but not optimum - Comression Rate when writing to a
compressed file/Ramdisk/MemoryBlock.

And decompressing is even much faster.

And more than this:

If a rarely used Block must be mapped into Memory because of a PageMiss then it is most likely that it will
be used r/o, and it will probably not be altered.

And so, if later this MEMPage must be swapped out again
then recompression - or even copying out - will be superfluous. (This compressed content is still on the pagefile and the "compression" takes zero time in this case).

Overall we may expect that not only there will be virtually more Memory available - both in Ram and on SwapFiles - we also may expect the I/O Operations from Memory to SapFle (nd vice versa) o be ready in half time, no matter what Bus Systems are used - Swapping itself gets faster because of less data to be swapped !!!

Since the system offers all the necessary Functional Blocks like "optimizing MMU", "compresing and decompresing MMU" and permanent Swapfile on RamDisk and on Striped RAID Disks and on USB-Stick and on FlushDisk we should be able to install a compressed memory Mnagement even without patching any System Device Driver :

We only must convince the Operating System of the fact,
that permanent PageFiles must have the "compressed"
Attribute set - that's all.

yours, Gerald

Maybe some fine day I will try it.
I will first install a RamDisk that eats up 70% of 1 GB Memory and I will create a permanent compressed Pagefile
on the RamDisk.
Furthermore I will also create a permanent compressed
PageFile on a USB Disk or on a striped RAID DIsk.
And I will inform you - hopefully I will be able to try it within 1/2 year.

yours, Gerald
posted by GeraldTrost at 6:23 PM on August 30, 2005