Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Well, today computers really only have one kind of storage, and it is usually some sort of disk, the operating system and the virtual memory management hardware has converted the RAM to a cache for the disk storage.

I used to think that too. Specifically Windows NT was said to need a pagefile at least as large as physical RAM. This was back when a workstation might have 16MB RAM and a 1GB disk. I thought this was because the kernel might be eliminating the need for some indirection by direct mapping physical RAM addresses to pagefile addresses. I was wrong.

On the Linux side, you would typically see the recommendation to make a swap partition "twice the size off RAM". Despite the possibility of using swap files, most distros still give dire warnings if you don't define a fixed-size swap partition on installation.

I don't think there was ever a solid justification for this "twice RAM" heuristic. A better method might be something like "max amount of memory you're ever going to need minus physical RAM" or "max amount of time you're willing to be stuck in the weeds divided by the expected disk bandwidth under heavy thrashing".

Regardless, if your server is actively swapping at all you're probably doing it wrong. It's not just that swapping is slow, it's that your database or your web cache have special knowledge about the workload that, in theory, should allow it to perform caching more intelligently.

I'd prefer to disable swap entirely, but there are occasions where it can make the difference in being able to SSH into a box on which some process has started running away with CPU and RAM.

But this guy is a kernel developer so he seems to feel that the kernel should manage the "one true cache". I like the ease and performance of memory-mapped files as much as the next guy, but I wouldn't go sneering at other developers for attempting to manage their disk IO in a more hands-on fashion.



On NT, if the kernel bugchecks and it's configured to do a full or kernel memory dump, it dumps to the pagefile [1] (which get's copied elsewhere after you reboot). If you were looking for another good reason to have a pagefile the size of your physical memory on NT, there you go :-)

[1] http://support.microsoft.com/kb/254649


Yep, me and like 3 other guys I know actually do look at kernel minidumps and have a big pagefile for that reason.


On at least one (admittedly, misconfigured) system of mine, Linux got up to a working set around 4x physical RAM before kswapd became CPU bound instead of disk IOPS bound and everything stopped working. Assuming that anecdote is the usual outcome, you can justify the RAM:Swap ratio rule of thumb as an upper limit on how much swap can be usefully used for anything other than a band-aid on a memory leak.

edit: NT really does need substantially more swap than most Linux configurations, as it always runs with the Linux equivalent of overcommit disabled and a high swappiness.


That's how the story plays out for me on a desktop system if a single process starts running away with memory.

But on a production server this is guaranteed to happen at the worst possible time (i.e. at the peak of the daily load cycle). As soon as you start swapping, it starts slowing down and the outstanding transactions begin stacking up. The response time goes all hockey-stick shaped and it's a death spiral.


If you have overcommit disabled and a process that is using most of your RAM (ahem, Firefox) wants to fork+exec, you will need a large amount of swap. This is not a problem for most Linux users since they run with overcommit on, but it could be a problem on more rigorous OSes. Just one of those dark corners that people don't think about.


Is not suspend to disk in Linux normally done with a swap partition? My understanding is that the swap size heuristic is influenced by that.


Yes, also in Windows. So it's another reason to have a swap partition, but not to actually swap to it in normal operation.


No harm swapping to it if you have it, as far as I can tell.


If you have a server attempting to process a steady stream of incoming transactions at the edge of its capacity and it begins turning memory IO operations into disk IO (which is 1000 times slower), that is harmful to response times.


There probably isn't a lot of sense in having swap for suspend to disk on a server anyway of course.

I've only heard the heuristic in the context of workstations/laptops.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: