All our servers now have 256 GB of memory. What can you do with a server that has that much RAM?
Last night, we finished expanding the memory in all our nodes. The project took about a year, and every server now has a quarter of a terabyte of memory. Having plenty of memory in a server makes sense because it lets us do interesting things.
Of course, users consume some of the memory themselves, with between 60 and 100 virtual servers running on each server today. Each VPS uses an average of 1067 MB of memory. The median is 680 MB, and half of all virtual servers fit within one GB.
Virtual servers have a default memory limit of 4 GB, and a single VPS can be allocated at most 16 GB. But only 9 VPSes have extra memory. The vast majority of users fit comfortably within 4 GB. The following graph shows actual usage:
The rest of the memory (the larger part) is used for caching. We use the ARC cache, part of the ZFS filesystem. Unlike the kernel's default LRU cache, it keeps more detailed information about individual pages: not just when they were last used, but also how often they are used.
When the system needs to evict some pages and replace them with others, it makes a more considered choice. It does not unnecessarily discard frequently used memory that happens not to have been used for a little while. Its statistics tell it that the data still matters, so it chooses less frequently used pages instead.
Evicted pages are not discarded completely, either. They go into the L2ARC cache on a fast SSD. If the system needs them, it can retrieve them faster than from spinning disks.