05.06.06

Low Memory Computing

Posted in General, Servers, Web at 7:25 pm by matt

There is a seemingly unstoppable trend in computing to have ever more and more memory available to applications. When we run across performance bottlenecks, one of the easiest fixes is usually to “add more ram.” However, there is a trend towards getting virtual private servers to host websites. This trend isn’t new. Businesses have been consolidating production servers onto virtual servers for a while now. The “new thing” is that more and more people are able to get their own slice of a real server from hosting companies. I just moved this blog, Urban Pug, and Clean Your Microfiber to a small virtual server from Quantact.com for a very affordable price. We’re at the point where you can split a decent server up 80-100 ways and give everyone decent performance.

There is a problem, however. If you put 8 gigs of ram in a system with, say, two high powered Xeon or Opteron servers, you can split the CPU cyles up and guarantee everyone a minimum amount of performance. This part is straightforward. The host system gives all guests as much CPU as they want, but when there’s contention, resources are limited with “fair” mechanism to ensure even CPU cycle distribution. However, with RAM, you can guarantee a “limited” amount of ram, but you can’t “burst” ram like you can with CPU cycles. The guest operating system can’t just be told, “hey, you have more ram now.”

That leaves us with the situation of having pretty damn fast virtual servers that are set to run with small amounts of ram. The problem that THIS creates is that standard applications such as MySQL and Apache expect a certain amount of ram to be present in modern configurations. When this happens, you can easily have a situation where you run out of ram and start swapping to disk a lot. If you’re in this situation, you might actually be better off limiting the applications in some way to use less ram (and thus potentially be slower under certain conditions).

It’s not a “win-win” situation. If you need a big fast server, you’ll still have to get a big fast server. If you just need something medium-to-small, it’s possible to do this on the cheap and still get good performance.

So, how do we do this? Basically, it comes down to limiting the ram MySQL uses and limiting the ram and number of processes apache uses (or using other applications altogether, like lighttpd). In my next few posts, i’ll discuss what i’ve learned from tweaking my own VPS, and hopefully get some feedback on how to do a better job.

Tags: , , , , , ,

1 Comment »

  1. Emergent Properties » Optimizing MySQL and Apache for Low Memory Usage, Part 1 said,

    May 10, 2006 at 2:41 pm

    […] MySQL and Apache can consume quite a bit of memory, if you’re not careful. This post discusses how to reduce the amount of memory they use without killing performance. The caveat, of course, is that you’re not going to be able to run a site with a large database and large amount of traffic with these settings. I’m going to try to explain the WHY more than the WHAT. All of this is in conjunction with my goal of reducing the amount of ram I use on my Xen based virtual server, as discussed previously in, Low Memory Computing. Before I begin, I’d like to say that you should also look at various system utilities that consume ram. Services like FTP and SMTP can and should be passed off to xinetd. Also, you should look at shells besides bash, such as dash. And, if you’re really serious about low memory, you might look at using something like BusyBox, which brings you into the realm of real embedded systems. Personally, I just want to get as much as I can out of a standard linux distribution. If I need more horsepower, I want to be able to move to bigger, faster virtual machines and/or dedicated servers. For now, optimizing a small virtual machine will do. First off, Apache. My first statement is, if you can avoid it, try to. Lighttpd and thttpd are both very good no frills webservers, and you can run lighttpd with PHP. Even if you’re running a high volume site, you can seriously gain some performance by passing off static content (images and javascript files, usually) to a lightweight, super-fast HTTPd server such as Lighttpd. […]

  2. ross mclachlan » Optimising performance said,

    December 12, 2006 at 2:49 am

    […] Optimising MySQL and Apache for low memory usage Part 1 Optimising MySQL and Apache for low memory usage Part 2 […]

Leave a Comment