Pitfalls of Scaling a Web Site

Scaling involves growing your infrastructure to keep up with the needs of your website.

Lately this has become an issue over at MacTips. You can see in the graph below bandwidth has tripled since February.

Now MacTips is pushing nearly 200GB per month.

This jump in traffic is largely caused by the MacTips Widget in the top 50, but other media like images and videos have contributed as well.

I think scaling is always a good problem to have. It means your growing.

But if not done properly, scaling can cause serious problems. Let’s look at some common pitfalls.

Not enough resources and service goes down

The first pitfall is not allocating enough resources. This leads to servers that are down and an inaccessible Web site.

Visitors become frustrated when your service can’t stay online and they forget about you.

Too many resources and waste lots of money

You over-compensate and purchase too much too quickly. This leaves you with resources that you waste for months and months.

While your visitors aren’t effected by this–this puts a serious strain on the wallet.

Going with the cheapest solution

Never go with the cheapest host. Likewise, don’t go with the most expensive unless your hosting mission critical data.

Somewhere in the 50-75 percentile should be perfect for most Web sites. The hosts in this range offer quality services with actual customer service.

This may not be important to you–but when your site is down it is the only thing that matters.

Keep it Simple Stupid

A decent simple solution is better than a complex good solution. By this I mean don’t introduce complexity where it’s not required.

Do you need replicated master/server DB servers? No? Then don’t do it.

Do you need memcached? No? Then don’t .

When looking at solutions for MacTips, I considered Amazon S3 and Joyent.

Amazon S3: Amazon S3 is great, but because it’s not based on a filesystem, syncing files becomes a big pain. This point alone made me ditch S3.

Joyent: Joyent offers virtualized servers at good rates. Planned on rysnc’ing static files (images/videos) to Joyent server because bandwidth was so cheap. After doing some research I saw some not-so-great reviews of Joyent.

Thanks but no thanks. The $50 I save now will come back to bite me when I can’t access my server and support takes 4 days to get back to me. I’m not a sys-admin, I don’t want an unmanaged server.

My Ideal Solution

So, what is my solution? Upgrading to my own server at my current host Idologic.

I think this is the best and easiest solution. It keeps everything on one server without the need for syncing.

It also keeps me with a hosting company I absolute love (Idologic rocks!).

This method is slightly more expensive, but the stress and aggravation I’m saved makes up for it.

I’ve chosen a server that should allow me to grow for the next 6 months. At that point I can reconsider my options.

Scaling is generally straight forward, however it can get quite complex. Consider your needs for the near-future and choose the easiest solution.

Often the easiest solution is best when it comes to scaling.