My server melted on launch day - It really does happen!

13 replies
Hi,

I know a lot of people here get the same emails I do from some of the usual suspects after a launch with something along the lines of "the response was so good our server practically melted and we couldn't let everyone in before the deadline, so we've extended it to give you chance to sign up" yada yada yada.......

And most people probably think it's some sort of marketing ploy - and maybe it often is.

But...

It does actually happen.

I just launched a new membership on Sunday and the server is the one my programmer runs our high ticket software systems on. It's been in use for several years and has been solid as a rock - so launching a wordpress membership site should be simple - right?

Well, we sent the email out letting people know it was now open and thousands of them ran across to the server and almost 100 signed up in the first 24 hours.

The server crashed! (for the techies among you - the Apache server crashed and also stopped some other services) and needed rebooting. Then as the load stayed up with thousands of hits an hour we kept needing to restart some services.

There ended up being 2 issues causing the problems:

1 - The PHP service needed more memory than expected. Wordpress recommend 50MB and we allocated 100MB but it wasn't enough and we've doubled it now and that helped.

2 - Wordpress has 'issues' with particular php versions so we're going to need to upgrade the version on php on the server. The issue with that is that when you do this php installs some things you don't want and uninstalls some things you do, so it's not necessarily straight forward.

Now that we're back to normal running and the surge has passed things look ok, but I just wanted to give a reality check to people who think these things are all just marketing tactics - these launch day issues DO genuinely happen and so next time you get one of those emails, consider that it might actually be someone who really did get a bunch of problems they weren't expecting.

On the plus side for us - it was our server so we had full access to get in and make all the changes. If you're using a limited hosting account and you need to rely on someone else understanding these problems and responding for your launch - it could be much more painful.

The lessons.....

1 - Not everyone is creating fake problems to sell you stuff

2 - Get your own server or a very good relationship with a provider in your timezone and let them know about your launch

Andy
#day #happen #launch #melted #server
  • Profile picture of the author Andy Fletcher
    Hopefully the advent of cloud based computing services will make this a thing of the past before too long.

    Pretty much everything I do now is built on cloud based services that scale nearly infinitely. So while technical problems do happen, they shouldn't be caused by "too much traffic" much longer.

    After all, people like Google, Facebook, Digg etc etc have all developed ways of scaling to manage users bases far beyond anything we dabble with at this end of the marketing spectrum. It'd be silly not to leverage what they've learned to help out own success.
    {{ DiscussionBoard.errors[2809827].message }}
  • Profile picture of the author Tyrus Antas
    A good way to prevent this is to use a static html salespage
    placed inside an iframe, hosted on Amazon S3.

    That way you avoid the ugly S3 url's while still getting
    rock solid scalability for a very low price.

    Congratulations for your launch!

    Tyrus
    {{ DiscussionBoard.errors[2809858].message }}
    • Profile picture of the author source47
      Originally Posted by Tyrus Antas View Post

      A good way to prevent this is to use a static html salespage
      placed inside an iframe, hosted on Amazon S3.

      That way you avoid the ugly S3 url's while still getting
      rock solid scalability for a very low price.

      Congratulations for your launch!

      Tyrus
      Great idea! Thanks Tyrus.

      -Curtis
      {{ DiscussionBoard.errors[2811289].message }}
  • Profile picture of the author Andyhenry
    Both good sets of advice guys

    Unfortunately, I guess like a lot of people I thought we were prepared and since we were only expecting a few thousand people to respond in the first few weeks I just thought it wouldn't be a problem.

    I have looked into cloud hosting and have actually set up accounts on several hosts (including S3 for videos etc.) but since we have a server and I have someone who is physically near it and can go in and swap out hardware etc if needed, I thought we'd already over-prepared

    But it's a lesson for anyone here thinking of doing a 'big' launch - you might think that when you send that email out to tell people you're open for business is when you've reached the finish line - but in reality your launch finishes weeks later and that email is often just the start of the launch.

    If you're not sure whether you've prepared enough - ncmedia's advice about throwing cheap traffic at it is spot on, sometimes, like in this case for me, it was only when using wordpress and php of certain versions and the php issue needed large volumes of traffic to expose it - we actually did the required fix on another of our servers a couple of years ago but forgot about it on this one because it's usually our test and demo server.

    Anyway - it seems you can never prepare enough and don't be too hard on people that do get these problems - one day it may be you.

    Andy
    Signature

    nothing to see here.

    {{ DiscussionBoard.errors[2809875].message }}
  • Profile picture of the author BloggerHigh
    Originally Posted by Andyhenry View Post

    almost 100 signed up in the first 24 hours.

    The server crashed!
    You need a new server.


    While Wordpress can be a bit heavy on disk and ram, less than 100 people shouldn't crash it, even on a shared Hostgator $5 job.
    {{ DiscussionBoard.errors[2809901].message }}
    • Profile picture of the author Andyhenry
      Originally Posted by BloggerHigh View Post

      You need a new server.


      While Wordpress can be a bit heavy on disk and ram, less than 100 people shouldn't crash it, even on a shared Hostgator $5 job.
      Sorry - I must've been unclear. That was the number that joined as paying customers. There were actually over 15000 visitors.

      It is our server hosted in a decent datacenter - it's not shared at all, we bought it, installed it and run it.

      The issue was to do with Wordpress and php versions - plus the number of visitors.
      Signature

      nothing to see here.

      {{ DiscussionBoard.errors[2811030].message }}
  • Profile picture of the author tht222
    It happened to MS when they launched the free Windows 7 Beta, so no matter how well you plan, you never know, but I would take a server crash on a product launch any day
    Signature

    Keyword Snooper - fast and super-affordable keyword tool

    {{ DiscussionBoard.errors[2809917].message }}
  • Profile picture of the author LauraJames
    Thank you for letting us know about this. I am sorry to learn that you had some difficulties with this process. It's a good thing you had access to your server to make the necessary changes? How long did it take to straighten out the issues?
    Signature

    Chief Executive Officer
    Best Designed Blogs
    Based in Canada and the USA
    None of our work is outsourced

    {{ DiscussionBoard.errors[2811090].message }}
  • Profile picture of the author Greg Jacobs
    Launching on Wordpress is a bad idea because it takes an inordinate amount of system resources (and DB connections) to load a page. High volume LPs should always be static HTML with images/videos on a CDN when possible.
    {{ DiscussionBoard.errors[2811105].message }}
    • Profile picture of the author Andyhenry
      Originally Posted by Greg Jacobs View Post

      Launching on Wordpress is a bad idea because it takes an inordinate amount of system resources (and DB connections) to load a page. High volume LPs should always be static HTML with images/videos on a CDN when possible.
      Agreed - we live and learn
      Signature

      nothing to see here.

      {{ DiscussionBoard.errors[2812744].message }}
  • Profile picture of the author john17
    Cloud is the way of the future. I always like to spread things out. Even if you are not doing a launch - we are in 2010 and downtime is unacceptable. (Twitter get with it!)

    Our routine is usually...

    Videos hosted on Cloudfiles
    Images on Amazon s3
    Site on dedicated server.
    Signature

    Start your own Video Sharing Website at VideoSharingEmpire.com

    {{ DiscussionBoard.errors[2811129].message }}
  • Profile picture of the author snowbored
    If you are using Wordpress be sure you have W3 Total Cache installed. Especially if you are on shared hosting. Even when you server isn't overloaded, just the improvement in page load time it gives is worth it. It has some cool features to integrate with S3 as well.
    {{ DiscussionBoard.errors[2811176].message }}
    • Profile picture of the author Jesus Perez
      Originally Posted by Tyrus Antas View Post

      A good way to prevent this is to use a static html salespage placed inside an iframe, hosted on Amazon S3.

      That way you avoid the ugly S3 url's while still getting
      rock solid scalability for a very low price.

      Congratulations for your launch!

      Tyrus
      This is a superb suggestion. Even better than Amazon S3 is the Rackspace Cloud. I've heard about Amazon S3 behaving erratically under heavy loads.

      Originally Posted by Greg Jacobs View Post

      Launching on Wordpress is a bad idea because it takes an inordinate amount of system resources (and DB connections) to load a page. High volume LPs should always be static HTML with images/videos on a CDN when possible.
      Agreed. Why use a salespage that has to "talk" to a MySQL server? There isn't anything on a salespage that requires MySQL. Use a static page and save the precious MySQL resources for members.

      Originally Posted by john17 View Post

      Cloud is the way of the future.
      Yes, if you're planning a huge launch, consider using a Cloud service like Rackspace Cloud or LiquidWeb Storm. Solutions like these can be instantly upgraded via a Cpanel. I mean instantly. You can go from 256mb of memory to 4gb in 1 minute with the click of a button. That's 30 minutes faster than shutting down the server, installing a 4gb stick and powering it back up.

      Originally Posted by snowbored View Post

      If you are using Wordpress be sure you have W3 Total Cache installed. Especially if you are on shared hosting. Even when you server isn't overloaded, just the improvement in page load time it gives is worth it. It has some cool features to integrate with S3 as well.
      Plugin Warning: W3 Total Cache completely screwed up my Wishlist Member installation. It might mess up other membership plugins as well.
      Signature

      {{ DiscussionBoard.errors[2811275].message }}

Trending Topics