Last year we had the opportunity to architect and manage the website for Stereosonic, in collaboration with our friends at Carter Digital. Stereosonic is an annual music festival held in Australia between November and December, and is widely considered to be Australia’s fastest growing music festival.
Big traffic websites are a lot of fun for the nerdy dinosaurs at Serversaurus – we get to fossick around traffic graphs, play with various technologies, and figure out efficient & cost effective ways to handle short-term traffic spikes, which then become just a trickle for a large portion of the year.
Throughout previous festivals, the Stereosonic web infrastructure had buckled under the traffic spikes which occurred during the musician lineup and ticketing announcements – however in the hands of Serversaurus, this was not going to happen!
Make WordPress fly – 2,500 users/second
Round-robin DNS v HA Proxy
To split traffic out across our Varnish caching nodes, we utilised round-robin DNS using our Anycast DNS platform, which effectively means running multiple A records which are served to clients in a round-robin fashion. This is an alternative to using something like HA Proxy – our aim was managing load, not solving the problem of redundancy (more on that another time).
As the C10k problem was possibly real, we needed at minimum two cache nodes. Using Varnish cache on Serversaurus cloud servers running Debian, we fed the backend of the cache servers from a low cost business hosting account running the actual WordPress based website. During the festival off-peak season, we could then easily revert to a lower cost shared solution without the Varnish servers and other related pieces.
Because the website ran a parallel mobile version, we also needed to avoid pushing dynamic queries to the code which detected whether to show the desktop or mobile sites. To do so, we avoided a trip to the website by detecting the browser/device type within Varnish, using their VCL language – this little trick ended up being a lifesaver.
For a seamless experience, WordPress hooked into Varnish, so that content editors would experience an editing experience they were used to – after each Save/Publish action, the cache would be flushed so the new content would go live immediately.
With a test setup, we pummelled the site with different variations of traffic, utilising the awesome Blitz.io load testing tools. Utilising their Sydney PoP, we were able to saturate the link at around 400mbps, with over 2500 hits per second. The site remained online, but began to slow after 3000/hits/second, however we’d achieved our goal – according to previous years analytics and our guestimations, this was more than enough capacity.
Let’s go live
Stereosonic has a formidable social profile, with 26k Twitter followers, and 389k Facebook likes, plus an enormous mailing list. The DJ lineup was posted across all of their networks as the clock struck 8pm – users began to flow into the site, graphs spiked, and absolutely nothing happened. With almost 10,000 simultaneous users, the infrastructure barely broke a sweat.
We sat at the bier hall table in our office, drank the last of our Little Creatures six pack, looked up at Stereosonic site on the big screen, and headed home an hour later.
Web hosting – a business where it’s a great day if absolutely nothing happens.