This is just a quick article on some technical details behind the site. The previous site was based on a typical “LAMP” stack (Linux+Apache+MySQL+PHP – http://en.wikipedia.org/wiki/LAMP_(software_bundle)), although for a couple of reasons we were using FreeBSD instead of Linux. The specific software in use was PostNuke as the general site software – sort of a 2nd generation web board – plus an integration of the popular PHPBB2 package into PostNuke called PNphpBB. That software was a “dead” project and any hope of enhancment was going to require an upgrade. The software also required PHP Version 4 which had known and unfixable security holes as it was also officially obsolete.
The new hosting since las spring was a virtual private server at Verio – one of the major hosting vendors and one of the older players in the hosting market. A virtual private server is a “lightweight” virtualization technology that lets you have nearly full control over your software environment including administrative access. The down side is you’re still sharing your server with typically 10-20 (or more with higher end hardware) other accounts many of which might be hosting serveral sites. So you can have 100 or more sites on your server.
The new software is the vBulletin 4 CMS – the latest outgrowth in a commercial forum software that has moved up to be a full CMS. That selection was pretty much sealed once we knew they would ship their major update with the CMS included in time for us to us. It’s probably the most successfull commercial forum software out there with a very large ecosystem of both free and commercial support, add-ons and knowledge.
As we got ready to roll out the new site we needed a new virtual server because the software required PHP 5 and it’s difficult (possible, but difficult) to run both on the same server as PHP 4. But due to concerns about performance – the new software is slick but it is much more performance demanding on the server side. We started to look around for “dedicated” hosting. Historically too expensive for a small club like ours but prices drop rapidly. By cooincidence I found a fairly large, stable budget provider that offered colocation and dedicated hosting. Dedicated hosting is like leasing a car – the provder gives you hardware, manages backups, etc. Co-location is like renting a parking spot. You provide hardware – the provider gives you power, a wee bit of space and bandwidth. The other catch was most dedicated and co-location providers charge for bandwidth. The provider we found had an affordable fixed-rate bandwidth plan. So hosting was identified. A gently used 1U rack mount server was found, some updated processors and bigger drives dropped in, and we had a new server for the BAC site. Here it is sitting in the Liebert rack in my garage, minus it’s cover.
For the curious the basic specs are:
- Dual CPU-Dual Core (4 total) Opteron 270HE (2.0GHz)
- 6GB ECC DDR 400 Registered SDRAM
- 2*1TB Drives in a 1TB RAID1 Array (sort of)
- Broadcom 5823 Hardware Crypto Accelerator
- For good SSL performance (later) if we need/want
- Remote Management Card
- Lets us monitor temp, fans, etc remote and reboot if needed
The new server was built on FreeBSD 8, MySQL 5.1 and PHP 5.2.
An interesting point is pricing in a co-loc is based on both power and bandwidth (space can matter but modern hardware uses up available power before space usually). Part of the issue is that power equals heat. These are the 2″ copper pipes (split off from 6″ cooled water feeds) feeding one of the 6 chillers in the room at the coloc. The gauges show a input pressure of only about 85 PSI (lower than tap pressure because of the high flow rate of the water). The thermometers show about a 20F rise in temp from the input to output. You multiply that much water by that much degrees by six chillers and you get an idea of the heat that has to be removed.
So the really good price point required us to stay under 1.5A at a steady “normal” load. The server was running right at 1.5A with the single core but faster clocked Opteron 248s it originally came with. These were older chips and would suck 70-95W each. Thanks to the technology being mature we came up with a nice pair of dual-core “high efficiency” Opterons. These are rated at 55W max each. With these the server load was down to about 1.2A or about 150W – not bad for some serious horsepower.
Here’s a picture looking down the center aisle in the coloc:
This is 1/2 of the room of this provider – who is only about 1/4 of the floor. The building was an older mid-rise block building in downtown LA that had been redeveloped into dedicated data center space other than the street level retail.
If you read the floor directory it’s a veritble who’s-who of hosting and networking.
It’s hard to describe these places. The building is a full block with a wide central all running the full length of the building/block. All the doors are wide double doors with cameras and big locks.
There are no people – just the hum and roar of fans and moving water.
So that is where we left our little server:
It’s on the top of the stack.
Sitting in downtown LA a block off the 110 Freeway on the west end of the fashion district. Also conveniently 2 blocks from a Metro Red Line station so I can avoid the pain of the drive up into down town LA for any future maintenance. If they could your little bits could run down the street to a game at Staples Center or grab dinner downstairs at Mortons or around the corner at Figueroa. Oh well…
But these boxes are run differently. The Operating System is rock solid – you update software do maintanance, etc. all without rebooting. 98% of the time you need a reboot for something mundane like moving a plug or adding memory. I’ve been places where we had FreeBSD and Solaris servers with uptimes north of 800 days – pushing years – without a reboot.
So there’s your site, literally.