A decade ago I used to work as a software engineer at Hewlett-Packard in the team that wrote the support utilities for their enterprise storage. It was pretty cool as part of my role involved managing our lab which consisted of plenty of racks of kit worth in excess of a million dollars to enterprises. This was 2005 and we were still dealing with storage in the terabytes. It was the first time I’d ever done any work with storage area networks (SANs) and in particular everything we had ran over fibre. I started to strangely enjoy storage after a year working with it and even ended up with a detailed knowledge of SCSI protocol at the time.
Fast forward ten years and the most advanced storage I currently have in my network is a external 2TB RAID-0 enclosure and a tonne of smaller disks housing everything from source code, business documents and years of VM machine backups. Not exactly cutting edge considering everything else.
Last year I built the ESX for my network and made do with a 500GB internal drive to use as an operating system data store. Since then I’ve added another few hundred GB internally to the server to use on my file server but that’s it. For the last couple of weeks I’ve been considering how I’m going to address my network storage needs and have been investigating exactly what I want to build.
In the middle of preparing this blog I had my main datastore drive fail in my ESX only 6 months after purchase. This was only ever meant to be temporary and this SAN was supposed to be online within a couple of months originally. If ever there was a point proven in investing in storage this horrible irony is probably it. I managed to salvage a lot of data but completely lost two of my VMs and have temporarily lost the extra internal drive that I had.
What Do I Need?
Before looking at anything else the first question is what exactly do I need? My network runs a combination of servers for the various projects I work on over time – from core Windows and Mac infrastructure to source control, database servers, web servers and a whole selection of virtual machines. I want to get all of this placed somewhere resilient, fast and affordable that I can manage centrally without dozens of disparate storage systems around my house.
Since everything I use is now virtualised this is a great start but there are certain key goals for me:
- Provision of SSD for boot drives and databases
- Provision of large bulk storage for source control, backups, etc.
- I’ll need 1 – 1.5TB of SSD storage and around 10 TB of bulk storage
- Accessible to both ESX servers on my network
- Some degree of future expansion
- Rack mountable
- Fairly quiet
- Fairly cheap
- Resilience of data – RAID or some other solution in addition to off-site backup (I’m considering Amazon Glacier here)
In Part 2 I’ll be looking at the key decisions I’ve gone through in deciding how to implement the key requirements for my SAN.