This is a repost from an old blog, dated appropriately
The client wanted a huge amount of disk space, cheaply. An off-the-shelf board and a bunch of drives were duly purchased, and a Linux distribution installed. Then they called me.
“OK,” I said. “Software RAID. No problem. We’ll configure Samba up, make a huge volume and get it online for you. It’ll take no time at all. We’ll do it as a nice RAID5 set, so it’ll cope properly if a single drive fails.”
(See where this is going yet?)
“No,” said the client. “We need lots of space. We’ve got another server for the really mission-critical stuff. This is just scratch space – and we need all we can get. If the array dies, we’ll just reload from the tapes.”
(Now do you see where this is going?)
“If you say so,” I said. “You really don’t care if you need to reload all the data? If you’re happy to know that in the event of a drive failure that we’ll just scrap the whole lot and rebuild a huge blank volume without the failed disk in it, we’ll go ahead.”
(You can see it coming, right?)
“Yes,” said the client.
(Here it comes …)
{ DriveReady SeekComplete Error }
“Ah,” said the client. “See, our main fileserver wasn’t ready yet, so we happened to store some really important stuff on the nice, huge volume that was sitting there. If the drive is dead, we’ll have to send it to the data recovery company we like to use.”
“You’re going to send a single volume of a seven-disk stripe set out for data recovery and work without your vitally important scratch space for as long as that takes, and still hold out hope that something will be recoverable? Even if they recover some data, we can’t restore the other six disks without the seventh.”
“We’d like to try,” said the client.
“I’d like to remind you of what I said on day one,” I said, maintaining the impassive calm of one who knew this day was coming and had prepared well. “When you give up, I’ll rebuild it as a RAID5 set, OK?”
“Er,” said the client, “we really do need all the space we can get. We can’t afford the overhead of that redundancy …”