Next time I'll: Implement a different backup and recovery strategy at distributed sites--one aimed at reducing the amount of data to be backed up at such sites. I'll avoid getting so close to the bleeding edge and ensure that I choose technology that provides the best balance between robustness and cost.
Most ill-timed backup failure: No backup failure is good, but the restore experiences stand out most. Restoring 200 GB of data to a server hundreds of miles away simply takes too long for anyone to be happy. That's why we're pursuing our storage strategy.
Why direct-attached storage is out: When hundreds of servers are involved, managing direct-attached storage and the associated backup activities is not as cost-effective as the alternatives.
Biggest mistake made in technology circles today: Forgetting that technology exists to serve the business.