Thanks to data deduplication, it is absolutely possible to achieve a 20:1 deduplication ratio and store virtually 200 TBs of data in 10 TBs of disk space. But it is important to understand how you get there, because the impact of deduplication can be easily misconstrued.

One of my responsibilities is to train new salespeople and our partners on the REO Virtual Tape Library product line. Many times, someone I’ve been training has done a quick calculation of the virtual space available in our deduplication Virtual Tape Library (11.25 TB @ 20:1 = 200+ TBs!) and leapt to the conclusion that the product will be ideal for large data center implementations where users are backing up 30 TBs of data. Incorrect!

Data deduplication is not a magic compression algorithm that somehow can take 200 TBs of data and compress it to 10 TBs. In order for data deduplication to be effective at delivering significant ratios (like 20:1), it relies on seeing the same data over and over–like in a backup scenario. A deduplication appliance may need to see 12 full backups or more before it is possible to achieve the desired 20:1 deduplication ratio.

So, yes you can fit 200 TBs of backup data in a 10 TB bag, but to do so you must stuff it in the bag 10 TBs or so at a time.

Jeff Graham

Senior Product Manager - Overland Storage

Print | posted on Thursday, June 12, 2008 4:24 PM