Data reduction redux: What's in a name?

May 7, 2009 – As mentioned in a previous post (see “Welcome to the data dedupalooza” ), I’ve been researching the topic of data reduction for primary storage for an upcoming article. One thing I’ve noticed is that the industry is playing fast and loose with the term ‘data de-duplication,’ sometimes using it in place of the more general ‘data reduction’ or ‘capacity optimization’ terms.

Maybe that was ok in the realm of secondary storage, but as you begin considering data reduction for primary storage you’ll want to be more specific about what technology a vendor uses, although all of the technologies produce the same result (to varying degrees): reduced storage capacity and costs.

In the secondary storage space, in addition to data de-duplication, some vendors use single instancing or compression or a combination of techniques. In the primary storage space, some vendors use data de-duplication, some use compression only, and some combine the two.

Again, all of these technologies have the same goal, but in the case of primary storage (and depending on the type of data sets you have), it may make a difference how a vendor is implementing data reduction. And it’s misleading to refer to any type of data reduction as ‘data de-duplication,’ despite the popularity of the term.

If you’re interested in data reduction (aka capacity optimization) to reduce your capacity requirements, delay new purchases, and slash power, cooling and space requirements, plan to attend our upcoming Webcast, titled “Leveraging Capacity Optimization to Reduce Datacenter Footprint and Storage Costs," Tuesday, May 12 at 10:00 PST, 1:00 EST. Presenters will include Noemi Greyzdorf, research manager, storage software, at IDC, and Peter Smails, senior vice president of worldwide marketing at Storwize.

The Webcast will cover data reduction for all types of storage -- including primary, nearline and secondary data – and provide tips on how to maximize the benefits of capacity optimization. To register, click here.

Consider data de-duplication plus compression

Data de-duplication gets a lot of ink and engenders a lot of debates, but the option of combining data de-dupe with real-time compression on primary storage is rarely discussed, maybe because there aren’t many vendors targeting compression for primary storage resources.

That’s what Storwize specializes in, and the company recently released the results of tests that suggest a serious look at combining de-duplication with data compression to further enhance overall data reduction.

Storwize claims that adding real-time compression can improve data reduction by more than 200%, based on its internal testing as well as customer deployments. (In the internal tests, Storwize used data de-duplication solutions from both Data Domain and NetApp.)

Storwize positions data de-duplication as being particularly advantageous for highly redundant backup data sets, but points out that using real-time data compression on the front end (primary storage devices) can significantly reduce de-duplication processing time and enhance throughput, in addition to reducing overall capacity requirements.

To assess the validity of the test results, check them out here

And if you’re among the early adopters of data compression on primary storage, respond below or email me at daves@pennwell.com and let me know what types of real-life data compression ratios you’re getting.