From genetic data to high-resolution video, unstructured data is breaking traditional storage architectures from the inside out. One of the most promising storage options for organisations producing large volumes of unstructured data is object storage. Object stores have long found a home in the cloud and inside data centres, becoming long-term repositories for high-value data - and for good reason. But with demand for storage capacity growing exponentially, how can organisations reap the benefits of object stores without costs spiralling out of control?

DATA RETENTION & PROTECTION ARE KEY
Object stores abstract away the location of an object, which enables higher levels of redundancy. This protects against device (e.g. hard disk) failure but also against failures of entire nodes or even entire data centres. Abstracting away object location also enables object stores to scale to sizes and topologies difficult to achieve with file systems. The user of an object store may be unaware of where exactly their data is physically stored. What looks like a single object store may be distributed across locations in multiple cities to achieve greater reliability against natural disasters. This level of durability could tremendously increase the capacity requirements of the underlying hardware, but with smart erasure coding algorithms, it can be achieved using less capacity than by mirroring the data.

A key issue when deploying object storage is data retention. The retention period for many kinds of data is specified by legal and other compliance constraints. One might expect that data not subject to compliance requirements is likely to be deleted sooner, but some data has value indefinitely. For example, geological and genetic data doesn't have an expiration date. These kinds of data sets can represent a tremendous investment and the data never expires.

THE COST CONUNDRUM
Demand for storage capacity is growing at a compound annual growth rate of more than 20 percent. Long-term repositories must become cheaper and deeper without losing durability. That means object stores must lower the overall total cost of ownership of storage. This includes not just the cost of the media, but associated expenses of owning a piece of equipment like the costs of acquisition, maintenance, power, cooling, and the enclosing building and the land it sits on. The majority of today's object stores are hard disk-based, which provides good performance and reliability, but the cost of power and physical footprint is significant. To lower TCO, object stores are incorporating tape. Yes, tape lives!

Tape has lower media costs and, unlike disks, requires minimal power and cooling when not being accessed. Tape tiers are ideal for large amounts of data stored for long periods of time. And for sequential IO, tape actually outperforms disk for both reading and writing. But tape works (and fails) differently from disks. The latencies to access data on tape can't be ignored. Best practice implementations will present tape as a separate tier to allow applications to help manage data access.

Exploiting the full advantages of tape in an object store also requires a deep understanding of how to properly manage and treat it. The object store must account for and survive failure modes that are unique to tape. It must also manage access patterns to reduce tape latencies and wear. And all of the required complexity must be implemented below the object interface, saving the user from experiencing it.

Because tape excels at sequential access, large individual objects will perform best. However, a well-implemented object store will group small objects into larger sequential streams to and from tape. With the right expertise, organisations can implement a tape-based object store for long-term data retention while keeping storage costs firmly under control.

More info: www.quantum.com