How Rancher Storage Services Unleash the Power of Software-Defined Storage

Today, our team at Rancher announced an exciting new feature called Persistent Storage Services. Persistent storage support builds on the work we’ve done with Rancher Convoy, and makes it dramatically easier to run stateful applications in production using Rancher. The Docker volume plugins, introduced in Docker 1.8 and further enhanced in Docker 1.9, enables developers to utilize a variety of persistent storage implementations as standard Docker volumes. Our new Persistent Storage Services capability complements Docker volume plugins by providing a backend implementation of a Docker volume plugin, and is the core storage technology in our recently announced hyper-converged infrastructure stack for Docker. Rancher’s new Persistent Storage Services have four primary characteristics:

1. A persistent storage service is a software-defined storage system deployed as containers. It typically includes multiple containers running on a cluster of hosts to provide redundancy and failover, building a distributed, resilient storage system out of local disk volumes.
2. A persistent storage service is deployed in conjunction with one or more applications that utilize it as persistent storage.
3. Rancher deploys and manages persistent storage services. Rancher configures the required Docker volume plugin to make persistent storage services consumable as Docker volumes. In addition, Rancher provides the service discovery and cross-app linking capabilities to enable the application to call any API exposed by the persistent storage service.
4. Applications can create Docker volumes backed by a persistent storage service, and can also go through service discovery to invoke vendor-specific APIs implemented by the persistent storage service.

One of the reasons I’m excited about the potential of persistent storage services is that it offers benefits to both application developers and the companies building software-defined storage. Traditional cloud and virtualization platforms hide the details of storage implementation from developers. Because of this, developers cannot utilize advanced vendor-specific capabilities and storage vendors cannot bring innovative features to developers. With Rancher’s new persistent storage services, developers can pick the storage software that is most appropriate for their application and utilize all of the unique features of that storage system. As illustrated in the following figure, developers can choose to deploy Gluster for applications App1 and App2 and deploy Nexenta for application App3. App1 and App2 can invoke Gluster-specific operations such as snapshot and remote replication. Similarly App3 can invoke vendor-specific operations supported by Nexenta. There are a couple of additional benefits when we deploy software-defined storage as containers:

1. Storage systems become as portable as Docker itself. AWS, for example, supports powerful software-defined storage features such as EBS and EFS. Such features may not be available on other clouds or on bare metal servers. By deploying persistent storage services in conjunction with the application, we can easily move a stateful application from AWS to other clouds or bare metal servers.
2. By packaging persistent storage services with individual applications, we reduce the failure domain of the software-defined storage system. Failure of one persistent storage service, for example, will not impact applications that uses other persistent storage services.

The following video demonstrates how to use Rancher persistent storage service:

Our first two persistent storage services will support Gluster and NexentaEdge. We are excited about both solutions as Gluster provides a widely applicable open source storage software solution and NexentaEdge is a truly revolutionary product that provides many powerful capabilities not present in open source solutions. We will no doubt be creating many other persistent storage services in the future. Storage services will be available beginning with the 0.45 release of Rancher, which will ship next week, please request a demo from one of our engineers if you’re interested in learning more. In our next online meetup we will be demonstrating how to leverage private storage services. *Will Chan is a co-founder and VP of Engineering at Rancher Labs. *