You have secrets in your cluster. Everybody does, it's a fact of life. Database passwords, API keys, deployment tokens, just to name a few. Secrets are hard to manage, even before you throw in the fact that most of us now are operating in a cloud environment. In software development, the common advice to "never roll your own crypto" tends to get thrown out the window when you're talking about infrastructure secrets. These home-grown solutions are hard to maintain and rarely audited properly. But the alternatives aren't great: go without or suffer vendor lock in. And that's without even mentioning tasks we all need to know we need to do like key rolling. Add containers into the mix and you've got a real headache on your hands.

At Cisco, as we developed our opensource microservices platform Mantl, we wanted to address all aspects of the software developers security concerns. Security is still one of the top concerns in cloud today and it should be.  To address security in depth, you need to start at the beginning of the project with security practices and controls in mind. In this post we are looking at secrets. We leveraged Vault, an open-source tool recently released by Hashicorp to manage secret sharing within a datacenter. We're going to highlight some key features that you'll want to know about, and then get into how to actually use the software in Mantl. Vault on Mantl means that you can have more secure integration with your cloud resources as well as enhanced operational procedures internally. We think that it sets Mantl apart as a microservices  infrastructure that takes security seriously.

The first, and arguably most useful, feature of Vault is automatic secret rolling. As the administrator, you set up and manage secure backends like PostgreSQL or AWS. Essentially, you give Vault permissions to create and manage users with a specific set of access permissions in these systems, and Vault will take care of issuing and revoking secrets. It has the concept of a lease on secrets, so a client knows how long the secret it has is good for, and when it will need to get a new one.

But secret rolling isn't very useful without the ability to audit secret issuance and access, and Vault delivers here too. It can write audit logs to a file on the system or to syslog for your log solution to slurp up for later review. And how do you connect with this system? Vault ships a handful of auth backends. For operators, Vault has LDAP, MFA tokens, and Github authentication. For automated consumers, there are the methods you'd expect like TLS certificates and token-based authentication plus "app ID", a mechanism for new nodes to authenticate with Vault.

You can either download Vault for your platform atthe Vault project download page. A note about the download and release process: Hashicorp publishes checksums signed by their GPG key for verification. The codebase itself, besides being open source, isprofessionally audited by iSec. That is not to say that it's perfect, but it's quite a step up from unsigned release binaries on Github. Of course, since Vault is open source you can also choose tocompile a release yourself with the instructions in their README.

Let's get started using Vault. We'll be running these examples locally, and I'm assuming that the vault tool is in your path. If you've just downloaded the release, navigate to the directory and at the command line use./vaultinstead. In these examples,$indicates terminal input, all other lines are output.

Dev Server

We'll be using Vault's dev server locally. To start it, runvault server -dev:

$ vault server -dev

==> WARNING: Dev mode is enabled!

In this mode, Vault is completely in-memory and unsealed. Vault is configured to only have a single unseal key. The root token has already been authenticated with the CLI, so you can immediately begin using the Vault CLI.

The only step you need to take is to set the following environment variables:

export VAULT_ADDR='http://127.0.0.1:8200