Our first Rust crate: decrypting ansible vaults

The use case: CI with Kubernetes

Of course we like to continuously deploy our projects. For older projects we use Ansible [2] to directly deploy to virtual machines. Our newer projects however are deployed to a Kubernetes cluster hosted by Google [3]. In order to generate all the necessary Kubernetes Objects for our projects, we have created our own command line utility called Kuberwave. I will discuss this utility in a future blog post, but for now it suffices to say that it is written in Rust.

In order to enable Continuous Deployment our CI-servers require:

• Credentials which grant access to the Kubernetes cluster.
• Credentials which grant access to a docker image registry (to enable kubernetes to pull new images).
• Credentials for resources like databases and third party API’s.

We encrypt these secrets in so-called ansible vaults [4]. Ansible comes with a handy command line utility called ansible-vault to create, edit and rekey these vaults. Vaults are stored in the git repository of the project itself. As long as we only hand the ansible vault key to the people needing direct access to the kubernetes project namespace (for example, for maintenance), this setup is safe enough.

Kuberwave also has the need to access these secrets. Because our staff is already comfortable with using these vaults, we’ve decided to also employ them for our Kubernetes setup.

Decrypting ansible vaults

For this I created ansible-vault-rs [5], a library that can decrypt ansible vaults. Note that it can not create or edit vaults, because I have no need (yet) for this functionality.

$ANSIBLE_VAULT;1.1;AES256
30613330376362653133323866376665376365303734633836666136616531396361623761363061
3962356534626335323730643565346134663930363430320a313666376239373162636637613961
64366333313066646635343735626661366537666665653930646562643533363364363037653761
3730336235613732390a363166396664666262316632633536646332316334363361303030383036
3538

A small ansible vault.

The ansible-vault format is not pretty, or clearly documented [6]. I had to resort to reading the original source code [7] to understand how the format works. I have various problems with this format:

1. It is purely an ascii-based file. Most of the file is comprised of base16 encoded text spaced nicely on 80 character long lines. Clearly this is not efficient, and would only be served by the fact that git diffs look nice. Except that the entire point of the file format is that the contents are encrypted, and thus look like gibberish.
2. The file is wrapped in two layers of base16 encoded text. It becomes quite complicated to implement a streaming reader for this format.
3. Unfortunately due to how an hmac is used, we are only able to verify the passphrase after we have read the entire file. Thus we first read the entire file into memory, verify the HMAC and only then yield the file as a complete byte buffer. The format would benefit from per-block HMAC verification.

Our implementation is thus not able to stream the file. For our use-case the vaults are rarely more than a few kilobytes in size. Also we will only parse a handful of files in a run. Most of the resources go into computing the derived key of a vault using PBKDF2. For us this is not a problem, but it might be to others wanting to decrypt large files or a large number of files. If I ever have to store and encrypt large files in a git repository (probably not, due to the nature of git repositories) we will consider moving to a different file format. For now Ansible vault serves our purpose just fine.

Publishing the crate

Initially the implementation was written as part of Kuberwave, but as an exercise I decided to publish it to crates.io. I added more explicit error handling (as per [8]) and added a few tests. When creating a separate cargo.toml file and re-adding the specific dependencies, I noticed I was using a deprecated package: rust-crypto. [9] This package has not been updated since May 2016. For a crypto library this is quite dangerous. Cargo would benefit greatly from explicitly deprecating this package, as already suggested in an issue [10]. I re-implemented the crypto to use the RustCrypto [11] family of crates. The only thing still missing is more extensive documentation. I will probably also implement creating vaults. I am already working on a tool to automagically generate the Kuberwave configuration for simple projects, which will (probably) need this.

Crates are easy

Publishing the crate to crates.io was easy a pie [12]. I only had to register to crates.io as a publisher, add some fields to the cargo.toml file, and run cargo publish. Even though this crate will probably not be used widely and isn't all that exciting, it does what it is meant to do, and I hope some of you can benefit from it.

For me it has shown Rust's package manager is easy as well as powerful. The next time I write something in Rust that is general enough to open source it, we will do so.

• [1] https://tweedegolf.nl/blog/23/rust-als-webplatform (Dutch)
• [2] https://github.com/ansible/ansible
• [3] https://cloud.google.com/kubernetes-engine/
• [4] https://docs.ansible.com/ansible/2.4/vault.html
• [5] https://crates.io/crates/ansible-vault
• [6] https://docs.ansible.com/ansible/2.4/vault.html#vault-payload-format-1-1
• [7] https://github.com/ansible/ansible/blob/devel/lib/ansible/parsing/vault/__init__.py
• [8] https://doc.rust-lang.org/rust-by-example/error/multiple_error_types/define_error_type.html
• [9] https://crates.io/crates/rust-crypto
• [10] https://github.com/DaGenix/rust-crypto/issues/440
• [11] https://github.com/RustCrypto
• [12] https://doc.rust-lang.org/cargo/reference/publishing.html