Protecting Consul with SHIELD Cloud

If you want to protect more than one host, you can install the agent on each. They will all then register with the SHIELD instance, with different identities, and you can configure backup jobs for each of them.

Remote Agent

You can also run the SHIELD agent somewhere else, and configure your backup jobs to connect to the Consul host(s) over TCP, using either their public IPs, or (if you have configured them) internal Linode IPs.

This can be useful if you'd rather not load additional software on your Consul host(s). You can also reuse the external agent to protect multiple Consul instances, since your backup jobs will specify the IP address of each.

System Prerequisites

SHIELD uses the Consul HTTP REST/JSON API for both snapshots and restoration, so there is no dependency on other installed software.

Configuring Backup Jobs

The Consul SHIELD plugin backs up the entire key-value tree stored in the cluster you point it at.

Authentication

Three parameters govern access to your Consul server: Consul Host/Port, Consul Username, and Consul Password.

By default, the plugin will assume that Consul is accessible on loopback on the default port (http://127.0.0.1:8500), and that neither TLS nor HTTP Basic Authentication are required.

Since Consul is a distributed system comprised of several disparate members, it is normal and reasonable to run consul on a standalone host, apart from the cluster, that you use solely for data protection purposes. The local Consul agent reports into the cluster and provides a loopback entrypoint (on http://127.0.0.1:8500) without authentication. This matches the defaults for this plugin quite nicely.

TLS Certificate Verification

If you instruct SHIELD to connect to Consul using TLS, by providing an https:// URL, it will also attempt to validate the X.509 certificate presented by Consul, for things like trustworthiness, validity, and expiration.

It is, however, not unusual to find live Consul clusters that are wholly disconnected from the traditional web-of-trust of the Internet Public Key Infrastructure (PKI). Such clusters either use self-signed certificates, or a custom X.509 Certificate Authority to issue their Consul certificates.

To support these scenarios, the Consul SHIELD data protection plugin has a parameter, Skip SSL Validation that will countermand the default validation logic, and accept any correct X.509 certificate.

While this can be useful, and does help work around expired but otherwise valid X.509 certificates, it does open up your data protection setup to server masquerades, impersonation, and other such attacks.

Restoring Consul from a Snapshot

When the Consul SHIELD plugin snapshots a Consul cluster, it does a recursive walk of the / (root) of the key-value hierarchy, and dumps the found keys and their values into the snapshot in JSON format.

Restoration works this process in reverse; as keys and values are read in from the snapshot, they are PUT to the Consul cluster, overriding any pre-existing values, and creating any missing keys.

Note: this is an additive process. If you restore a snapshot to a non-empty key-value store, any keys that were not present when the snapshot was taken will continue to exist after the restore completes.

Configuration Reference

This section details all available configuration parameters for the Consul data protection plugin.

Consul Host/Port

The hostname or IP address port of your consul endpoint.

Skip SSL Validation

If your Consul certificate is invalid, expired, or signed by an unknown Certificate Authority, you can disable SSL validation. This is not recommended from a security standpoint, however.

Consul Username

Username to authenticate to Consul as (via HTTP Basic Authentication).

Consul Password

Password to authenticate to Consul as (via HTTP Basic Authentication).