Planning Your Data Protection
SHIELD can perform Consul backups in a few different ways, depending on your network throughput, access control, and personal preferences.
Local Agent
The simplest method is to run the SHIELD agent on the same box as
Consul, and connect to it over the loopback address, 127.0.0.1.
If you want to protect more than one host, you can install the agent on each. They will all then register with the SHIELD instance, with different identities, and you can configure backup jobs for each of them.
Remote Agent
You can also run the SHIELD agent somewhere else, and configure your backup jobs to connect to the Consul host(s) over TCP, using either their public IPs, or (if you have configured them) internal Linode IPs.
This can be useful if you'd rather not load additional software on your Consul host(s). You can also reuse the external agent to protect multiple Consul instances, since your backup jobs will specify the IP address of each.
System Prerequisites
SHIELD uses the Consul HTTP REST/JSON API for both snapshots and restoration, so there is no dependency on other installed software.
Configuring Backup Jobs
The Consul SHIELD plugin backs up the entire key-value tree stored in the cluster you point it at.
Authentication
Three parameters govern access to your Consul server: Consul Host/Port, Consul Username, and Consul Password.
By default, the plugin will assume that Consul is accessible on
loopback on the default port (http://127.0.0.1:8500
),
and that neither TLS nor HTTP Basic Authentication are required.
Since Consul is a distributed system comprised of several disparate
members, it is normal and reasonable to run consul
on a standalone host, apart from the cluster, that you use solely for data
protection purposes. The local Consul agent reports into the cluster
and provides a loopback entrypoint
(on http://127.0.0.1:8500
) without
authentication. This matches the defaults for this plugin quite
nicely.
TLS Certificate Verification
If you instruct SHIELD to connect to Consul using TLS, by providing an
https://
URL, it will also attempt to validate
the X.509 certificate presented by Consul, for things like
trustworthiness, validity, and expiration.
It is, however, not unusual to find live Consul clusters that are wholly disconnected from the traditional web-of-trust of the Internet Public Key Infrastructure (PKI). Such clusters either use self-signed certificates, or a custom X.509 Certificate Authority to issue their Consul certificates.
To support these scenarios, the Consul SHIELD data protection plugin has a parameter, Skip SSL Validation that will countermand the default validation logic, and accept any correct X.509 certificate.
While this can be useful, and does help work around expired but otherwise valid X.509 certificates, it does open up your data protection setup to server masquerades, impersonation, and other such attacks.
Restoring Consul from a Snapshot
When the Consul SHIELD plugin snapshots a Consul cluster, it does a
recursive walk of the /
(root) of the key-value
hierarchy, and dumps the found keys and their values into the snapshot in
JSON format.
Restoration works this process in reverse; as keys and values are read in from the snapshot, they are PUT to the Consul cluster, overriding any pre-existing values, and creating any missing keys.
Note: this is an additive process. If you restore a snapshot to a non-empty key-value store, any keys that were not present when the snapshot was taken will continue to exist after the restore completes.
Configuration Reference
This section details all available configuration parameters for the Consul data protection plugin.
- Consul Host/Port
- The hostname or IP address port of your consul endpoint.
- Skip SSL Validation
- If your Consul certificate is invalid, expired, or signed by an unknown Certificate Authority, you can disable SSL validation. This is not recommended from a security standpoint, however.
- Consul Username
- Username to authenticate to Consul as (via HTTP Basic Authentication).
- Consul Password
- Password to authenticate to Consul as (via HTTP Basic Authentication).