What is SHIELD?

SHIELD is a data protection system, born in the cloud, and designed to be flexible and robust. It runs scheduled data protection tasks, taking snapshots of your critical data, encrypting them, and storing them safely.

SHIELD uses agents and data protection plugins to interact with your protected data systems.

The SHIELD Agent

The SHIELD Agent is a software process that executes on your servers, alongside or near your important data systems. Agents register themselves with your SHIELD, and carry out tasks on its behalf — things like taking snapshots and restoring from backup archives.

Data Plugins

A data plugin is a modular bit of logic that wraps up everything pertinent to protecting the data in a particular class of data system. Through plugins, SHIELD is able to support a wide variety of databases, key-value stores, and more.

SHIELD comes with several plugins out of the box, and we're adding new ones as we expand our overall capabilities. If you're curious, we have a list of supported data protection plugins.

Protected Systems

Protected system, or sometimes data system, is a generic term for identifying a single logical entity housing data that you care about. Your Redis session store is a data system. So is your replicated multi-node PostgreSQL database cluster.

Scheduling Backups

Once you have an agent or two installed, you can start defining your protected systems, and setting up the schedules for taking snapshots and performing backup operations.

SHIELD offers robust scheduling facilities; you can set multiple scheduled jobs for a protected system, run your backups at hourly, daily, weekly, and monthly frequencies, pick time of day, and more.

Archives & Retention

Every successful backup operation generates an archive — a singular file asset that comprises a complete snapshot of the protected system, at the time the backup was performed. These archives are encrypted and kept in highly-available storage so that SHIELD can retrieve and replay them later, during restore operations.

Archives are retained in storage for however long you wish; each has a retention policy. When an archive has outlived its retention policy it is automatically expunged from storage.

Accessing Your SHIELD

You can access your SHIELD by logging into your SHIELD Cloud Dashboard. Screenshot of the shieldcloud.io main dashboard

Clicking the Login button will launch your SHIELD web console in a new browser window or tab. You can click on the clipboard icon in the username and password fields to copy those values to your clipboard.

Installing Your First Agent

Before we can start protecting your data systems, SHIELD needs an agent to interact with. The agent will be responsible for talking to the data system in question, be that MySQL, PostgreSQL, local filesystem, or something else.

For now, we're going to back up files on the local disk.

The easiest way to install the SHIELD agent is via the curl | sh method. Pick a Linode instance and get on the terminal via SSH or LiSH.

$ curl -s https://dev.shieldcloud.io/get/fooo4bs.68JB6x2WxVuI78JD | sudo /bin/sh

   ######  ##     ## #### ######## ##       ########
  ##    ## ##     ##  ##  ##       ##       ##     ##
  ##       ##     ##  ##  ##       ##       ##     ##
   ######  #########  ##  ######   ##       ##     ##
        ## ##     ##  ##  ##       ##       ##     ##
  ##    ## ##     ##  ##  ##       ##       ##     ##
   ######  ##     ## #### ######## ######## ########

  ------- shieldcloud.io ---- data protection -------

  installation YNgccW5i3c2dtFJ7miZTaGps

installed SHIELD agent version shield v20200731.212844

Note: each account has its own unique installation URL. You can find yours on the right-hand side of the main dashboard. Never share your link!

You can check the process table to verify that the agent process is indeed running:

# ps -ef | grep agentd
root  110052       1  0 20:45 ?        00:00:00 /opt/shield/bin/shield agentd --agent-config /opt/shield/etc/shield.yml
root  110073  109268  0 20:45 pts/2    00:00:00 grep --color=auto agentd

You can also check the Agents of SHIELD area in your SHIELD web console's administration control panel to validate the proper installation and registration of your new agent:

SHIELD Agent reporting for duty

Configuring Your First System

We've got a SHIELD and we've got an agent installed and ready to go. All we need now is a protected data system. For that, we're going to log into the SHIELD web console and click Configure a new backup job, from the left-hand sidebar.

This starts you out on a guided, multi-step configuration form for defining everything about your data system, how and when to back it up, and how long to keep the resulting archives in storage.

For the purposes of this guide, we're going to skip most of the system-specific settings, and just focus on getting the local filesystem on the local Linode instance protected. For more in-depth discussion of particular data systems, check out our data systems docs.

Step 1: Information Gathering

Before you start configuring any data system, it's important to take a minute or two and figure out what you're trying to protect and how you'd like to protect it.

Since we're looking to protect the local filesystem, we'll need to know the following:

Other things you may need in other scenarios include credentials, hostnames and IP addresses, site- or system-specific configuration parameters, and more.

Step 2: The Data System

This is where we put in the details about what we're protecting. The type of data (is it MySQL? Postgres? Vault?), what role it plays in your infrastructure, etc.

Each data system has a name and some notes. The name will show up on the command-line and in the SHIELD web console, and serves to uniquely identify the system. For longer descriptions, links, and other important bits of information, you can use the "Notes" field.

Here, I've named my data system "Important Files" and left my future self some more context.

The Plugin and Agent fields denote how to backup the data and who to use to do it, respectively. For our purposes, select the Local Filestem Plugin (fs), and the agent you installed in the previous section.

This will bring up the Backup Plugin Properties form, which is specific to the selected plugin.

Here, we need to fill in the following information:

Base Directory
What part of the filesystem hierarchy to protect. We're going to enter /usr/share/doc.
Verbose Logging
By default, the filesystem plugin is pretty quiet. We'd rather have a verbose accounting of which files are being backed up, so flip this switch to the "on" position.

Step 3: Cloud Storage

This section is not strictly necessary and will be removed in a future version of the SHIELD web console.

Step 4: Scheduling

What good is a data system if we don't schedule any backup jobs?

With SHIELD, you can run your data protection operations whenever it makes the most sense, given your workload, peak usage, and personal preferences. SHIELD supports hourly, daily, weekly, and monthly backup schedules.

Hand-in-hand with scheduling is archive retention — how long do you want to keep your snapshot archives in cloud storage, before they are automatically expunged?

We'll run this example backup job once a day, at 3am UTC, and keep two full weeks of snapshot archives.

Step 5: Review

Finally, we get to review all of our selections and make any last-minute changes and adjustments before we create our new configurations.

Click that Create Backup Job button; you've earned it!

The System View

Welcome to the Data System Detail View. This is the place to go for any and all information about a protected system.

From here you can see the configuration of this data system, it's footprint in storage, all of its snapshot archives, the relevant job schedules, and a timeline of tasks and operations.

Restoring From Archive

Having snapshots of your important data systems doesn't do you a whole lot of good if you can't easily (and correctly!) restore the data in them.

From the system view, you can see the snapshot that SHIELD took when you first configured the data system:

SHIELD happily restoring our filesystem backup to /usr/share/doc

For each snapshot, SHIELD shows you an identifier (the UUID), when the snapshot was taken, how big the archive is, and it's current status. Usually that will be "valid".

Clicking on the restore link will initiate a replay of the data in the snapshot to the data system. In our case, that's a tarball of files that were in /usr/share/doc, and restoring the snapshot will put those files back.

We have snapshot archives to restore!

Congratulations!

What's Next?

If you want to learn more about SHIELD, check out the documentation for all the data systems it supports.