Backups

There are two types of data: Backed up data, and data you're prepared to lose. The process of backing up is dead simple. Deciding what to backup and getting into the habit of backing up is the trickier part.

Threats to your data

Keep these threats in mind when planning a backup strategy:

• Bitrot and file corruption
• Burglary and theft
• Fires
• Hard drive failure
• Software failure
• Human error
  • Accidental deletion or modification
  • Misplacement of data storage
• Intentional sabotage (by someone you may know personally)
• Malware, viruses, ransomware, etc.
• Natural disasters
  • Floods
  • Tornadoes and storms
  • Coronal mass ejection
• Power irregularities
  • Dips and brownouts
  • Lightning strikes
  • Surges and spikes

A reasonable defense against all of these requires a minimum of three independent copies of the data. One or more of those copies must be powered down (offline) and at a different physical location (offsite). Remember, RAID is no substitute for a backup. Using a modern file system like BTRFS or ZFS with ECC RAM can help protect against bitrot. Surge protectors and UPS are essential protection from power irregularities. However, no protection system can guarantee absolute safety from a lightning strike, so always unplug during thunderstorms. If using RAID, a controller with built in battery backup helps prevent write failures during unexpected power outages.

What to backup

First, decide on a place to keep important data. A directory named "data" or "stuff", or a second hard drive. We'll call this our "big data" directory. Most of your important things can be placed in subdirectories within this, such as:

• Applications with addons, extensions and plugins
• Books/Audiobooks
• Documents
  • Email
  • School files
  • Work files
• Drivers
• Games
• Music
• Movies and TV
• Offline code repositories
• Operating systems (see install kits, below)
• Password databases
• Photos
• Virtual machines
• Wallpapers

You may have your files in an unsorted mess, such as a catch-all "Downloads" directory. No problem! Make another subdirectory under big data:

• Unsorted

Some files can't be moved to your big data directory because they either live in a specific place, or don't exist until you manually create them:

• Application and operating system configurations
• Browser bookmarks
  • Adblock custom filters
  • NoScript whitelist
  • RequestPolicy whitelist
  • Saved passwords and logins
• Calendars
• Chat logs
• Contacts
• Database/FTP/RDP/SSH/TeamViewer server bookmarks
• Offline email
• Things in "My Documents" and "AppData"
• Video game save files
• VOIP/Teamspeak server bookmarks and identity keys

To have a copy of these things in your big data directory, you may have to manually "export" or "backup" from the applications they're used with. Alternatively you may need to backup a copy of the corresponding AppData or .config folder. Some of this you will be able to automate with a script. Don't forget to keep the script in your big data directory too!

You may want to include a datestamp in the file names of these things to keep multiple copies. If your password database becomes corrupt you'll want to restore an earlier version, rather than a perfect backup of the same corruption!

Install kits

You should aim to be able to go from an unformatted drive to a complete working system, with all applications fully installed and configured, WITHOUT network access. If you need network access to download or activate anything then you don't really own it and don't have it backed up.

To do this, you will need copies of a few things:

• Your Operating System(s) isos
  • Tools to put the iso onto a thumbdrive such as Rufus
• Drivers for all of your hardware
  • Wired and Wireless NICs
  • GPUs
  • USB3
  • Soundcard
  • Bluetooth
  • Printers
• Firmware for your motherboard/peripherals/devices
• Applications you use
  • Configuration files for them, including window managers and ricing configs
  • Plugins (Web Browsers, etc)
    • Configuration files for the plugins (adblocklists etc)
• Windows Specific stuff
  • Activation tools such as DAZ Loader and/or Microsoft-Activation-Scripts
  • dotNet
  • DirectX
  • msxml
  • vcredist 2005, 2008, 2010, 2012, 2013, 2015
  • xnafx
  • Games for Windows Live
  • Java

Make sure all of the installers you have are full installers and not web setups.

Alongside your main backup, have a USB thumbdrive with your operating system installer on it. It's handy to have all of your drivers, applications and configuration on this thumbdrive too.

Once a month or so, check on these files to see if new versions are available. Perhaps when you run your regular backup?

Android

If you have a custom bootloader applied such as ClockworkMod, or TWRP, you can likely create a full backup of your device including data and configuration to an external storage card. You can also use free-libre apps like OAndBackupX (requires root) found on f-droid to backup apps and data.

Once complete, copy the backup from your storage card to your "big data" directory.

How to backup

With all your data in a central location, you're ready to back it up.

Depending on your needs, investing in a file server, NAS or SAN may be worthwhile. A simple option is to get yourself an external hard drive of sufficient capacity for you data. Encrypt it if you want.

We use rsync to backup data.

Once the external drive is mounted:

 $ rsync -av --delete /path/to/mydata/ /path/to/externaldrive/

• -v being verbose
• --delete [optional] delete files on your /externaldrive/ which no longer exist in /mydata/
• -a being "archive mode, which includes:
  • -r recursive mode
  • -p preserve permissions
  • -t preserve times
  • -g preserve groups
  • -o preserve owner
  • -l copy symlinks as symlinks

Note: if you are backing up your full system, use the --exclude on your backup folder directory so that the process doesn't loop. --dry-run is also helpful.

rsync will update any modified files and create any new ones.

There are many other software options which work in a similar way to rsync. See Unison (all platforms) or SyncToy (Windows). There is also a GUI front end for rsync for newbs: Grsync.

Keep the external drive unplugged and stored away when not in use. You don't want a power surge taking out your computer and your backup in one hit.

Cloud storage

Cloud storage is a bad idea because if you:

• don't have network access
• have a slow connection
• have a data limited connection which costs extra for excessive data
• lose your password
• get phished/cracked
• get banned from the service

then you don't have a backup and have lost your data.

Other considerations:

• Unencrypted backups can be read by the cloud provider
• Cloud provider could go offline/shutdown without warning
• Cloud provider may be in a different legal jurisdiction to you
• Cloud provider can raise the cost of subscription at any time
• Data could be leaked by cloud providers in a hack and be read if unencrypted

When to backup

This depends on how often your data changes. Once a week is a good. Once a month at least.

You may want to run an ad-hoc backup when something important changes such as encryption keys or password databases.

Testing the backup

Most of your backup (except drivers) can be tested in an isolated Virtual Machine to see if you've missed anything or if some of your install files require internet access.

If you're fairly confident in your backup, wipe your computer and test it. It's better to find out today that something was lost than finding out when it really matters. If this seems too scary, recheck your backed up data until it gives you confidence.

External Links

Archive Team - Backup Tips.

See also

Home server

Storage devices