Highly Reliable Systems: Removable Disk Backup & Recovery

Speed Problems with Cloud – 1000 Times Slower than Local Backup?

By Derry Bryson

Backing up data to the cloud is an awesome way to make sure data is protected without thinking about it too much.  New cloud based companies, software, and data vaulting locations appear daily.  But there can be speed problems with cloud backup.  Backing up  locally can literally be 1000 times faster – and local restore is about 300 times faster.  Important statistics to know when a server goes down.   Let’s take an example.  How long do you guess it would take to upload 100 Gigabytes of data if you had a fairly typical DSL with 3Mbps download/512Kbps upload speed?  According to the calculator (Refer to the bandwidth calculator link at the very bottom of this post) the FASTEST you could ever experience would be 19.5 days!  In reality you should assume 50 to 70% of this number (at least a 30% to 50% reduction) for real world connection overhead.  If you limit your backup to use the Internet connection only at night, or if your connection speed varies with traffic, it will take longer.


Speed calculator

These limitations are partly why Highly Reliable Systems still includes removable hard drives in their products.  The drives can be used to “speed seed” remote backups, or provide hybrid local/cloud solutions.  The same 100Gigabytes transferred to Direct Attached Storage (DAS) using USB3 or eSATA and a large hard drive or RAIDPac would take (best case) about 15 minutes.   Compare that to a NAS LAN transfer with Gigabit Ethernet and you could expect the transfer to slow down to 30 minutes (using an iSCSI connection).  So for fast backup using DAS and NAS are on the order of 1000 times faster than a 512Kbps upstream DSL line!  Notice that restoring the same 100 Gb of data would be a bit faster due to the 3Mbps downstream speed.  Perhaps as fast as 3.25 days (although you should probably increase this theoretical number by 30%)

Here are a few considerations when troubleshooting speed (and other) problems with cloud backup:

  • Asymmetric connection speeds.  Online backups require a fast Internet connection. The upload speed of your connection must be sufficient to support the transfer of daily changes.  Connections are often not symmetrical. DSL and cable typically have much slower upload versus download. For example a DSL line might have 3Megabits/second down and only 512 Kilobits/second up.  Your online backup speed will be dependent on the second, slower transfer rate.
  • Bandwidth Cost.  In some locations, the cost of buying a fast connection to the Internet is prohibitive or even impossible.  In these situations, a local removable drive or hybrid solution where “speed seed” can augment the cloud backup is needed.
  • Variable speed.  Many cable connections are “shared” with neighbors, meaning speeds can vary depending on the time of day and how much data people around you are using.  You will also experience problems if users on your local LAN are using the network or Internet heavily.  For example, if a user was watching a netflix movie, the upload performance might be impacted even though in theory downloading a movie should impact download.  Compressing and encrypting data (such as through a VPN connection) can sometimes add overhead to connections too.
  • Latency and Jitter.  Latency is the delay between the sender and the receiver decoding it, this is mainly a function of the signals travel time, and processing time at any nodes the information traverses.  Jitter causes variability in the signal speed.
  • Physical signaling overhead.  Not all of your advertised bitrate is available to you.  A certain amount of your bandwidth used by your modem and ISP in just in keeping the line open, but even bigger bandwidth loss occurs due to “packet overhead”.  On a network each message is wrapped in IP packets and then those are typically wrapped in TCP packets, which are wrapped in Ethernet frames.  Think of putting a letter inside an envelope, and then in a bigger envelope, and so on.  So there is a lot of wasted bits in just addressing where your data goes.
  • Internet productivity loss.  Be wary of backup impacting Internet performance during working hours.  Some cloud backups can be scheduled.  Others cannot.  Many examples occur where backups that are supposed to happen only at night impact the daily productivity of users.
  • Large file daily change.  If you are backing up very large individual files that change frequently, the upload speed may not be sufficient to back up such files nightly.  Video production houses often run into such problems.  In addition if if you are backing up a very large number of files, backup software may not be able to keep up or may even run out of resources and hang.
  • No Archival or long term storage. Cloud solutions usually do not provide a good way to Archive files.  Archiving includes the ability to retain older files (typically 1 year or more old) for access long term.  This long term retention need may be driven by laws such as HIPAA, Sarbanes Oxley, or lawsuits demanding older emails.  Note that saving data in the cloud for years and years would be prohibitively expensive.  we think it is better to store inactive data on removable hard drives.
  • Not a true Disaster Recovery solution. Online backups are not typically designed for full system restore. To quickly get full servers and systems back online, you should have local storage.  No one wants to wait a month for several hundred Gigabytes to download.
  • Beware wireless connections.  Wireless connections are subject to interference from Microwaves, wireless telephone handsets, rock walls, and even humans moving through the space.  All of this can make wireless connections a particularly bad choice for doing backups.
  • Failed backups.  Some cloud solutions give up if connections are temporarily unavailable.  Always check logs, and if possible have your software email you upon both success and failure of backup jobs to insure you really are getting the protection you’re paying for.

Use this link to get an idea what to expect out of your cloud backup solution.  Make sure to consider scheduling, as it is normal not to allow 100% connection usage during business hours.

What do you think?

Your email address will not be published. Required fields are marked *