Manufacturers rely upon data to manage capacity, quality and on time delivery. As the volume of data grows, so does the importance of system uptime to maintain performance expectations. Data loss and downtime not only impact budget, business interruption can tarnish reputations and sever relationships.
When creating a security policy for your organization, one of the most important considerations that must be made relates to backup strategy and backup policy that your company will implement in the case of any cybersecurity loss event. Having a backup policy that you can rely on not only ensures that your business can become operational after an incident, but allows an opportunity for manufacturers to recover important documents in the event of human error. As new forms of ransomware and wiperware continue to attack small businesses and manufacturing companies, many that are affected find that they cannot continue business operations as vital business data is left unrecoverable due to not having an effective backup policy initially created.
One excerpt from Verizon’s 2019 Data Breach Investigations Report suggests that manufacturers are becoming more relevant targets to cyber attacks, “Manufacturing has been experiencing an increase in financially motivated breaches in the past couple of years, but espionage is still a strong motivator. Most breaches involve phishing and the use of stolen credentials.” They go on to disclose, “For the second year in a row, financially motivated attacks outnumber cyber-espionage as the main reason for breaches in Manufacturing, and this year by a more significant percentage (40% difference). If this were in most any other vertical, it would not be worth mentioning as money is the reason for the vast majority of attacks. However, Manufacturing has experienced a higher level of espionage-related breaches than other verticals in the past few years.”
Throughout this article, we will explain why having a backup policy is important, what is needed to create an effective backup policy, and cover the 3-2-1 rule for backups. We will also cover additional backup terminology, and cover disaster recovery as a service and how disaster recovery as a service is different from using backups.
Why is a backup policy important?
When seeking guidance for a backup policy, business owners and executives often trust the provider they pay to develop and implement an effective backup policy. On the surface, that seems logical.
Backup policies are actually data insurance policies, and not merely an item on a checklist. Why do we make this distinction? Manufacturing business owners continue to experience business interruption and financial loss because they believe they are protected. They pay their invoice each month and are told they are protected. These unverified claims go undetected, until there is a dire need to restore, typically from hardware failure or ransomware. When the provider reports that the data was not backed up and therefore, not recoverable, who is impacted the most by this discovery? As your provider walks away, you are left scrambling to keep your business alive and reputation intact.
What is an effective backup policy?
When developing an effective backup policy, manufacturers and business owners should focus on five major points, design a system that fulfills your strategy for backup and recovery, develop a schedule of how often backups are performed, documentation for all secure data backup and restore procedures, create a matrix indicating how long backups should be retained for recovery, and logs of scheduled restore test results.
When creating an effective backup policy, implementing a system that will allow you to execute your backup strategy for protecting and restoring data from accidental loss or corruption is fundamental. Various backup products offer manufacturers different levels of protection. Features and functions can also vary across vendors, but there are a number of mistakes we continue to see business owners, executives, and sometimes IT personnel make when protecting their data. One such error the we commonly observe is the belief that the backup copies everything. Once an incident occurs when the backup is needed, the manufacturer is left empty handed as their backup service was configured to a default state and didn’t include business essential data. These mistakes continue to be disruptive and costly to many business owners today.
These people, with good intentions, believe all backups are essentially the same, so they choose a product that has the lowest cost. They move forward with an installation of the software, backup some files, and feel good about their progress. They now believe that this incomplete process fulfills their duty of care and will provide the ability to recover data from accidental loss or corruption.
The scenario identified above is the most common that we see from manufacturers who struggle from prolonged downtime. They misunderstand the key points to effective backup and recovery. We are not backing up files for the sake of backing up, we are backup up files in order to restore data from accidental loss or corruption. The companies that choose one of many cloud backup providers quickly learn why these backup programs are cheap as high failure rates and slow restore speeds are not predictable for manufacturing businesses needs for recovery.
The next major focus of creating an effective backup policy is creating a schedule of when backups should be performed. Simple in theory, but more complex when you are focused on recovery. Let me explain. We start with an inventory of business services, applications, systems, and data. Small manufacturers have fewer critical applications and systems that support their operation, when compared to mid-sized manufacturers. That makes this process much faster.
Whether you need to protect and recover data on physical or virtual servers in a data center, regional office, or in the cloud, for small or large IT environments, you must manage to the RTO and RPO your business demands. These priorities are approved by executive management who assign capital to offset the cost of downtime.
- RTO is Recovery Time Objective. From when the incident occurs, how long to recover?
- RPO is Recovery Point Objective. From when the incident occurs, how much data is lost?
Once this is answered for all systems, you now have a schedule for your backups. For small manufacturers, this typically means that the amount of data being created and changed each day is significantly less than a mid-sized manufacturer. This means that a small manufacturer could attain the same RTO and RPO as their mid-sized competitor, but for a smaller investment.
The third major focus of creating an effective backup policy is documentation for all secure data backup and restore procedures. This runbook of documentation would outline installation procedures for backing up new systems, it would also outline the RTO and RPO expectations for certain types of data residing on certain types of systems, locations of backup data, and restore procedures. Documentation allows other people less familiar with the system, to perform critical functions of backup and recovery. This reduces risk of prolonged downtime for manufacturers when incidents occur. The documentation should also be regularly reviewed and updated to match any changes in the backup platform or software used.
The documentation should also cover the procedures that are to be followed when either creating backups or restoring a system, as following the backup procedure ensures that backups effectively usable and hold the companies standard for what is to be included in the backup, while also providing clarity on how to properly restore a system and ensure that the recovery is successful. Backup and recovery procedures can vary based on what service you are using to create backups and what environment you are using to store the backup, but should always have a reference to the service policy number and the contact number to the provider in the case of emergency or if an error is encountered. The Backup procedure documentation should include how to create a backup that follows the company standards, instructions on how to package and store backup media that is stored offline, how to migrate backup data to the cloud if your business utilizes a cloud storage environment, and how to properly log and record backups and any errors encountered. The items that should be included for the recovery procedure documentation should include how to safely shut down a machine in the case of hardware failure, how to properly wipe a machine in the case of a ransomware attack, how to retrieve or access the backups, and how to restore the system if additional software is needed from the service provider.
The next focus area is to create a matrix indicating how long backups should be retained for recovery, also known as a data retention policy. Your retention policy defines how long backups from each system would be kept within the backup system. Some backup retention policies may call for certain systems to be archived after a certain period of time, while other manufacturers keep many years of backups on-line and ready for restore. When considering your retention needs, we also encourage manufactures to consider the 3-2-1 rule. It is a common approach to keeping your data safe in almost any failure scenario. The rule is: keep at least three (3) copies of your data, store two (2) backup copies on different storage media, with one (1) of them located offsite. We will discuss this concept in greater detail later in this article.
Retention is important, as it provides various points that systems and documents can be recovered from. In fact, we have successfully restored numerous files from extended retention policies. In one case, contract documents from 5 years earlier were accidentally purged from other systems. A discovery for a lawsuit became the motivation to find these files. Due to the businesses data retention policy, we were able to able to recover the set of contract files, which prevented a lawsuit and save our client tens of thousands of dollars.
The final major focus of creating a log of scheduled restore test results to ensure the data being backed up is recoverable. An effective backup policy should include performing periodic tests to measure the effectiveness of the backups, and to identify and address any discovered issues. Regular testing and reporting on backup effectiveness will help business owners build true confidence in their backup strategy, and help create a realistic timeline of how much time is needed to restore critical systems after an incident occurs. Did the test meet your RTO and RPO expectations? It is better to find out during testing your backups than when your business is offline.
Understanding the 3-2-1 rule for backups
When implementing a backup policy for your business, manufacturers are encouraged to follow what is known as the 3-2-1 rule. The 3-2-1 rule is a backup strategy that recommends having three copies of your backups, having two of those backup copies store on different storage mediums, and having one copy that is stored offsite. While the concept of the 3-2-1 rule can be confusing, our expansion upon the insight offered by Quest will help explain the 3-2-1 rule can help simplify this concept.
Rule number one: Having three backup copies
Having at least three copies of your data (one live + two backups) provides manufacturers an increased success rate of being able to recover. Included with having a higher success rate of being able to recover, having three copies of your data provides reassurance during the chance of a loss event, such as a fire, ransomware attack, data corruption, or accidental deletion, that at least one copy of business data is not affected. Several of these events often result in at least one copy of your data being destroyed.
Rule number two: Having backups stored on two different storage mediums
Rule number two focuses more on how the backup data is stored, and what media is used to store it. Having multiple backups stored on the same storage medium can allow the opportunity that a single event will affect or destroy the multiple copies of you backup. When considering the different mediums that your backups are stored on, manufacturers need to consider the cost of the medium, the rate of data transfer to and from the medium, and the expected lifespan of the storage media. Additionally, manufacturers should also consider the environment of where these storage mediums are to be stored, as heat and moisture can shorten the life span of the storage mediums. Popular storage mediums include external hard drives, magnetic storage tape (we do not suggest), iSCSI storage drives, or the using a cloud environment.
Rule number three: Having a copy stored offsite
Having a copy of your backup stored offsite offers peace of mind if a local event were to affect your business. Fires, ransomware, and insider threats can result in the destruction of the locally stored copies. Offsite backup copies can include a storage facility where physical copies of backups are stored, data centers, and cloud environments managed by the backup service provider. While having an offsite copy of your data should be considered a last failsafe for data recovery, one consideration that must be identified is how long it would take to retrieve the offsite data. Cloud environments offer accessibility to backups to businesses who have access to the Internet, but are often extremely slow for large restores. Backups stored on physical mediums may require time for the data to be packed and shipped to the data owner. Depending upon the size of the data volumes to be recovered, shipping recoverable data on a physical device can accelerate recovery efforts and bring your business back on-line in less time.
Guidelines for securing your backups
Manufacturers and producers should also keep in mind that while their backup data can aid in business continuity, attackers can also target these backups for destruction and data theft. The one method to ensure that your backups are secure from data theft is to encrypt your backups. In the case that attackers can access your backups, hackers and cyber criminals can sometimes read the data from backups. To prevent attackers from reading the data stored in backups, manufacturers and producers can use encryption to further protect the data stored in the backups. Keep in mind that if you apply encryption to your backups, that you must keep the keys used to encrypt and decrypt your backups in a safe environment that can be accessible offline, and limit who has access to these keys. Keeping these keys in an environment accessible from offline access protects the keys from being stolen or encrypted during a ransomware attack, and limiting access to these keys can prevent malicious insiders from accessing the data stored in the backups.
Another method that can help secure business backups is to control access to the location of where backups are stored and conducted. Allowing any employee access to the location where backups are stored can increase the chances of that backups can be altered or destroyed, whether accidental or intentional. During the creation of the backup and recovery policy, a list of personnel should be created to identify who is permitted to interact and access the locations where backups are stored, and any onsite systems that are used for creating backups. Additionally, the areas that backups are stored and the systems used for creating backups should be secured to prevent access from unauthorized personnel. This should include a method to track employees that access the location where the backups are stored, and video surveillance to verify who had accessed the systems during a given time.
Other backup terms you are likely to hear
What is a full backup?
A full backup is a complete backup of all files on a designated system. This includes the operating system, applications, and all data that exists on a single machine. Since full backups contain more data, these backups require more time to perform and restore from.
What is incremental backup?
An incremental backup copies only the data that changed, since the last backup operation. This means that a smaller amount of data is being copied per system, so the backups complete much faster. Incremental backups are known to take longer to restore from, as the restore uses that last full backup created plus the incremental backups that were captured.
What is differential backup?
A differential backup is similar to an incremental backup the first time it is performed, in that it will copy all data changed from the previous backup. However, each time it is run afterwards, it will continue to copy all data changed since the previous full backup. Thus, it will store more data than an incremental on subsequent operations, although typically far less than a full backup. Moreover, differential backups require more space and time to complete than incremental backups, although less than full backups.
What is incremental vs differential backup?
Incremental backup only includes the data that has changed since the previous backup operation. Differential backup contains all of the data that has changed since the last full backup.
What are snapshots?
Use incremental forever snapshots to capture an entire application and its relevant state, for complete application and system recovery with near zero RTOs and aggressive RPOs. Some systems allow for application-consistent protection for Microsoft Exchange, Microsoft SQL, Oracle, etc., including virtual environments running VMware vSphere and Microsoft Hyper-V.
What is data deduplication?
Data deduplication is a process that eliminates redundant copies of data and reduces storage overhead. Data deduplication techniques ensure that only one unique instance of data is retained on storage media, such as disk, flash or tape. Data deduplication is used for backups to reduce the amount of storage space used, and to remove duplicate copies of files and applications.
What is replication?
Data replication is the creation of a copy or multiple copies of your backups, and are stored to one or more target locations. These locations can include on premise, remote office, public cloud, or private cloud. Data replication is different from a backup as the data from replication is still accessible and are actively usable, where data backups are copies of data that are not actively accessible to the system.
Disaster Recovery as a Service
Disaster Recovery as a Service, or DRaaS, is defined by Techopedia as “a cloud computing and backup service model that uses cloud resources to protect applications and data from disruption caused by disaster.” Disaster Recovery as a Service offers businesses more than just a backup service of your data, but instead offers businesses a cloud-based platform to be used as a replica of the primary work environment. DRaaS is often considered as part of an organization’s business continuity plan because of the ability for an organization to continue business daily business operations while the repairs are made to primary systems regularly used. While DRaaS also uses backups in its services, businesses are often confused as to why they should still incorporate a backup policy when using DRaaS, or what the difference is between creating an effective backup policy and using DRaaS. In the following section, we will explain the difference between backups and Disaster Recovery as a Service.
Understanding the difference between backups and disaster recovery as a service
Businesses often confuse backup services and Disaster Recovery as a Service, but don’t understand why they are offered as separate services. Data backups are a service that is specialized in creating copies of data that can be used in the case where only the data is affected, but the system is still operational. For example, if a business were to experience a ransomware attack, while the data on those systems may be encrypted, the affected systems can be wiped and have the data restored from a recent backup.
Compared to backups, Disaster Recovery as a Service, also known as DRaaS, is a cloud-based computing and backup service model that uses cloud resources to protect applications and data from disruption caused by disaster. This provides organizations a total system backup, which is replicated to a secondary infrastructure, and allows for business continuity in the event of system failure. DRaaS is often offered in conjunction with a disaster recovery plan (DRP) or business continuity plan (BCP).
With DRaaS, files are not only committed to local backups, but entire server compute environments are replicated to a different location. This allows operations to failover and continue supporting business operations in the case that the data or servers affected. An example for the use of Disaster Recovery would be in the case that a facility is severely damaged or destroyed during a fire, both the original data and the critical systems are no longer usable. In this case, the use of Disaster Recovery as a Service would be used to allow the business to continue its normal office operations, although manufacturing and shipping would still be offline. Simply put, backups can be used to create points of recovery for entire applications, servers, and data sets, while Disaster Recovery creates an environment where business applications, servers, and data sets can be brought on-line (failover) to resume operations related to the critical services provided by those selected systems.
Who is our backup provider?
Choosing a backup service provider is much trickier than it may seem. There are many vendor options and a multitude of providers offering backup services. Business owners must consider the capabilities and limitations of the services being offered and alignment with your backup strategy. We acknowledge that you are in a tough position. You don’t know, what you don’t know. You need help and want to trust that someone will be honest and help you make the right decisions.
Understanding your business, the environment, your processes, the amount of data that needs protected, your desired RTO and RPO, cannot be quoted over the phone. If this is how you choose to proceed, we wish you the best in your journey.
For those manufacturers that have a sensitivity to downtime, we might be a good fit, even if you want to self-perform or share responsibilities with the installation and deployment. While the marketing of backup and recovery services appear great on paper, the real-world function and recovery can fail to meet expectations. Imagine being the person making the decision to devote capital to a backup and recovery solution, and it fails to meet expectations during a downtime event. These are avoidable career altering decisions. You do not have to make decisions in a vacuum.
Creating an effective backup strategy and backup policy can be complicated, but with the right assistance, accomplishing this work can be less complex and time consuming. Certitude Security™ focuses on protecting manufacturers from injustice, which includes cyber criminals and self-harm. If you desire increased predictability in being able to quickly recover from data loss events that cause business interruption, visit our website today to speak with a team member about your business needs.