Manufacturers rely upon data to manage capacity, quality, and on-time delivery. As the volume of data grows, so does the importance of system uptime to maintain performance expectations. Data loss and downtime not only impact the budget, but business interruption can also tarnish reputations and sever relationships.
When creating a security policy for your organization, one of the most important considerations that must be made relates to the backup strategy and backup policy that your company will implement in the case of any cybersecurity loss event. Having a backup policy that you can rely on not only ensures that your business can become operational after an incident but allows an opportunity for manufacturers to recover important documents in the event of human error. As new forms of ransomware and wiperware continue to attack small businesses and manufacturing companies, many affected find that they cannot continue business operations as vital business data is left unrecoverable due to not having an effective backup policy initially created.
One excerpt from Verizon’s 2019 Data Breach Investigations Report suggests that manufacturers are becoming more relevant targets to cyber attacks, “Manufacturing has been experiencing an increase in financially motivated breaches in the past couple of years, but espionage is still a strong motivator. Most breaches involve phishing and the use of stolen credentials.” They go on to disclose, “For the second year in a row, financially motivated attacks outnumber cyber-espionage as the main reason for breaches in Manufacturing, and this year by a more significant percentage (40% difference). If this were in most any other vertical, it would not be worth mentioning as money is the reason for the vast majority of attacks. However, Manufacturing has experienced a higher level of espionage-related breaches than other verticals in the past few years.”
Throughout this article, we will explain why having a backup policy is important, what is needed to create an effective backup policy, and cover the 3-2-1 rule for backups. We will also cover additional backup terminology and cover disaster recovery as a service and how it differs from using backups.
Why is a backup policy important?
When seeking guidance for a backup policy, business owners and executives often trust the provider they pay to develop and implement an effective backup policy. On the surface, that seems logical.
Backup policies are actually data insurance policies and not merely an item on a checklist. Why do we make this distinction? Manufacturing business owners continue to experience business interruption and financial loss because they believe they are protected. They pay their invoice each month and are told they are protected. These unverified claims go undetected until there is a dire need to restore, typically from hardware failure or ransomware. When your IT provider informs the data was not backed up and, therefore, not recoverable, who is impacted the most by this discovery? As your provider walks away, you are left scrambling to keep your business alive and reputation intact.
What is an effective backup policy?
When developing an effective backup policy, manufacturers and business owners should focus on five major points, design a system that fulfills your strategy for backup and recovery, develop a schedule of how often backups are performed, documentation for all secure data backup and restore procedures, create a matrix indicating how long backups should be retained for recovery, and logs of scheduled restore test results.
When creating an effective backup policy, implementing a system that will allow you to execute your backup strategy for protecting and restoring data from accidental loss or corruption is fundamental. Various backup products offer manufacturers different levels of protection. Features and functions can also vary across vendors. Still, we continue to see several mistakes: business owners, executives, and sometimes IT personnel make when protecting their data. One such error we commonly observe is the belief that the backup copies everything. Once an incident occurs when the backup is needed, the manufacturer is left empty-handed. Their backup service was configured to a default state and didn’t include business essential data. These mistakes continue to be disruptive and costly to many business owners today.
These people, with good intentions, believe all backups are essentially the same, so they choose a product that has the lowest cost. They move forward with installing the software, backup some files, and feeling good about their progress. They now believe that this incomplete process fulfills their duty of care and will provide the ability to recover data from accidental loss or corruption.
The scenario identified above is the most common that we see from manufacturers who struggle from prolonged downtime. They misunderstand the key points to effective backup and recovery. We are not backing up files for the sake of backing up; we backup files to restore data from accidental loss or corruption. The companies that choose one of many cloud backup providers quickly learn why these backup programs are cheap. High failure rates and slow restore speeds are not predictable for manufacturing businesses’ needs for recovery.
The next major focus of creating an effective backup policy is creating a schedule of when backups should be performed. Simple in theory, but more complex when you are focused on recovery. Let me explain. We start with an inventory of business services, applications, systems, and data. Small manufacturers have fewer critical applications and systems that support their operation when compared to mid-sized manufacturers. That makes this process much faster.
Whether you need to protect and recover data on physical or virtual servers in a data center, regional office, or in the cloud, you must manage to the RTO and RPO your business demands for small or large IT environments. These priorities are approved by executive management, who assign capital to offset the cost of downtime.
- RTO is Recovery Time Objective. From when the incident occurs, how long to recover?
- RPO is Recovery Point Objective. From when the incident occurs, how much data is lost?
Once this is answered for all systems, you now have a schedule for your backups. For small manufacturers, this typically means that the amount of data being created and changed each day is significantly less than a mid-sized manufacturer. This means that a small manufacturer could attain the same RTO and RPO as their mid-sized competitor, but for a smaller investment.
The third major focus of creating an effective backup policy is the documentation for all secure data backup and restore procedures. This runbook of documentation would outline installation procedures for backing up new systems. It would also outline the RTO and RPO expectations for certain types of data residing on certain systems, locations of backup data, and restore procedures. Documentation allows other people less familiar with the system to perform critical functions of backup and recovery. This reduces the risk of prolonged downtime for manufacturers when incidents occur. The documentation should also be regularly reviewed and updated to match any backup platform or software changes.
The documentation should also cover the procedures that are to be followed when either creating backups or restoring a system, as following the backup procedure ensures that backups are effectively usable and hold the companies standard for what is to be included in the backup, while also providing clarity on how to properly restore a system and ensure that the recovery is successful. Backup and recovery procedures can vary based on what service you are using to create backups and what environment you are using to store the backup, but should always have a reference to the service policy number and the contact number to the provider in the case of emergency or if an error is encountered. The Backup procedure documentation should include how to create a backup that follows the company standards, instructions on how to package and store backup media that is stored offline, how to migrate backup data to the cloud if your business utilizes a cloud storage environment, and how to properly log and record backups and any errors encountered. The recovery procedure documentation should include how to safely shut down a machine in the case of hardware failure, how to properly wipe a machine in the case of a ransomware attack, how to retrieve or access the backups, and how to restore the system if additional software is needed from the service provider.
The next focus area is to create a matrix indicating how long backups should be retained for recovery, also known as a data retention policy. Your retention policy defines how long backups from each system would be kept within the backup system. Some backup retention policies may call for certain systems to be archived after a certain period of time. At the same time, other manufacturers keep many years of backups on-line and ready for restore. When considering your retention needs, we also encourage manufacturers to consider the 3-2-1 rule. It is a common approach to keeping your data safe in almost any failure scenario. The rule is: keep at least three (3) copies of your data, store two (2) backup copies on different storage media, with one (1) of them located offsite. We will discuss this concept in greater detail later in this article.
Retention is important, as it provides various points that systems and documents can be recovered from. In fact, we have successfully restored numerous files from extended retention policies. In one case, contract documents from 5 years earlier were accidentally purged from other systems. The discovery from a lawsuit became the motivation to find these files. Due to the business’s data retention policy, we were able to recover the set of contract files, which prevented a lawsuit and save our client tens of thousands of dollars.
The final major focus of creating a log of scheduled restore test results to ensure the data being backed up is recoverable. An effective backup policy should include performing periodic tests to measure the backups’ effectiveness and identify and address any discovered issues. Regular testing and reporting on backup effectiveness will help business owners build true confidence in their backup strategy and help create a realistic timeline of how much time is needed to restore critical systems after an incident occurs. Did the test meet your RTO and RPO expectations? It is better to find out during testing your backups than when your business is offline.
Understanding the 3-2-1 rule for backups
When implementing a backup policy for your business, manufacturers are encouraged to follow what is known as the 3-2-1 rule. The 3-2-1 rule is a backup strategy that recommends having three copies of your backups, having two of those backup copies stored on different storage mediums, and having one copy stored offsite. While the concept of the 3-2-1 rule can be confusing, our expansion upon the insight offered by Quest will help explain the 3-2-1 rule can help simplify this concept.
Rule number one: Having three backup copies
Having at least three copies of your data (one live + two backups) provides manufacturers an increased success rate of recovering. Included with having a higher success rate of being able to recover, having three copies of your data provides reassurance during the chance of a loss event, such as a fire, ransomware attack, data corruption, or accidental deletion, that at least one copy of business data is not affected. Several of these events often result in at least one copy of your data being destroyed.
Rule number two: Having backups stored on two different storage mediums
Rule number two focuses more on how the backup data is stored and what media is used to store it. Having multiple backups stored on the same storage medium can allow the opportunity that a single event will affect or destroy the multiple copies of your backup. When considering the different mediums that your backups are stored on, manufacturers need to consider the medium’s cost, the rate of data transfer to and from the medium, and the expected lifespan of the storage media.
Additionally, manufacturers should also consider the environment of where these storage mediums are to be stored, as heat and moisture can shorten the storage medium’s life span. Popular storage mediums include external hard drives, magnetic storage tape (we do not suggest), iSCSI storage drives, or using a cloud environment.
Rule number three: Having a copy stored offsite
Having a copy of your backup stored offsite offers peace of mind if a local event affects your business. Fires, ransomware, and insider threats can destroy locally stored copies. Offsite backup copies can include a storage facility where physical copies of backups are stored, data centers, and cloud environments managed by the backup service provider.
While having an offsite copy of your data should be considered a last failsafe for data recovery, one consideration must be identified how long it would take to retrieve the offsite data. Cloud environments offer accessibility to backups to businesses who have access to the Internet but are often extremely slow for large restores.
Backups stored on physical mediums may require the data to be packed and shipped to the data owner. Depending upon the data volumes’ size to be recovered, shipping recoverable data on a physical device can accelerate recovery efforts and bring your business back online in less time.
Guidelines for securing your backups
Manufacturers and producers should also keep in mind that while their backup data can aid in business continuity, attackers can also target these backups for destruction and data theft. The one method to ensure that your backups are secure from data theft is to encrypt your backups. If attackers can access your backups, hackers and cyber criminals can sometimes read the data from backups. To prevent attackers from reading the data stored in backups, manufacturers and producers can use encryption to protect the data stored in the backups.
If you apply encryption to your backups, you must keep the keys used to encrypt and decrypt your backups in a safe environment that can be accessed offline and limit who has access to these keys. Keeping these keys in an environment accessible from offline access protects the keys from being stolen or encrypted during a ransomware attack. Limiting access to these keys can prevent malicious insiders from accessing the data stored in the backups.
Another method that can help secure business backups is to control access to where backups are stored and conducted. Allowing any employee access to the location where backups are stored can increase the chances that backups can be altered or destroyed, whether accidental or intentional. During the creation of the backup and recovery policy, create a list of personnel permitted to interact and access the locations where backups are stored and any onsite systems used to create backups.
Additionally, backups are stored, and the systems used for creating backups should be secured to prevent unauthorized personnel access. This should include a method to track employees that access the location where the backups are stored and video surveillance to verify who had accessed the systems during a given time.
Other backup terms you are likely to hear.
What is a full backup?
A full backup is a complete backup of all files on a designated system. This includes the operating system, applications, and all data that exists on a single machine. Since full backups contain more data, these backups require more time to perform and restore from.
What is incremental backup?
An incremental backup copies only the data that changed since the last backup operation. This means that a smaller amount of data is being copied per system, so the backups complete much faster. Incremental backups are known to take longer to restore, as the restore uses that last full backup created plus the incremental backups captured.
What is differential backup?
A differential backup is similar to an incremental backup the first time it is performed in that it will copy all data changed from the previous backup. However, each time it is run afterward, it will continue to copy all data changed since the previous full backup. Thus, it will store more data than an incremental on subsequent operations, although typically far less than a full backup. Moreover, differential backups require more space and time to complete than incremental backups, although less than full backups.
What is incremental vs. differential backup?
Incremental backup only includes the data that has changed since the previous backup operation. A differential backup contains all of the data that has changed since the last full backup.
What are snapshots?
Use incremental forever snapshots to capture an entire application and its relevant state for complete application and system recovery with near-zero RTOs and aggressive RPOs. Some systems allow for application-consistent protection for Microsoft Exchange, Microsoft SQL, Oracle, etc., including virtual environments running VMware vSphere and Microsoft Hyper-V.
What is data deduplication?
Data deduplication is a process that eliminates redundant copies of data and reduces storage overhead. Data deduplication techniques ensure that only one unique instance of data is retained on storage media, such as disk, flash, or tape. Data deduplication is used for backups to reduce the amount of storage space used and remove duplicate copies of files and applications.
What is replication?
Data replication is creating a copy or multiple copies of your backups, and they are stored in one or more target locations. These locations can include on-premise, remote office, public cloud, or private cloud. Data replication is different from a backup as the data from replication is still accessible and is actively used. Data backups are copies of data that are not actively accessible to the system.
Disaster Recovery as a Service
Disaster Recovery as a Service, or DRaaS, is defined by Techopedia as “a cloud computing and backup service model that uses cloud resources to protect applications and data from the disruption caused by the disaster.” Disaster Recovery as a Service offers businesses more than just a backup service of your data. Instead, it offers businesses a cloud-based platform to be used as a replica of the primary work environment.
DRaaS is often considered part of an organization’s business continuity plan because it continues business daily business operations. At the same time, the repairs are made to primary systems regularly used. While DRaaS also uses backups in its services, businesses are often confused about why they should still incorporate a backup policy when using DRaaS or the difference between creating an effective backup policy and using DRaaS. In the following section, we will explain the difference between backups and Disaster Recovery as a Service.
Understanding the difference between backups and disaster recovery as a service
Businesses often confuse backup services and Disaster Recovery as a Service but don’t understand why they are offered separate services. Data backups are a service that is specialized in creating copies of data to be used in the case where only the data is affected, but the system is still operational. For example, if a business were to experience a ransomware attack, while the data on those systems may be encrypted, the affected systems can be wiped and restored from a recent backup.
Compared to backups, Disaster Recovery as a Service, also known as DRaaS, is a cloud-based computing and backup service model that uses cloud resources to protect applications and data from the disruption caused by the disaster. This provides organizations a total system backup, which is replicated to a secondary infrastructure and allows for business continuity in system failure. DRaaS is often offered in conjunction with a disaster recovery plan (DRP) or business continuity plan (BCP).
With DRaaS, files are committed to local backups, but entire server compute environments are replicated to a different location. This allows operations to failover and continues supporting business operations if the data or servers are affected. An example for the use of Disaster Recovery would be if a facility is severely damaged or destroyed during a fire, both the original data and the critical systems are no longer usable. In this case, Disaster Recovery as a Service would be used to allow the business to continue its normal office operations. However, manufacturing and shipping would still be offline. Simply put, backups create points of recovery for entire applications, servers, and data sets. In contrast, Disaster Recovery creates an environment where business applications, servers, and data sets can be brought on-line (failover) to resume operations related to the critical services provided by those selected systems.
Who is our backup provider?
Choosing a backup service provider is much trickier than it may seem. There are many vendor options and a multitude of providers offering backup services. Business owners must consider the capabilities and limitations of the services being offered and align with your backup strategy. We acknowledge that you are in a tough position. You don’t know what you don’t know. You need help and want to trust that someone will be honest and help you make the right decisions.
Understanding your business, the environment, processes, and amount of data that needs protecting, your desired RTO and RPO cannot be quoted over the phone. If this is how you choose to proceed, we wish you the best in your journey.
For manufacturers with sensitivity to downtime, we might be a good fit, even if you want to self-perform or share responsibilities with the installation and deployment. While the marketing of backup and recovery services appears great on paper, the real-world function and recovery can fail to meet expectations. Imagine being the person deciding to devote capital to a backup and recovery solution, and it fails to meet expectations during a downtime event. These are avoidable career-altering decisions. You do not have to make decisions in a vacuum.
Creating an effective backup strategy and backup policy can be complicated, but with the right assistance, accomplishing this work can be less complex and time-consuming. Certitude Security™ focuses on protecting manufacturers from injustice, which includes cyber criminals and self-harm. If you desire increased predictability in quickly recovering from data loss events that cause business interruption, visit our website today to speak with a team member about your business needs.