Data Storage Threats in 2016 — Are Your Records Secure and Accessible Now... and in the Future?
Is the Legal Department Negligent in Responding to Current Information Technology and Court Rulings?
Dramatic changes are occurring for the organizations that experience litigation in the current climate of Electronic Records Laws. If management reads the legal journals you will be reminded that C-Level Management are required by the courts to plan for any and all litigation, the liability burden is squarely on management. There is a dramatic push to protect “Electronic Records” when in the past these types of digital files were not considered records but merely disaster recovery back up tapes.
Legal Departments must be prepared for E-Discovery1 in litigation and C-Level Management is demanding that the IT Platform and Records Management Platform develop a reconciliation for protection of electronic records and the process of providing a Datamap.
A Datamap captures institutional memory housed within the IT environment and makes that information immediately available, as is required to meet the court mandated 99 day requirement for presentation. The e-discovery data map should be a catalog of the organization’s electronic records. The data map should include: 1) a description of the information maintained; 2) a location of the backup media, 3) the online location of the originating application, the servers on which the metadata and electronic records are stored, 4) the location of any replicated backups such as the cloud or proprietary storage archives; and finally, 5) the electronic records and data retention policy for the original and replicated backups as well as discrepancies in practice noted during the discovery process.
Electronic records are considered different from paper documents because of its intangible machine readable format, volume, transience and persistence. Electronic records are typically accompanied by metadata that can play a critical role in ‘best evidence’ as well as establishing authenticity.
International Standards Organizations (ISO) and others such as The National Fire Protection Association (NFPA) have provided recognition that responds to court rulings about the requirement to protect electronic records and electronic data processing equipment. The use of ServerVaults, as well as Class 125 Media Vault Chambers to protect disk-drives, backup media have become mainstream as a means to protect electronic records in all formats.
The Electronic Record is a Ticking Time Bomb for Management & Legal Departments
Recently, the National Fire Protection Association requested discussions among various technical committees as to whom is responsible for protecting “Electronic Records”. The National Fire Protection Association Standard 232 is for the “Protection of Records” and is based on the simple destruction of records in paper, microfilm, or any format recognized by State and Federal Courts; which now includes electronic records. NFPA 75 is the “Standard for the Fire Protection of Information Technology Equipment.” The courts have placed an expectation that electronic information stored within servers and other electronic storage devices (machine readable) must be protected from destruction and spoliation with legal penalties and sanctions for failure to perform.
This NFPA 75 Standard covers the requirements for the protection of information technology equipment and information technology equipment areas from damage by fire, smoke, heat or water. Risk considerations include business interruption as well as the fire threat to the installation. With the extreme heat generated by blade servers, a prudent planner should also consider the risk of the data center or server room to the overall facility. This would be especially true in an organization where a fire in the data center would create a public relations nightmare for the organization. The courts have construed such loss of data as willful or negligent behavior and sanction businesses for spoliation of their electronic records as defined in Sarbanes-Oxley, Federal Rule 26 and the requirement for protecting “Electronically Stored Information” (ESI rulings).
Specific criteria of the NFPA 75 Standard — “Standard for the Fire Protection of Information Technology Equipment.” address construction requirements, materials and equipment, construction of information technology equipment, fire protection and detection equipment, records kept or stored in IT equipment rooms, utilities, and emergency and recovery procedures. The recognition of electronic records will place greater demands on information technology manufacturers and fire protection engineers to protect the server room and data center.
How Will the Courts Distribute Liability in 2016?
The courts have cast a wide net in requiring protection of records and electronic records, email and records stored “in the Cloud.” The computer manufacturers have enough experience with equipment failures and fires that have resulted in loss of electronic records that they now can be held culpable in the loss of records due to their design engineering failures. The server manufacturers have placed their emphasis on speed of processing and density of storage and failed to consider the risk to the electronic records.(e.g., Blade servers housed in high density cabinets create heat at a level that the air conditioning suppliers cannot adequately cool.) The perfect storm may be brewing for the providers of the Cloud with a known liability that is similar to asbestos litigation in the past.
I project judges will issue adverse rulings on the following in 2016:
- Computer equipment manufacturers will be held partially liable for loss of electronic records that occur due to fires in server rooms and data centers due to the defect in design of the servers in that they pose a significant risk due to their heat production and their failure to design for appropriate cooling required to run at peak efficiency.
- Cloud providers will be held liable for loss of records stored in their servers if they cannot be returned to the client for defense in litigation. The Cloud has offered an expectation that they will safely store the electronic records and meta data and Cloud service provider will have the capability to return all such information assets when required by the organization.
- Cloud providers have represented that clients no longer are required to maintain back up or business continuity sets to restore their data centers as the Cloud can perform that function. This will result in these providers being named in future litigation to share or avert E-Discovery costs and absorb liability.
- Cloud providers will be sanctioned by the courts for failing to meet the requirements of providing an accurate data map and appropriate E-Discovery in a timely fashion to the court’s demands.
- Cloud providers will be sanctioned for an inability to transfer stored data from one provider to another due to communication deficiencies.
- Law firms may be held liable for failing to advise clients of the risks of moving electronic records and disaster recovery platforms to the Cloud when they know the cost of E-Discovery will be enormous in these data storage environments and may impede the legal defense.
Predicting the Future?
The CEO, CFO and CIO (C-Level officers) and Risk Managers will soon recognize the latent issues of public Cloud storage describer above not as Black Swans2 but as uniform risks for all organizations. They will seek to develop a strategy to avert these identifiable risks due to the enormous exposure they represent.
The Cloud and the hardware manufacturers typically will not respond until the courts name hardware and service providers as a culpable in the loss of electronic records. Until that time, management must seek methods to protect their server room; and, the electronic records that reside on the equipment as well as on the back-up tapes for purposes of business continuity or business resumption. Managers can no longer take the approach that what resides on the electronic equipment are not records.
The NFPA 232 and 75 Standards are now consistent in their recognition and definition of electronic records as a record on information technology equipment or communications equipment (as defined by NFPA 75) or on off-line storage media such as computer tape, magnetic disk drives, and flash drives.
C-Level Managers are often surprised to learn of the following requirements in the National Fire Protection Association Standard-NFPA 75 Standard: “Standard for the Fire Protection of Information Technology Equipment.”
“6.2 Record Storage.
6.2.1 The amount of records within the information technology equipment room shall be kept to the absolute minimum required for essential and efficient operation.
22.214.171.124 Only records that are essential to the information technology equipment operations shall be permitted to be kept in the information technology equipment room.
126.96.36.199 An Automated Information Storage System (AISS) conforming to the requirements of 8.1.4 shall be permitted in the information technology equipment room.
8.1.4* Automated information storage system (AISS) units containing combustible media with an aggregate storage capacity of more than 0.76 m3 (27 ft3) shall be protected within each unit by an automatic sprinkler system or a gaseous agent extinguishing system with extended discharge.
9.2 Records Stored Outside the Information Technology Equipment Room.
9.2.1* All vital and important records shall be duplicated. Duplicated records shall be stored in a remote location that would not be exposed to a fire involving the original records. Records shall be stored in fire-resistive rooms in accordance with NFPA 232, Standard for the Protection of Records.”
When management from IT, Records Management and the C-Level discuss this in an intelligent manner, they realize that the Federal Courts place an interpretation on the electronic records that makes complying with this requirement difficult to achieve in a practical sense. Records within the electronic equipment are constantly changing, updating, and de-duplicating. Creation of an exact duplicate record is impossible in practice. The generating of a “Datamap” is an extremely difficult or impossible task depending on the complexity of the system.
To win in court on the electronic records battlefield, everyone must be totally prepared in advance. The method of storing and protecting records must be designed based on the absolute expectation that the technology and systems you use will be called into question in a court of law. Are you prepared to defend your methodology?
Records protection & electronic records protection are now synonymous. One cannot protect records without protecting the computer equipment, servers, disk drives, RAID Systems, back-up tapes, as well as mandating the storage from all “Bring Your Own Devices” (BYOD) by an intelligently designed and protected platform of Information Technology Equipment.
New Problems to Contend With...
The offsite storage industry is consolidating at a tremendous pace. Hundreds of offsite storage vendors have sold out to the large investor-backed national providers. Fees charged to remove records from offsite storage are often referred to as “Hostage Fees” have dramatically increased because corporations have no choices available in their market to keep rates competitive. Monthly storage fees are increasing and with Hostage Fees of $12.00 to $30.00 per box, no one can afford to remove their documents to seek out a new service provider.
Storing backup tapes in a Class 125 Media Vault was a commonly available feature offered by many of the offsite providers but the investor backed storage companies are not interested in offering superior security because they lack any real competition in the marketplace.
While business organizations have moved to the Cloud to escape rapidly escalating storage fees and onerous Hostage Fees designed to extort the client; the Offsite Public Cloud storage presents numerous problems as we have defined but the prime consideration is the lack of accountability in a public Cloud. The software agreements often forfeit ownership rights and waive liability for hacking and industrial espionage. (The service providers offered the storage with the implicit understanding that the data would be available to them for data mining and this was a prime driver in their cost benefit analysis.)
For this reason records are moving back on site. Server Vaults allow the organization to store their electronic records in a Server Room that can withstand physical intrusion while offering magnetic shielding, Class 125 Fire Protection, tornado and hurricane resistance and resist 4.0 to 7.0 levels of seismic activity.
Electronic Records have evolved as the Best Evidence repository. The genesis of the record’s life begins within the Server Room and its electronic equipment, the records can move to a silo or jukebox to archive the electronic records and the back-up tapes can be protected in small media-rated safes or Class 125 Media Vaults so that the electronic records remain protected within the owner’s site.
Concerns about Electro-Magnetic Pulse (EMP) weapons, solar flares, hacking and intrusion can be controlled within the owner’s protected ServerVault. In order to enforce a Fail Safe environment, the Owner must control every element of the data center, server rooms and the records center and archive.
Best Practice has been Totally Redefined
In 30 years, no server vault has failed, no Class 125 Media Rated Vault has failed and no offline back up tape has ever been hacked in the owner controlled vault model. Best of all, shortening the distance to the Server Room from the users eliminates the uncertain component: data transfer. Hardwiring from server to user cuts communication costs, eliminates a point of vulnerability and then using Media Vaulting to back up the platform eliminates storage costs. This translates to reduced E-Discovery costs in litigation.
Multiple data centers seemed like a good idea until the business owner experiences the cost of the duplicating hardware, the cooling costs, the utilities of running multiple platforms, and the cost of duplicate staffing. The addition of staff also creates another point of exposure to risk. An astute manager looking at the Total Cost of Ownership (TCO) or what some refer to as the “Endowed Cost of Long Term Storage” realizes that a much more secure and much less costly model exists.
This is accelerated by the fact that LTO3 tape density is increasing at an explosive rate. The amount of tape that needs to be stored is being reduced due to these ever-increasing capacities and makes tape backup storage onsite by the owner a better investment than ever before.
In addition, Linear Tape File System (LTFS) is revolutionizing the industry. When you mount an LTFS tape into your file system, it becomes visible as if it were a disk. This is improving the searchability of Electronic Records on the in-house equipment while at the same time reducing the costs.
E-Discovery is Changing the Mindset of Management and Their Legal Counsel
The most shocking day for many IT Professionals is when the CIO and Legal Counsel for your firm walk into the Data Center looking for a “Data Map” and want to schedule time for their E-Discovery needs. C-Level Managers are learning how important these E-Records are to the organization and this is driving more money to developing programs to protect these records.
Litigation “Holds” are extremely stressful and waiting until you are in the middle of litigation is the worst time to begin developing your program. The costs of a hasty attempt to create a datamap due to a management failure to prepare for potential litigation can exceed $250,000 due to the attorney time, forensic consultants, digital recovery consultants, records management consultants and your own staff’s diversion to this process.
When your organization saves an electronic document as a file; and, you move it to a desktop or storage server for an undetermined amount of storage time this now resides in the electronic archive, then it is in fact an electronic record. Your organization must be able to produce this electronic record and identifying metadata that identify the date of creation and other pertinent information to the records creation at an exact moment in time when E-Discovery calls for its production. This sounds easy until you have three attorneys and the CIO looking over your shoulder. And they ask to have you show where it is on the DataMap?
The Cloud Presents Risk in Ways Not Yet Contemplated...
There are a host of reasons to be concerned about Cloud Storage, but few have discussed these major concerns. Let’s take a long range view of the Cloud and where it will lead us.
- Digital Archives grow by 50% to 300% annually. The more heavily virtualized environments grow at higher level range.
- Digital information will increase 40 times in the next decade.
- The cost of Digital Storage over time in a format that is online is insupportable.4
- The cost of the cooling and power consumption are rising and are now significant in the ownership cost of the data.
- Given the tremendous increases in data storage, the Cloud becomes an extremely costly model over time. It is extremely costly to pull data out of the Cloud on a permanent basis.
- The vendors in this market have discovered that once clients are vested in an account, it is too expensive to leave so they no longer have an incentive to provide discounts or anything else.
- The client has no power over the hardware, software or even geographic location of their electronic records archive.
Much like Finding the Movie Video Buried in the Couch Cushion...
Remember years ago when you would find that video case that was three months past due and you would think how dumb it was to pay $400.00 for “Caddyshack” due to your unexpected late fees. Well just wait until you see the geometric growth in Cloud Storage costs.
In this rapidly growing universe of records, the client is losing leverage over the amount of records and the increasing costs of storage. The combination of accessing more records coupled with the storage costs makes this about as practical as being buried in a rented tuxedo. The client has lost control of the data in many of the service agreements and the vendor, not the owner, is making the decisions. If the storage vendor is unhappy with their returns, they can elect to no longer maintain the data. This is a long term a fool’s game.
The Cloud is not only impossible to protect, its true vulnerability is primarily from escalating costs and a business model designed to benefit the cloud operator not the client. There is eventually a total loss of control of the data itself. More importantly, the costs of E-Discovery will be crippling and force businesses to settle rather than endure the cost of performing the Datamap and Discovery process. (Has your attorney made you aware of this?)
The client with their complete tape library in self-controlled storage places more of the E-Discovery expense burden on the plaintiff; while they can go about their business functions. Should your Cloud vendor go out of business (my prediction: 70% will fold or be acquired by another company in the coming years) then you have very difficult decisions. This is not as simple as moving your backup tapes to another Media Vault storage vendor.
Is your legal counsel making you aware of these developments in the world of protecting the organization from litigation and defining real world risks? Who is looking at the big picture for your organization? Dramatic changes have occurred and a discussion about a world affected by electronic records should be on your 2016 agenda.
The court system is becoming more abusive to those organizations that cannot efficiently produce their E-Discovery product; and at the same time the IT managers have moved to high risk and low reward platforms. We live in a world of dramatically increasing risks of hackers, malware, sabotage and terrorism; yet management and their legal department are not recognizing the dangerous landscape that now surrounds them.
In 2016 many will wake up and see the Black Swan is now a dangerous dragon.
- E-Discovery is the process of producing electronic records by accurately identifying, collecting and producing electronically stored information (ESI) along with a court defined ‘Datamap’ in response to a request for production in litigation or an investigation. The Datamap identifies the information assets that must be placed in a legal hold and preserved with utmost protection. ESI includes, but is not limited to, emails, documents, presentations, databases, voicemail, audio and video files, social media, and web sites.
- Black Swans are defined as low probability risks that potentially deliver a high impact loss
- The term an abbreviation for “linear tape open” refers to an open format where users have access to multiple sources of storage media products that will be compatible. The high-capacity, single-reel implementation of LTO technology which continues to increase in density, encryption technology and “drag & drop” capabilities.
- The Economics of Long-Term Digital Storage; Copyright 2012 David S. H. Rosenthal, https://www.fsl.cs.sunysb.edu/docs/unesco12/UNESCO2012-storage-econ.pdf