Lost Data Costs Money!

Hitachi Systems Storage Index sponsored an independent survey involving 840 CIO interviews in 20 countries. The result shows that data availability matches business continuity as a primary concern for 81 percent of CIOs.

However, according to a report on data loss in Europe¹, more than six percent of PCs will suffer data loss in any year — a total of 1.7 million incidents. And there is no reason to think the position in North America (or anywhere else in the world) is any better.

Hitachi’s report identified six main causes of data loss:

• Hardware failure, including damage by power surge and hard drive failure (42 percent)
• Human error, including accidental deletion (31 percent)
• Software corruption (13 percent)
• Virus (seven percent: a survey by security firm ICSA.net reported that 40 percent of companies reported data loss due to viruses)
• Theft, especially laptop theft (five percent: According Securityfocus, surveys indicate that about 10 to 15 percent of laptops are stolen by criminals intent on selling the data)
• Hardware destruction (including floods, fires, lightning, and power failure), three percent

Each incident will have one of two outcomes: successful recovery of data or permanent loss. Calculating the cost of data loss considers each of these possibilities.

Data recovery experts claim that in 80 percent of cases, lost data can be restored - at a cost. Even if the recovery is by an in-company support engineer, the cost of their salary averages out at around $35/hour. Typically, data recovery takes around six hours - a cost of $210. Using an external data recovery company increases this cost to $420, on average.

However, a more significant cost is the cost of lost productivity by the user, impacting sales and profitability. The salary cost of lost productivity is around $32.50 - an average hiding huge differences. Using this average, the six-hour loss of productivity adds another $195.

If we assume that the other 20 percent of cases result in permanent loss of data, the cost can rocket. It may take hundreds of man-hours to rebuild data - in some cases, if the data is historic or experimental, it may never be possible to reproduce it. Several sources suggest that the value of 100 megabytes of data is approximately $1 million. Thus, if the average loss is just one megabyte of data, the loss costs $10,000.

Bringing all these figures together, the average cost of each incident is $2,615 (ranging from $615 for retrieved data to $19,615 for permanently lost data). Data loss costs European business $4.5 billion annually - and even this does not take into account potential loss of customers, loss of business, loss of market share, compliance failure, or legal breach.

In a $100m ($180m) outsourcing contract, Scottish National Health Service complained of loss of patient records, both digital and paper. Records had been misdirected to other hospitals and some cases, pages were lost. A British Medical Association spokesman said: “This is a serious breach of security.” In another sensitive case, thieves stole a PC and back-up disks from the National Association for the Care and Resettlement of Prisoners: it held data on prisoners. Voters were disenfranchised in a UK election when they were deleted from the electoral role by a data processing firm.

Theft of PCs and laptops, with their data, is an increasing concern. One company, Cosmetics to Go, went bankrupt after the theft of a PC containing its full customer database.

Disgruntled employees are often the cause of loss of data - and frequently they destroy or remove back-ups, too. Digital Technologies of Hartford, Connecticut, experienced this to their cost when an employee deleted their Web pages. While the site was unavailable, the company lost business for a week while the pages were re-created. In another case, a tire distributor lost nearly $2 million when a credit controller was fired: he kept a spare set of keys, re-entered the building, destroyed all invoice records and planted logic bombs in systems and programs - including the payroll.

Sometimes data that protects or reveals fraud may be erased. In the infamous case of the Bank of Credit and Commerce International (BCCI) following its crash in 1991, fraud investigations were ongoing in the USA, Luxembourg, and the UK. An attorney admitted that he had erased parts of a diary kept by Zafar Iqbal, the bank’s former chief executive. The entries were thought to show when the Abu Dhabi authorities first became aware of fraud at the bank (suspected of being known to authorities since 1984). The attorney’s defense was that he had erased the entries because they were held in an insecure location. There was equal concern about the possibility of lost evidence at WorldCom following its false traffic reports that led to a restatement of earnings by $7.8 billion. The downfall of Arthur Anderson in 2002 followed destruction of evidence during challenges of its audits of Enron.

Another company, BAC Computer, discovered its client records were scrambled - it was later revealed that an ex-employee had set up a similar business and was targeting their clients. The cost of damage was put at some $550,000 a month. A brief review of SunGard’s list of invocations shows a number of cases where invocation occurred as a result of data loss or corruption. In one typical case, a computer failure when archiving a database journal rendered the log file useless, and the Carlstadt, New Jersey, recovery site was invoked.

The implications are clear: regular and effective back up is vital. Products are now available that not only save this waste, but also improve IT Help Desk productivity in problem solving. One such product snapshots the entire hard disk content including operating system, applications, data files, preferences, and user settings. The information is automatically compressed at three levels (file, block, data) and stored in a network repository. Standard settings (e.g., for basic set-up and applications for many PCs) are stored only once and cross-referenced for optimum capacity usage and speed of recovery. This means that in many cases, it will be more productive for the Help Desk to roll back the last snapshot than to spend hours in problem diagnostics. And should a user lose data, restoration from individual files or the entire content of an individual PC can be done within minutes rather than hours. Subject to access rights and policy, the more computer sophisticated users could do this without help desk intervention.

It pays to check restoration from system and data back-ups (but in a test environment, until recovery is proven). A colleague was inspecting a client’s IT operations when he noticed an operator load a tape and key in a few instructions. Almost instantly the tape drive leapt into life - and stopped just as quickly. “What was that?” my colleague asked. “Oh,” the operator replied. “That was the back up. It used to take hours, but it’s ever so quick now that we’ve got the new program.” When checked, the program merely wrote a header to tape. Effective back-ups had not been taken for weeks.

A similar case arose when one organization tried to restore from its back ups: it found it had been writing zeros to tape for months.

Another issue can arise when trying to restore data: some devices have such sensitive head alignment that only the device that wrote the cartridge can read it. It pays to check read capability before it’s really needed.

One classic case involved an operator who used a removable back-up disk when the live data files crashed. He could not read from that, either, so he inserted it into another drive. That failed to read also. He then inserted the final back up in the second drive: It failed. What he had done was to put a good disk into a damaged drive, trashing the disk; put the primary back up into the same drive, trashing the back up; transfer the subsequently damaged disk to a good drive, trashing the drive; and put the last back up into a damaged drive, trashing the disk.

Yet another problem can arise where the device reads a block from memory, transfers it to tape, verifies the data on tape is the same as was read, then moves along a track repeating the process. A stepping motor then drops down to the next track, and the process is repeated in the opposite direction. On some devices, if the stepping motor is jammed, the device may simply go back over the same track, overwriting what was previously saved.

One survey by SecureIT found that over half of respondents had recently suffered data loss and 75 percent of them blamed faulty or non-existent back ups. Theft accounted for eight percent of the loss. And, even if back ups are taken, there may be files that are in use that are not backed up.

The survey also noted a steady increase in the amount of mission-critical data held on networks — up from 34 percent to 80 percent in two years. The occurrence of a serious network problem at least once a month was reported in 28 percent of networks. Data loss is costing UK business over $1bn ($1.7m) a year, according to a Prodata survey. In Germany, a survey of national computer centers a few years ago resulted in a call for risk analysis and back-up plans.

According to the UK Department of Trade and Industry, 70 percent of organisations that experience serious data loss go out of business within 18 months. The choice is clear — it could be a case of back up or die.

The payback speaks for itself — using a back-up tool, one organisation identified a return on investment of $6.6m over 2 years from savings on data loss and improved IT Help Desk productivity.

References

¹ The Cost of Data Loss in Europe by David Smith & Cheryl H J Bowker

Andrew Hiles is a director of the Kingswell International, consultants in service management and customer-supplier relationships.

He is the author of the CD Help Desk Framework and co-author of the book Creating a Customer-Focused Help Desk: How to Win and Keep Your Customers, published by Rothstein Associates, Inc. You can read more about Andrew and his work a www.kingswell.net.