Fitorch        
Results 1 to 26 of 26

Thread: The danger of data loss with digital storage.

  1. #1

    Default The danger of data loss with digital storage.

    Digital storage is good, but I think it has a pretty serious disadvantage too. Your picture, video, music and etc don't degrade from each step of copying, but it either works perfectly or it doesn't work at all.

    I just recorded a TV show to DVD-R, but the drive errored out during finalizing. Now the recorded disc would not play at all and I get is "bad disc" message. The recording is there, but it's just not accessible by ordinary means. Samething's happened w/ digital camera too. Small corruption. Everything on card completely gone.

    The data is basically completely lost less data recovery service. If it was analog, you'd just lose that small portion where the error occured.

    Something like this happened to you guys yet? How do you feel about digital storage?

  2. #2
    Flashaholic* powernoodle's Avatar
    Join Date
    Feb 2004
    Location
    secret underground bunker
    Posts
    2,513

    Default Re: The danger of data loss with digital storage.

    This has been a concern of my with pics I burn on CD. I have not found a good alternative other than making 3x copies, and then planning to make copies of the copies at some point in the future before the originals degrade.

    cheers

  3. #3
    Flashaholic* greenlight's Avatar
    Join Date
    Aug 2004
    Location
    chill valley
    Posts
    4,298

    Default Re: The danger of data loss with digital storage.

    I have lost digital information, mostly by burning data discs when I was still learning. Then the discs wouldn't read right, although the computer could see the directory structure.

    You might be able to recover data from the memory card.

    All things considered, the data storage is pretty convenient.

  4. #4
    Super Moderator
    DM51's Avatar
    Join Date
    Oct 2006
    Location
    Borg cube #51
    Posts
    13,339

    Default Re: The danger of data loss with digital storage.

    CDs & DVDs degrade over time. Hard drives eventually fail without warning. It is best to back up frequently to make sure nothing will be lost.

    Consider getting a portable external hard drive with a USB2 connection and a simple freeware incremental backup program such as "Handy Backup". You can get 100+Gb of storage in something smaller than a pack of cigarettes. Backing up this way is much less of a chore than burning a new CD, so you are more likely to keep up to date with it.

    Keep copies of data on CD/DVD as well, and check the copies from time to time to make sure they are OK. It is extremely unlikely that the CD and the external hard drive will both fail at the same time.

  5. #5
    Flashaholic*
    Join Date
    Dec 2001
    Location
    West Michigan, USA
    Posts
    490

    Default Re: The danger of data loss with digital storage.

    the point about checking the backup from time to time is important. It would be great if the backup software had a way to periodically verify the backup is good.
    Bright Scouter
    Del

  6. #6
    Flashaholic* abvidledUK's Avatar
    Join Date
    Aug 2005
    Location
    UK
    Posts
    2,148

    Default Re: The danger of data loss with digital storage.

    Don't forget the ANALOGUE backup.

    Ie: print out your best photo's onto photo paper, and keep in dark, except when viewing.

    I still backup-record anything important onto S-VHS too, ie first and last TV programmes etc.

    Once, twice, thrice bitten....

    At least DVD+R(W) discs are now so cheap, any disc errors, I junk them.

    I have used data recovery software off Internet for camera storage retrieval, to some degree.

    I did find, that one memory card reader couldn't cope with 512mb cards, corrupting data sometimes, I had another reader that did read ok.

    I also find that I can usually download from the camera, even if card is dodgy.

    I also download twice, once leaving data on card, followed by backup copy deleting data once first copy checked. I save one as "Camera Originals", just in case I mess up editing.

    Belt, braces and Superglue.

    And, as others do, I now backup to CDR on a daily basis, all my Excel, Word etc data, I get many days to each disc. I then store all my discs, separate locations. I do sometimes want "last years" version.

    I also backup all my data to two separate external HD's as well as to DVD's.

    I even print out most things..

    Photo DVD's tend to also get sent to relatives, in case of theft, fire. loss etc.

    Finally, I e-mail my most important data to myself, password protected, and leave it on my ISP's server.
    Last edited by abvidledUK; 01-10-2007 at 12:25 PM.
    Batteries, batteries, I need more batteries ........

  7. #7
    *Flashaholic* gadget_lover's Avatar
    Join Date
    Oct 2003
    Location
    Near Silicon Valley (too near)
    Posts
    7,129

    Default Re: The danger of data loss with digital storage.

    The danger's the same as it's always been.

    My mom took pictures at our most recent get-together. The film never engaged.

    My dad's birthday was caught on 35 mm. The photo lab lost the pictures.

    My house was burglerized when I was a kid. The burgler took the camera and the cardboard box it was in. The hundreds of slides were never found.

    At least I can backup my PC and all it's data on a schedule. I am a geek, so I have an automated tape backup, but you can also drop data to DVD. DVD-R is cheap.

    No matter what your media, the need for backups never changes.

    Daniel
    ================================================== =
    I have got to plan my procrastination better!

  8. #8
    Flashaholic* Sub_Umbra's Avatar
    Join Date
    Mar 2004
    Location
    la bonne vie en Amérique
    Posts
    4,749

    Default Re: The danger of data loss with digital storage.

    Quote Originally Posted by abvidledUK
    ...Belt, braces and Superglue...
    I worry about it quite a bit. I just bought my first DVD-RAM disks, which is actually a format that was designed for data and also works well for video -- as opposed to DVD which was designed for video and sort of works for data. DVD-RAM has ten times the error correction of DVDs and may be over-written 100,000 times and boasts a thirty year data life.

    IMO the real problem with digital storage ultimately centers around the longevity of the interface. (Or the lack of longevity.) Many cities in the 70s built up massive databases that they could no longer access just 25 years later. The data is fine but they no longer have the interface to access COBOL, or whatever obsolete language was written to whatever obsolete media they were using back then.

    A quick look at the brief history digital storage will show that it has no decent track record whatsoever when compared to the systems we have relied on in the past. While I'm not implying that books are the answer to long term data storage it should be noted that I have books over 100 years old and it is a very simple matter for me to access the data contained in them because the interface is built into me. As long as we just look at the data and ignore the very transient nature of digital formats and their interfaces our archival schemes may only be considered short term, at best.

    IMO the best thing about digital storage is that it is small and once something has been digitized it is a simple matter to copy it into newer digital formats as they emerge.
    Last edited by Sub_Umbra; 01-10-2007 at 02:07 PM.

  9. #9
    *Flashaholic* gadget_lover's Avatar
    Join Date
    Oct 2003
    Location
    Near Silicon Valley (too near)
    Posts
    7,129

    Default Re: The danger of data loss with digital storage.

    Forgive me for being contrary, but paper is not a great media either. You can't assume that "the interface is built into me" either.

    I have pounds and pounds of paperwork from the last 15 years. I've had occasion to go back through it looking for specific reciepts. It's not practical. The receipts the ones I found) are often faded and fragile. The search engine in cardboard boxes sucks. Once you find the paperwork you have the problem of interpreting it. I have no idea what the 3rd column of my household ledger meant back in 19 75. It's simpley titled "SP".

    On the other hand, I have 40 Qic80 based tapes sitting here, written in the 1990s. I was looking for a file on one of them, and found that my travan tape drive had given up the ghost. Dang! I failed to follow Sub_umbra's very good advice.... Copy to new media as soon as you change formats.

    Daniel
    ================================================== =
    I have got to plan my procrastination better!

  10. #10
    *Flashaholic* IsaacHayes's Avatar
    Join Date
    Jan 2003
    Location
    Missouri
    Posts
    5,876

    Default Re: The danger of data loss with digital storage.

    Burn your disc slower than it's rated speed. That always helps as some media isn't as good as others. Flash corruption recover software is out there too.

    You can pull data off a corrupted DVD with CRC errors. Use DVD Decrypter, and go into Tools - Settings, Click the I/O tab, Options check Ignore Read Errors. Also if you don't want to sit there while it re-reads for hours on one CRC, set software read and write retries to 0. Then you can rip it as an image or files to the HD, and then re-burn it. Sure there may be some glitches/pixels in the DVD where the CRC occured, but you can have a "clean" burn that can be re-burned again later with no fuss.

    There may be other software that can ignore CRC when reading a disc, but I don't know off hand...

    You may also be able to finalize the disc by putting it back in and with Nero and don't add any data, but go to the final burn tab. I forget what you click, and there might be a more direct way in NERO rom instead of express, but basically it just writes the lead out. The disc will take a long time to read in a DVD player, but will come up. Then you can probably pull the image back off the DVD, and re-burn it. I did this when a friend tried to burn 8x media in his old 2.4x dell drive that doesn't have firmware updates. I was able to finalize it in my DVD burner, then re-do it.

    And yes, always pull your old data and move it to your new storage stuff. I've had very good sucess with HD's though, my 486 and PentiumII still have data on them sitting in the closet as back up, and still is accessable. I've pulled that data to new storage, but it's nice to know that it's still there. After HD, I trust Optical next, and Flash the last, but perhaps Flash is better than Optical for shelf life though... A USB flashdrive can be corrupted by not disconnecting it right from windows depending on the situation, so always back those up when you can. I've got a few redundant copies on CD-R, so it will be intersesting to see if they last as long as the HD backups.
    Last edited by IsaacHayes; 01-10-2007 at 02:41 PM.
    Mags: 2C: R/O, True Cyan, UV. 3C:Penta-XR-E Q5, Single SSC P4. Nitecore D10, Fenix LOD-CE, ArcAAA

  11. #11

    Default Re: The danger of data loss with digital storage.

    Quote Originally Posted by IsaacHayes
    Burn your disc slower than it's rated speed. That always helps as some media isn't as good as others. Flash corruption recover software is out there too.
    The device I just had a problem with is a stand-alone DVD-R "VCR". To start recording, the deck prepares the disc and it is started by pressing the red "record" button just like in VHS.

    It can be recorded to the end like this and it would playback on this device, but to make it a proper DVD, the disc needs to be finalized. The deck asks you if you'd like to close & finalize the disc after the whole recording capacity is used up or when you try to eject the disc. You hit "enter" and it goes like that.

    This time, it just said "error" and the recorded disc would not playback anything just because it failed the closing. Since it's just a TV show, I'm not going to bother with extensive recovery.

    Since it records in real-time, there is no "speed" selection.

  12. #12
    *Flashaholic* IsaacHayes's Avatar
    Join Date
    Jan 2003
    Location
    Missouri
    Posts
    5,876

    Default Re: The danger of data loss with digital storage.

    Pop the disc into your PC, open nero, and click through until you get to the burn screen. It may ask you do you want to switch to ISO mode blah blah, hit yes as that's what the DVD is made in. Then just hit burn and it should finalize and write the lead out.

    Then if it doesn't work or is slow, you can trash it or re-rip on your PC. If it works then that wasn't a whole lot of work..
    Mags: 2C: R/O, True Cyan, UV. 3C:Penta-XR-E Q5, Single SSC P4. Nitecore D10, Fenix LOD-CE, ArcAAA

  13. #13
    Flashaholic* Sub_Umbra's Avatar
    Join Date
    Mar 2004
    Location
    la bonne vie en Amérique
    Posts
    4,749

    Default Re: The danger of data loss with digital storage.

    Quote Originally Posted by gadget_lover
    Forgive me for being contrary, but paper is not a great media either...
    That's true but paper has preserved data successfully many times longer than any digital media. Digital data is far less than 100 years old and 100 years isn't that old for a book. The oldest of digital data ever archived can no longer be accessed -- except through very intensive efforts which cost far more than the data's worth in most cases. Ask yourself how you would access old digital data -- data from the 70s and 60s? As far as data goes, digital data is not old at all!

    Even if you keep it dry, keep it from burning and keep it from being stolen, we will still have to wait decades and decades to extract 100 year old digital data as easily as I may access the contents of a 100 year old book that has been protected from the same threats. That's assuming that digital archiving technology changes direction and eventually begins to develop a more enduring interface. 100 years from then digital data storage will have the beginnings of a good track record. Until then we don't seem to have much choice -- we just have to keep moving it from format to format to cover our bases.

  14. #14

    Default Re: The danger of data loss with digital storage.

    External hard drives are very cheap. Recently I saw 250 gig maxtor drives for $50 at a box store.

    Backing up data on multiple external drives should provide good defense against data loss. If you have the data on 3 seperate drives I doubt you'd ever lose it (it'd be astronomically rare for 3 drives to faile at the same time).

  15. #15

    Default Re: The danger of data loss with digital storage.

    I've had everything fail at one time or another. Old papers turning yellow or the pencil/ink fading and smearing, to various media/drive failures.
    Make multiple backups on multiple media formats. Keep file formats as standardized as much as possible. Then keep backups is different locations.

  16. #16
    Flashaholic
    Join Date
    Jul 2006
    Location
    Ottawa, Ont. Canada
    Posts
    466

    Default Re: The danger of data loss with digital storage.

    I worry abo0ut the same thing ---- Is it really burned ?
    My primaray drive is tiny 20 gb but I have a new one 75 gb thats just sitting installed but ..... with nothing on it (well a few songs) I would love to swap harddrives

  17. #17
    *Flashaholic* gadget_lover's Avatar
    Join Date
    Oct 2003
    Location
    Near Silicon Valley (too near)
    Posts
    7,129

    Default Re: The danger of data loss with digital storage.

    Backing up data on multiple external drives should provide good defense against data loss.
    That's' not usually true. The problem is that it's still writable. I've seen people with viruses destroy thier back up copies by trying to read each one while the virus was still active. I've seen multiple disk drives erased because of a bad controller. In that case the user thought it was the disk drive at fault, so he put his backup drive online.

    Multiple formats, read only medium, removable and at least one copy stored off-site. That's the profesional way to do it.

    Daniel
    (I really have no room to talk. I have tapes, disks, dvds and hard drives sitting all over the room. None are properly stored nor protected.)
    ================================================== =
    I have got to plan my procrastination better!

  18. #18
    Flashaholic*
    Join Date
    Mar 2002
    Location
    New Jersey
    Posts
    1,750

    Default Re: The danger of data loss with digital storage.

    Quote Originally Posted by Handlobraesing
    I just recorded a TV show to DVD-R, but the drive errored out during finalizing. Now the recorded disc would not play at all and I get is "bad disc" message. The recording is there, but it's just not accessible by ordinary means.
    Did you try playing it or accessing the files on it in your PC? Data DVDs don't need to be finalized and I think video DVDs don't either. Most or all DVD players require it, but the drive in your PC doesn't.

  19. #19

    Default Re: The danger of data loss with digital storage.

    Quote Originally Posted by Eugene
    I've had everything fail at one time or another. Old papers turning yellow or the pencil/ink fading and smearing, to various media/drive failures.
    Make multiple backups on multiple media formats. Keep file formats as standardized as much as possible. Then keep backups is different locations.
    These deterioration can lose some contents, but usually not a complete loss. If a corner of picture or a page of a book gets burned or ripped, that portion is gone, but the rest is good.

    If a CD cracks, deforms or suffers from corrosion (air gets under the layer and metal foil gets pitted away, it happens sometimes) and the damage is in the data catalog/index area, ordinary device can not access anything.

    Corporations with unlimited funds and the FBI can retrieve the data from such, but the former usually know better than having only one copy, so such recovery is so outrageously expensive it is reserved for forensic evidence.

  20. #20
    Flashaholic*
    Join Date
    Mar 2002
    Location
    New Jersey
    Posts
    1,750

    Default Re: The danger of data loss with digital storage.

    DVDs can be a good way to archive data. Using more than one hard drive is good, but DVDs have some advantages. If you drop a hard drive, you will probably lose everything. DVDs can be easily kept at multiple locations. That's what mothers are for, you know.

    But to get reliable DVD storage, you have to know how to get good burns, and like always, you need to make more than one. I don't generate much valuable data, so a DVD backup of all important stuff once a month or two is adequate. If last month's DVDs should somehow be lost, I wouldn't lose much by using an earlier one.

    To make good DVD burns you need to get good blank DVDs. You can't necessarily tell what media you have by looking at the label. Some media give bad burns, with lots of correctable errors, and some media can make good burns today, but 6 months from now they may be unreadable. These discs may be readable today in your current drive but might not be readable tomorrow, or in a different drive.

    To know what discs you have, you need to see the MID (manufacturer's ID) that is recorded on the disc. The CD-DVD speed program (usually called CDspeed) will show you that. People that have thousands of DVDs and are particular about them agree that the MCC MIDs (Mitsubishi Chemical Corp.) and the MIDs used by Taiyo Yuden are the best. Most +R and -R discs sold by Verbatim have a MCC MID. To get Taiyo Yuden you can buy them at rima.com

    Then you have to scan the burned disks. Just reading them back is better than nothing, but that just tells you they are readable today in that optical drive. There are tests that can give you more confidence that the disks will be readable in other readers and readable in the future.

    The CDspeed program has two tests for this. The Transfer Rate Test can be run on any DVD drive. This test attempts to read the disc at full speed, and plots the speed during the read. Any dips in the graph indicate the drive slowed down because it encountered excessive correctable errors. Well, if your CPU is busy or you have a bad connection to the drive, that will cause dips also.

    The Disc Quality test can only be run on drives that can report the number of errors it encounters. The disc is read at a constant speed and the errors per block are shown on a graph. All discs have errors (correctable errors). The difference between a good disc and a bad disc is the number of errors.

    You can learn about this and more at cdfreaks.com

  21. #21
    Flashaholic
    Join Date
    Jul 2004
    Location
    Virginia
    Posts
    444

    Default Re: The danger of data loss with digital storage.

    My parents have baby pictures of me that are severely color shifted to magenta. There is one that is still approximately correct color but the process was expensive, difficult, and never caught on. It's a GAF print on what appears to be a plastic coated photographic paper.

    Maybe I should write down some PGP and Scramdisk passwords. Even if the media survives, my memory may someday fail.

  22. #22
    Flashaholic Fluffster's Avatar
    Join Date
    Nov 2006
    Location
    Reykjavik, Iceland, Arctic Ocean
    Posts
    119

    Default Re: The danger of data loss with digital storage.

    Quote Originally Posted by Datasaurusrex
    Backing up data on multiple external drives should provide good defense against data loss. If you have the data on 3 seperate drives I doubt you'd ever lose it (it'd be astronomically rare for 3 drives to faile at the same time).
    Your data is still gone if your house burns down.
    I keep my digital photographs on my desktop computer and a copy on an external drive. Once a week I synchronize those with an identical setup my father has across town, so we mirror each other's pictures.

    I know I should add some kind of WORM medium to the mix, but I know I'm too lazy to keep up with that.

  23. #23

    Default Re: The danger of data loss with digital storage.

    Quote Originally Posted by Handlobraesing
    These deterioration can lose some contents, but usually not a complete loss. If a corner of picture or a page of a book gets burned or ripped, that portion is gone, but the rest is good.

    If a CD cracks, deforms or suffers from corrosion (air gets under the layer and metal foil gets pitted away, it happens sometimes) and the damage is in the data catalog/index area, ordinary device can not access anything.

    Corporations with unlimited funds and the FBI can retrieve the data from such, but the former usually know better than having only one copy, so such recovery is so outrageously expensive it is reserved for forensic evidence.
    Thats why I said multiple backups on multiple media formats. You can't really do that with paper. For electronic use CD, Disk, Tape, etc.

  24. #24

    Default Re: The danger of data loss with digital storage.

    Quote Originally Posted by eluminator
    DVDs can be a good way to archive data. Using more than one hard drive is good, but DVDs have some advantages. If you drop a hard drive, you will probably lose everything. DVDs can be easily kept at multiple locations. That's what mothers are for, you know.

    But to get reliable DVD storage, you have to know how to get good burns, and like always, you need to make more than one. I don't generate much valuable data, so a DVD backup of all important stuff once a month or two is adequate. If last month's DVDs should somehow be lost, I wouldn't lose much by using an earlier one.

    To make good DVD burns you need to get good blank DVDs. You can't necessarily tell what media you have by looking at the label. Some media give bad burns, with lots of correctable errors, and some media can make good burns today, but 6 months from now they may be unreadable. These discs may be readable today in your current drive but might not be readable tomorrow, or in a different drive.

    To know what discs you have, you need to see the MID (manufacturer's ID) that is recorded on the disc. The CD-DVD speed program (usually called CDspeed) will show you that. People that have thousands of DVDs and are particular about them agree that the MCC MIDs (Mitsubishi Chemical Corp.) and the MIDs used by Taiyo Yuden are the best. Most +R and -R discs sold by Verbatim have a MCC MID. To get Taiyo Yuden you can buy them at rima.com

    Then you have to scan the burned disks. Just reading them back is better than nothing, but that just tells you they are readable today in that optical drive. There are tests that can give you more confidence that the disks will be readable in other readers and readable in the future.

    The CDspeed program has two tests for this. The Transfer Rate Test can be run on any DVD drive. This test attempts to read the disc at full speed, and plots the speed during the read. Any dips in the graph indicate the drive slowed down because it encountered excessive correctable errors. Well, if your CPU is busy or you have a bad connection to the drive, that will cause dips also.

    The Disc Quality test can only be run on drives that can report the number of errors it encounters. The disc is read at a constant speed and the errors per block are shown on a graph. All discs have errors (correctable errors). The difference between a good disc and a bad disc is the number of errors.

    You can learn about this and more at cdfreaks.com
    One of my tests for important data has always been to place the CD or DVD in another computer and attempt to copy all the data to the drive then open a few at random.
    If your stuck with only Microsoft Windows on your burning system then only burn after a reboot and don't do anything else while burning.

  25. #25
    Flashaholic* Neg2LED's Avatar
    Join Date
    Apr 2004
    Location
    'straya, mate!
    Posts
    1,127

    Default Re: The danger of data loss with digital storage.

    I don't burn to discs, too danger-prone. if I'm going to use a disc for something i use it within 24 hours of burning...Mostly I use RAIDed hard drives - RAID 5 on the server and most of my computers have RAID 1 on the data drive. I also make it a policy to not keep any file I don't want to lose on a single machine. If I can't afford to lose it, then I duplicate it to multiple machines - usually my desktop, the server and my laptop.

    I'm thinking of building a NAS....

    --neg
    "Barad-Dur was an inside job. Everyone knows dirty little hobbitses can't melt mithril beams."
    | too many 47s products
    | Mag2D nFlex LuxIII/6AA NiMh Mod (from when that was good) | Absurd Jayrob 5A MT-G2 MagLite | User 4798 |

  26. #26

    Default Re: The danger of data loss with digital storage.

    All the RAID'ed drives in world won't protect from a virus, or os issue. Sometimes write once media has its advantages. no one media type is perfect, thats why you backup to multiple types.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •