Storing digital information for a very long time

[sandbender]

I did not want to hijack StigOfTheDump's thread on this sites compatibility with the Opera browser however I thought the points raised in the posts by Toddy and British Red could be explored further...

"...I was at a conference a couple of years ago, the discussions centred around the future of the archives of the past and present, and their accessiblity.

The earliest attempts at computerisation were now totally unreadable by the vast majority of computers. It wasn't just the storage medium either, it's the entire language and the formats too.
Code cracking has nothing on some of the lengths the archivists had to go to in their attempts to 're-access' critical documentation..."

"...data compatibility and backward application compatibility are critical to technology. I've lost count of the "Bambleweenie 2000" technologies that have come and gone (remember OS2 anyone? )..."

A prime example of computer data becoming inaccessible would be the BBC's Domesday Project from the mid eighties...

"In 2002, there were great fears that the discs would become unreadable as computers capable of reading the format had become rare (and drives capable of accessing the discs even rarer)."​

This site details the hard work and reverse engineering that was needed to recover the information on those discs.

In an earlier thread I posited that "a long lived digital storage medium will be invented and will become as popular as the DVD, thus preserving an image of our 'digital' lives well into the future", well it would seem that researchers at the University of California have devised a technique for storing data that could potentially last more than a billion years.

Which could mean that researchers in the distant future will be able to browse the BCUK posts and try to draw conclusions about who we were and our various motivations for posting here. Alternatively if it all goes pear shaped there may be a trade in these very long lived discs or chips because of they will make good hide scrapers or nice jewelry.

Perhaps of more use for the future will be the ability to store our precious documents and books for posterity, perhaps the best example of this being The recent scanning of Venetus A, the oldest existing copy of Homer's Iliad.

"After a thousand years stuck on a dusty library shelf, the oldest copy of Homer's Iliad is about to go into digital circulation.

A team of scholars traveled to a medieval library in Venice to create an ultra-precise 3-D copy of the ancient manuscript -- complete with every wrinkle, rip and imperfection -- using a laser scanner mounted on a robot arm.

A high-resolution, 3-D copy of the entire 645-page parchment book, plus a searchable transcription, will be made available online under a Creative Commons license."​

See the book here (operates in a similar manner to Google Maps).

Read an article about the scanning process here.

 

[Toddy]

Brilliant
I'm old enough to thoroughly appreciate and delight in the ways that knowledge is so very accessible now
Examples such as the one you give are brilliant

I have no real issues over different programs and OS being used ( in my household that would be heresay ) and I do have a respect for the multitudinous ways that variety and multiple strands of research greatly augment the pace of development. However, I do think sometimes the developers forget, or don't care, that communication between systems is vitally important too.

I watched my infant sons learn to use code on an old Kim-1 (a 6502 chip and cassette tapes) and now I'm the mother of netgods somehow I'm still a computer clutz

cheers,
M

 

[Tadpole]

The more you spend on the writeable DVDs the longer they will last, but in all honesty the ink/dye used in even the best is maybe good for 10 years. After that there is no guarantee that even if you have a DVD reader capable of being fitted to a new ultra modern new generation machine, that the ink/dye has remained intact enough to give enough contrast so your home burn DVDs are still readable

 

[Melonfish]

i wish i could read my old amiga floppies is all.
some cracking games on there..

 

[gregorach]

It's not enough to have durable media. You have to have a working device that can read from that medium, and the ability to interpret the results. An LP record is a very durable recording medium, but it's no use at all if you don't have a turntable and know correct the RIAA de-emphasis curve to apply to recreate the original signal.

The Voynich manuscript is still perfectly legible, it's just that nobody knows how to read it, or even what language it's in. And what value would the Egyptian hieroglyphs have had to us without the Rosetta Stone?

Even in the Domesday example, it wasn't that the media itself was degraded to the point of unreadability - it was that the ancillary technology needed to read it was almost lost. I still have a large collection of cassette tapes, and I presume they still work, but I don't have a working tape deck...

Fired clay tablets, or iron gall ink on vellum - these are truly durable media. But even then, they're useless if you don't know the language.

 

[Shewie]

i wish i could read my old amiga floppies is all.
some cracking games on there..

I've still got mine in the loft, along with two Atari consoles (wood and the new slimline version), a Spectrum ZX and the mighty 128kb +3 with built in disc drive.

Can you remember the horrible screaching noise we had to go through with the tape player, adjusting the volume to get it to load properly.

Happy days

 

[Tengu]

Ive got a modem for a Commodore 64, but no Commodore 64.

how do I get online??

 

[British Red]

It's not enough to have durable media. You have to have a working device that can read from that medium, and the ability to interpret the results. An LP record is a very durable recording medium, but it's no use at all if you don't have a turntable and know correct the RIAA de-emphasis curve to apply to recreate the original signal.

The Voynich manuscript is still perfectly legible, it's just that nobody knows how to read it, or even what language it's in. And what value would the Egyptian hieroglyphs have had to us without the Rosetta Stone?

Even in the Domesday example, it wasn't that the media itself was degraded to the point of unreadability - it was that the ancillary technology needed to read it was almost lost. I still have a large collection of cassette tapes, and I presume they still work, but I don't have a working tape deck...

Fired clay tablets, or iron gall ink on vellum - these are truly durable media. But even then, they're useless if you don't know the language.

All true, but I am certain that the degradation to obsolescence is increasing both in rapidity (time to die) and frequency (number of dead "languages"). An understanding of Latin and Greek bought me insight into centuries or recorded information. The lifecycle of , applications, db schema, digital media and interprative hardware is now such that information can become irrecoverable well within the lifetime of the author.

However how much is worth saving is of course a different topic.....

 

[sapper1]

Am I missing something here?
Surely the way to store information so that it can be accessed is to store the hardware to read it at the same place and time.If you have all your info on cassette then store a cassette player with it,if it's on dvd then store a dvd player with it.

 

[gregorach]

That assumes several things:

1. The device in question will continue working indefinitely without maintenance. This is never true, so far as I know.
2. Any external inputs the device requires (e.g. electrical power at the correct voltage and frequency) are still available.
3. The device produces output which can be interpreted directly. A DVD player doesn't - you need an additional device (a TV set), which in turn has its own maintenance headaches. In 100 years time, you probably won't be able to hook up an original first-generation DVD player to a current TV, because the signal semantics (number of lines, refresh rate, etc) will no longer be compatible. Heck, the entire TV paradigm might be dead by then.

 

[Tadpole]

if you take the disk media, the process didn't even take 50 years for 8in disk to become unusable.

 

[sandbender]

"...It's not enough to have durable media. You have to have a working device that can read from that medium, and the ability to interpret the results. ..."

Very good points, another factor to consider is whether or not future generations will be 'permitted' to read materials from the past.

"Oldthinkers unbellyfeel Ingsoc"

 

[sapper1]

The dvd was an example.If you have a power sourcw now,and the hardware to read the info now why can't it all be stored together.A generator and a hardware system to read the info shouldn't be that hard to sort out,or is it just too simple?
I have in front of me a computer and a printer that all works together to read memory cards and discs if I add a generator then it can be stored and used when neccessary.
As for the hardware breaking down can it not be repaired then,I know a place that can still supply valves for radios and amplifiers that are no longer made,you can even get parts made for a modelt ford.
8 inch disc is not unusable, if you have an 8 inch disc player then they will still play.I think people have fallen into thinking that as it may not be the latest technology then it wont work.This is nonsense my cassette player still plays cassettes and I can still get batteries for it or plug it into the mains,my 1930's radiogram still recieves radio signals the only time it won't is when we switch to digital radio signals,but that doesn't mean it doesn't work.

 

[gregorach]

I have in front of me a computer and a printer that all works together to read memory cards and discs if I add a generator then it can be stored and used when neccessary.

And what does the generator run on?

As for the hardware breaking down can it not be repaired then,I know a place that can still supply valves for radios and amplifiers that are no longer made,you can even get parts made for a modelt ford.

Sure, but if you have the knowledge needed to repair the hardware or build the parts, you have the knowledge to rebuild the whole thing from scratch anyway. I thought we were talking about long periods of time, the sort of period over which an entire culture can completely disappear, next to which the couple of hundred years since the Industrial Revolution are a mere instant.

And of course, all this talk of making sure that the media is durable and you've got the hardware to read it, etc, is still useless if nobody speaks the language the information is recorded in. As I said before, it doesn't matter if you've got the hieroglyphs, without a Rosetta Stone.

 

[sandbender]

This is nonsense my cassette player still plays cassettes and I can still get batteries for it or plug it into the mains,my 1930's radiogram still recieves radio signals the only time it won't is when we switch to digital radio signals,but that doesn't mean it doesn't work.

Digital information is less forgiving of errors, your analogue compact cassettes may develop faults and suffer from drop out but you may still appreciate the music.

If part of an .ISO image stored on a DVD is corrupted then that information is unrecoverable.

That said I can copy my book of bad poetry onto fifty DVD's and post it to various locations around the world or put it on a web site for others to copy and thus increase the likelihood of a copy of my book of bad poetry still being around and read a few hundred years hence.

Whereas printing my bad poetry onto a clay brick or writing it into a vellum notebook or recording it onto a compact cassette makes it more difficult to copy and essentially puts all my eggs into one or two baskets.

Of course this presumes a continuity of civilization where better and more long lived storage systems are brought into being, a hundred years from now Apple OS 10.4.11 will be a thing of the past, however in much the same way that I can play my Commodore 64 games and read diary entries written on my Apple Newton via emulation it isn't a big leap to assume that I or someone will still be able to read stuff written on my Macbook today via perhaps several layers of emulation running on whatever system is being used in the future.

Obviously this presumes that I keep updating my backup files to whatever substrate is popular every ten to fifteen years.

Of course if civilization is 'interrupted' by a huge rock falling on our heads or similar then fired clay bricks may be the way to go

 

[sapper1]

Gregroach,This is what I'm saying if you store the information with the means to read it (this includes the power source) then it can be read.Generators can be run on manpower,liquid fuels,solar power, electricity,water power,gas fuels and batteries.I only hope the information is found by someone smart enough to not need this explaining.

 

[Goatboy]

It's all well and good trying to store all this information and ways of retrieving it, but we also have to encourage a "want / need" to revisit it. More and more people are becoming more Eloi like, never reading a book or learning how things are done. They feel that someone else may do it for them or they needn't learn to cook as there are always ready meals.

So unless a thirst or need for knowledge is there I'm afraid that all the great libraries of the world will become roosts for pigeons.

GB

 

[Toddy]

If it's found within a couple of lifetimes perhaps.........but without study Middle English is unintelligible to almost the entire English speaking world.........in a society that no longer values literacy as it once did, and where text speak is already the prefered written medium of many of the youth, how long will reading be a familiar concept either ?

Nothing can be preserved in stasis, humanity is complex, society is in flux. Go with the flow or be buried in the mud and only perhaps preserved as a fossil .........and I'm an archaeologist and this is me saying all this. One of my colleagues a couple of hundred years from now will find such a stash and describe it as of ritual significance

cheers,
Toddy

 

[Goatboy]

Hi Toddy,
Do you ever worry as an archaeologist that you may attribute the wrong use or significance to an object or find. As you say thing's don't have to be that old to be confused. Any good junk shop is bound to trow up something that most folk wont recognise, so many things go out of fashion or are superceded by newer things. How many folk would recognise a marrow spoon, or a set of wallnut picks as they're not so commonly eaten now.
Some of the old farm implements and tools that can be found in barns makes the mind boggle.

For a bit of fun have a look here. http://www.odd-tools.com/

TTFN
GB.

 

[tobes01]

Aren't we overlooking the point that as (relative) oldies, we look to store data on a physical device we keep with us; kids treat the Internet as their storage. And the Internet never forgets things that are accessed, so it will continue to store data that is relevant to someone, somewhere. Even if there's a really big bang, there will be copies in a data centre somewhere.

Being of the old git persuasion myself, I use a Drobo RAID drive for my storage, with copies on DVD in the gun cabinet.

As an aside, a colleague had to take a laptop on a (non-bushcraft) camping holiday last year to finish a book he was writing. His kids asked him "what use is a laptop without an Internet connection?" Says it all for the way they see the future...