IHeartBadCode

what long term storage would be the best option for storing digital information

The biggest factor with passive storage, something that’s not refreshed. Optical media that’s made to last. M-DISC comes to mind, but there’s no proof that worse case 100 years is a valid claim. Chemically speaking, a well kept disc should keep 100 years, but that’s chemical composition in an ideal case. Nothing in manufacturing is perfect, so impurities are always going to be there robbing the lifespan of these discs.

Magnetic tape ideally lasts decades if not close to a century, but these are tapes that are kept in incredibly controlled conditions. If you’ve ever worked in the server world you’ll know that any plain Jane LTO magnetic tape can’t be trusted after collecting dust for anywhere close to five years.

We have scrolls, we have books, and we have stone tablets that have endured centuries, but the key in all of those is how well they were kept. The construction matters, but the bigger aspect is the environment they were kept in. For digital media, we don’t know any real way to keep digital data in a passive state for centuries because, well, we haven’t had digital data for that long. We’ve got really old punch cards that are close to that age, but even then, some of the oldest stacks are now sitting in hermetically sealed cases and are actively upkept to prevent UV damage by clear coating those cases on a regular basis.

And that’s the thing with digital media, keeping it in an active storage rather than passive may be the key for centuries of longevity. USB sticks are fine so long as someone remembers to plug them in and allow them to refresh every some many years. Most USB sticks use ceramic capacitors, so leakage there isn’t too much an issue. The bigger thing might be corrosion of the various traces and pins, but if well kept, that might take decades to eventually make an impact.

Sometimes, I like to parallel digital long term storage as the Ship of Theseus. If you keep moving the data from one device to another, it’s still the same data. And in that sense, the data can live forever. Even if there’s a gap of say two decades, if you can still get to the data and convert it into something modern, the data lives on. It’s not the original medium, but with digital data, it doesn’t have to be, that’s the neat thing about digital data.

I think people still are working on trying to wrap their heads around digital data versus the way we used to do it. You know, someone might have the family bible and we’ve got to keep it nice and tidy and careful with it, because with analog data the medium and the information are one in the same. And I think sometimes people look at family digital photo collections like that. Like it’s the family bible and that we’ve got to keep it safe. But if it’s a USB stick that you pull out every so often, look over it, and call it day. Maybe move the photos from the Walmart USB stick that you got in 2016 to the new 800TB USB-F stick you just got from neo-Amazon in 2073, those photos can live forever. You don’t have to be careful with them anymore.

I think that’s one of the reasons that open formats matter so much. If you stored all your family videos in Windows Media Format, what happens when Microsoft dies in the Second US Civil War of 2038? That’s not helping you in 2073 to open those files on a format you can never figure out. But say you stored it in some open format. Now all you need is an implementation of that format and a compiler. And poof, now you have a modern codec to read the files of the before times.

It’s one of those fun maybe slightly existential kinds of things. Nothing lasts, no matter how hard we try, nothing will last. All things forgotten decay, we can only slow that decay down, but we can’t prevent it. But things that live, things that pass through the hands of the living, those things endure because there are people who put time, one of the most precious resources we have, into them. Our reward for that investment of time is something that continues beyond the decay.

I like to think of it as the balance of the universe. You get to keep this, but only if you give a bit of time to pay for keeping it. And sometimes it’s crazy to think of how much that applies to. Also I likely shouldn’t reply after having a few drinks. Wooooo!!

Environment makes all the difference for passive storage, sorry I really went out there on the reply.

Best bet is long term optical discs or long term magnetic tape. USB keys are not good for long term storage. USB keys use NAND memory that is a series of floating gate metal oxide semiconductors (FGMOS). These operate by using Fowler-Nordheim tunneling, in where a charge is carried along a regular style fin field-effect transistor (FINFET) and a charge above the transistor’s channel causes some electrons to quantum tunnel into floating gates that are isolated by oxides.

While these floating gates are sealed off from everything, so the charge should stay “indefinitely”, quantum effects cause some of the electrons to “leak” out of the floating gate, causing a degradation of the stored signal. Typically there’s a refresh circuit within the USB key’s integrated circuit that takes care of that and USB data can last seemingly forever. However, that refresh circuit requires a small amount of power, which if you store the USB stick somewhere for years on end, will never get powered.

This is the reason why flash memory only assures data can be retained for about ten years without power. Eventually the electrons “trapped” in floating gate have enough time to tunnel out of the floating gate completely obliterating the signal. The tunnel events aren’t many per second, but give enough time, and all of those events add up. Paired with the whole thing that USB sticks mostly no longer use binary logic levels. Most are now using something like four or eight logic levels. So instead of there just being on and off, there is 0V-0.7V = 00, 1V-1.7V = 01, 2V-2.7V = 10, 3V-3.7V = 11 logic levels. So a small amount of charge loss can create a different bit pattern.

One thing to look at for long term storage is something like M-DISC. The matter by which the burned data onto the optical media is made is via a process that takes about 10,000 years (estimated) to break down. However, the disc itself is in a polycarbonate thermoplastic that has an average breakdown of only about 1,000 years in extremely dry environments and about a tenth of that in your average sealed lock box environments.

Your average spinning disk hard drive can store information for some time, but the storage requirements are pretty intense and even then hard drives loose about 1% of the magnetic strength per year without power. And about 70 years is the max before the various magnetic bits that form the low level format of the disk have degraded without power to the point that the disk has too many bad sectors to be called usable. But outside of that, the biggest fault is mechanical failure. No matter how well you think you’ve stored a drive, it’s never good enough and the spinny bits always fail from becoming too fragile from pervasive oxidation. Basically the drive will spin up only to tear itself apart as some weaken part of the armature flies into the spinning platters.

But USB sticks will only give you about a decade before the stored information fades away into the quantum ether.

Really I’m just disappointed in the lack of blackjack here.

In all fairness paranoia is important in things like finding someone trying to slip a backdoor into xz. Going by the most recent Windows 11 update, Microsoft seems to be “we’ll fix it in production”.

Nah it’ll be like what we think of this kind of art.

do as I say not as I do

Nintendo: Money! Fuck everything else.

All other attributes derive from that.

I mean ever since XZ binaries are a no go. Like the trust is gone, the bridge is burnt, unless there’s like an insanely good reason, anything this heavy in BLOBs ought to be treated as suspect. Benefit of doubt isn’t coming back.

I literally had to cite the page number from the Consolidated Appropriations Act of 2023 Public Law 117-328 that covered how the $800M that Trump keeps telling everyone FEMA spent on migrants was a completely different fund than the disaster relief fund that FEMA uses for hurricanes. Which the DRF was established originally as it’s own fund in the Stafford Disaster Relief and Emergency Assistance Act of 1988 Public Law 100-707

It’s page 4,730 where that item is located for anyone wondering.

I fucking hate what online interactions have become. I think I’ve easily read over 200,000 pages of government legislation, federal regulation, and legal proceedings since June because of the lies one orange shit stain keeps telling. I really do hope that the Republicans can move past that fucker, it was a lot easier to talk politics.

for a reason that you are listening songs from another country or reading books, that was not accepted by censorship

Man are you going to have a wild time reading the First Act of Supremacy of 1534 from the United Kingdom. Couple of follow up bangers from it like the Act of Supremacy in Ireland of 1560. All that happening distinctly before communism was even invented.

Because my country, Ukraine, was under communists and it was not good time with all genocides, holodomor, repressions, red terrors and other things

Yes, but none of that is unique to communism, that’s just corrupt government. Anywhere that develops systemic inadequacies and a culture of impunity can instantly become such. That’s just something that is independent of the underlying system of economics. Like many capitalist systems like to point out that bourgeoisie who are after their own interest act as some check on the government who is usually in a power struggle for control. And that power struggle is what ensures no one side wins out.

But there’s nothing technically stopping the rich from becoming the actors of the government and when we as a society excuse profiteering in office, well then there’s no barrier from the rich just becoming the government. Which that’s just the French ancien régime that ultimately lead to the French Revolution.

So it’s NOT specific to just communism. It’s just that’s the most recent and easiest one to point out because of how blatant/brazen that system had become with it’s corruption. Even with all of the “nay-saying” that might happen with United States detractors with their usual hum of “Oh well they’re all corrupt!” Even with how passive some are with it, the corruption is nowhere near the level of being out in the open that was with the USSR. Politicians still weasel their way around because they know that there’s still some bottom level of ensuring checks on that corruption that exist. And we have those checks not because we are a capitalist society.

I think the idea that some economic system promotes some civic purity or prevents some form of government corruption is a bad linking of things that ought not be linked, because a pure capitalist society doesn’t magically inherit some barrier of corruption. That barrier has to be formed independent of the underlying economic system.

I’m not trying to detract from what happened under the USSR but that has way more to do with how power got consolidated post World War I and everything that lead to the toppling of the Russian Monarchy. The system of communism played a role in that consolidation of power, yes, but literally any tool could have been used if you have someone with the mindset of Vladimir Lenin who wanted to rapidly consolidate power during the Bolshevik revolution. I mean look at the current Myanmar Civil War and some of the ideas of General Min Aung Hlaing, no need for implementation of communist ideology there, he just wants to be in power, doesn’t believe that the current transfer of power is legitimate, and is willing to get a lot of people killed in proving that point.

I think given the current situation in the United States, the belief that you NEED communism to have totalitarianism is a dangerous linking of things that can actually happen independent of each other. You just need someone to wear down government legitimacy enough to start a civil war, that’s all you need. Everything else is just tools at your disposal to get that goal done.

So you have to understand the nuance here I’m trying level. I’m not saying it WASN’T COMMUNISM, what I’m saying is that it can be communism, but ultimately you just need someone who wants to consolidate power rapidly and exists in a society that will forgive abuses of power enough, sometimes that’s done by de-legitimizing the current system enough. That’s it, that’s all that’s required. Communism can play a role in that somewhere, but it doesn’t have to.

Thermal is a wall to contend with as well. At the moment SSDs get the density from 3D stacking the planes of substrate that make up the memory cells. Each layer contributes some heat and at some point the layer in the middle gets too hot from the layers below and not being close enough to the top to dissipate the heat upwards fast enough.

One way to address this was the multi-level cell (MLC) where instead of on/off, the voltage within the cell could represent multiple bits. So 0-1.5v = 00, 1.6-3v = 01, 3.1-4.5v = 10, 4.6-5v = 11. But that requires sense amplifiers that can handle that, which aren’t difficult outright to etch, they just add complexity to ensure that the amplifier read the correct value. We’ve since moved to eight-level cells, where each cell holds an entire byte, and the error correction circuits are wild for the sense amplifiers. But all NAND FGMOS leak, so if you pack eight levels into a single cell, even small leaks can be the difference between sensing one level from another level. So at some point packing more levels into the cell will just lead to a cell that leaks too quickly for the word “storage” to be applied to the device. It’s not really storage any longer if powering the device off for half a year puts all the data at risk.

So once going upwards and packing hits a wall, the next direction is moving out. But the more you move outward, the further one is placing the physical memory cells from the controller. It’s a non-zero amount of distance and the speed of light is only so fast. One light-nanosecond is about 300 millimetres, so a device operating at 1GHz frequency clock has that distance to cover in a single tick of the clock in an ideal situation, which heat, quantum effects, and so on all conspire to make it less than ideal. So you can only go so far out before you begin to require cache in the in-between steps and scheduling of block access that make the entire thing more complex and potentially slow it down.

And there are ways to get around that as well, but all of them begin to really increase the cost, like having multi-port chips that are accessed on multi-channel buses, basically creating a small network inside your SSD of chips. Sort of how like a lot of CPUs are starting to swap over to chiplet designs. We can absolutely keep going, but there’s going to be cost associated with that “keep going” that’s going to be hard to bring down. So there will be a point where that “cost to utility” equation for end-users will start playing a much larger role long before we hit some physical wall.

That said, the 200 domain of layers was thought to be the wall for stacking due to heat, there was some creative work done and the number of layers got past 300, but the chips do indeed generate a lot more heat these days. And maybe heat sinks and fans for your SSD aren’t too far off in the future, I know passive cooling with a heat sink is already becoming vogue with SSDs. The article indicated that Samsung and SK hynix predict being able to hit 1000+ layers, which that’s crazy to think about, because even with the tricks being employed today to help get heat out of the middle layers faster, I don’t see how we use those same tricks to hit past 500+ layers without a major change in production of the cells, which usually there’s a lot of R&D that goes behind such a thing. So maybe they’ve been working on something nobody else knows about, or maybe they’re going to have active cooling for SSDs? Who knows, but 1000+ layers is wild to think about, but I’m pretty sure that such chips are not going to come down in prices as quickly as some consumers might hope. As it gets more complex, that length of time before prices start to go down starts to increase. And that slows overall demand for more density as only the ones who see the higher cost being worth their specific need gets more limited to very niche applications.

What can men do against such reckless hate?

Lucky bastards.

That’s not untrue though. The number of times the average consumer has over corrected to only put themselves into a position where they over correct yet again is sobering.

I’m not dissing AI or the average consumer, I’m just saying they often paint their asses into a corner over and over again only to complain when they paint themselves into a corner.

Number of times I’ve heard people complain about having to pay the monthly price for iCloud is a non-zero number. But then you suggest getting a phone with an SD slot and they’re aghast that you’d suggest them giving up their blue bubbles. They hate ads on their Smart TV but how would they watch their Disney+ without a Smart TV? I’ve got a CVS receipt sized list of things consumers complain about that consumers wanted but now find themselves in the awkward position of “not like this.”

C’est la vie. But there’s not going to be any kind of discourse that’s going to prevent folks shooting themselves in the foot. All we can do is just present some other options for when they inevitably have a bloody stump where their foot used to be. They’re going to use AI no matter what and eventually they’ll be wondering how their back is against the wall with all this paint surrounding them. Yet again.

I don’t think there’s any point any trying to persuade them of anything really. All we have to decide is what to do with the time that is given to us. So yeah, the average consumer is completely going to snag these shit PCs up in droves, they also are going to eventually regret that, but that’s a future them problem apparently.

You and the person you replied to are correct.

Honest question as I only use Windows every once and a while. Can you get to that disk management thing from the Control Panel? The one that looks like gparted.