Keeping the old stuff has always been a mess. Lots of focus has been placed on redundancy but not on backups. Here's the next step in our long term storage! gosymply.com/ Use code: SLOMO5
Having worked with many versions of LTO drives/tapes over the last 20 years I know all too well that they can/do fail, don’t fall into the trap of thinking that the data/tapes won’t degrade over time or get chewed up by a bad tape drive leader. I highly recommend you create two tapes for each backup and keep a good backup of your backup library master data base, so you don’t have to rescan all the tapes to rebuilt it. It’s also important that you follow the correct storage guidelines (temperature, humidity etc)
I was thinking the same thing, ... I also worked with tapes and they are not always reliable... storage is an important part; I would keep the tapes in an airtight box keeping it away from heat.
I too had worked with tapes in the past with a large 50+ tape machines and was thinking the same thing. At least once a week a tape would fail requiring backups to be ran almost constantly. Plenty of multi tape systems out there which could allow easy striping as just a backup... Of the backup.
As an IT guy all I have to say is "MAKE SURE YOUR BACK UP YOUR CATALOG DATABASE" this is the database that tells you what is on each tape and test its restore every so often on a test machine. This is so important and no one thinks about backing this data up. Make sure the backup is automated and is taken off site and ensure you have multi revisions of this database. If you don't have this catalog all your tapes are useless or you will have to re catalog all you tapes. If you only have 4 not an issue but once you get to a 100... you could spend a month cataloging your tapes!
@@jirehla-ab1671 I'm sorry you want to reuse your SSD or HDD after it's repaired? If you have any issue with any media you 100% replace it and recover the data to a new medium if it's important
@@steve_main cause I already backed up the data to the cloud i am also planning to backup it to my other google drive, do you think it is also better to seek replacement through a warranty or just buy 2 new hdd / ssd instead?
@@jirehla-ab1671 If it's under warranty then have a replaced under warranty. why would you buy new hardware if it's under warranty? 100% replace it if you've had any issues with it. if not, any data you put on it expect it to be gone
@@steve_main cause for me I accidentally fall it to the floor and I don't know if there's already physical damage to the external hdd and western digital said that the warranty will be VOIDED if there is a physical damage and will NOT be replaced
If you really care to preserve everything then 3-2-1 backups are critical. Tape can be a key part of this, but it can't be the only thing. 3 - Total copies 2 - Different types of media 1 - Copy at a different physical location
plus 1 - Copy offline. If you delete a file unintentionally and the deletion is immediately mirrored to all your backups you are screwed. Better have one backup run from time to time so you can go back to your backup from last week or so, where the file has not been deleted yet.
for example: a file saved on your computer and mirrored to the cloud plus backup every x days to an external hard drive fullfills all of those criteria.
As has been said below already, make sure you have two copies. One on-site and one off-site. The fact that you haven't lost data is more of a miracle than anything else. Also tapes do suffer from bit rot as well (they are magnetic as well) Insure you store them in a stable environment, while they are great for archive storage the archive part only works if they are treated as per their specs.
Yes, now he has 45 tapes, time to get a second set of 45, a second drive and make a copy of each one as it is done. That will at least give single redundancy. Then a nice data safe that is rated for at least 3 hours fire resistance to store them in, preferably in a room above flood level, and with non flammable walls, and temperature control. That 30 year life assumes storage at constant 20C and 50%RH, which is only achievable in a massive data centre, and also assumes that you use no compression, and maximum error correction on the tapes.
To repeat what others already said... But since it's important and could avoid data loss. Do NOT expect 30 years out from those tapes. LTO is by no means safe from data/bit rot and for important data you should rotate every 5-10 years depending on storage conditions. The 30 years they market them with is during very specific conditions when it comes to temp and humidity, treat it as marketing buzz. Also, do NOT throw them around in luggage and avoid X-rays, strong magnetic fields and so on. And for the love of God, do NOT drop them. They are fragile and you just have to disassemble one to realize exactly how fragile they are. Have had more then one eaten by tape drives in the past (LTO4/5). You should have two copies of all critical data. If possible, use LTFS to avoid being locked into one vendors proprietary db's and system and keep backups of any eventual databases. But yes! When dealt with properly and when knowing about the quirks and limitations and gotchas LTO is very cost effective and nice. But like everything, LTO have its limitations and should not be treated as a indestructible bullet proof Jesus-medium that will save us all. 😎👍
While this is a big improvement vs random hard drives, as someone who archives data for a living I'd caution you against just putting everything on these tapes once and assuming they'll be safe for 30 years. That guarantee is more like a warranty than a specification, and I've had tapes fail much earlier than their rated lifespan. An immediate solution would be to use some form of parity or mirroring to ensure every file can't be lost with a single tape failing, but longer term I'd highly recommend backing up your archives using multiple forms of storage, in multiple locations, especially since this is your livelihood and if something like a flood hit wherever you store your tapes, you'd lose everything. A cheap option would be to make a JBOD server using your existing HDDs you just moved your data off of. JBOD is short for just a bunch of disks, and is exactly what it sounds like. It would also let you use a filesystem that automatically checks for and fixes bitrot, like ZFS, and mean you would have access to your entire archive without having to plug in a certain tape, or even multiple as LTO tapes can store a single file across multiple drives to ensure you're using all the storage it offers. This is why they have the barcodes as standard, as many commercial archives using LTO tapes will have a tape library, which are basically jukeboxes with tape instead of vinyl, and the software can automatically switch tapes to retrieve data spread across multiple drives.
Not to forget that that spec only applies if you store the tapes and drives dust-free and climate controlled (temperature and humidity). That's a lot to ask even for a small business.
Magnetic tape is so cool, i'm still shocked after i found out that a vhs can hold gigabytes of digital data depending on the length. 18tb in such a small package is unbelievable data density
Theoretically yes. vhs is helical scan, the data is on a constant spiral, not linear. it doing so because vhs is optimised for analogue video, each scan line is physically present on the width of the tape. data however, is linear in nature, thats why we have data tapes like lto are always store in linear fashion. makes reading and writing much faster and searching easier.
@@gernhartreinholzen3992 Back? Tape never left at least in industrial and professional enviroments. Lto is a backup standard for decades now with 9 generations of tape and more are planned so its not obsolete by any means.
@@gernhartreinholzen3992 We never stopped using tape. A lot of supercomputers have physical tape storage libraries where an arm chooses a tape and inserts it into the drive for the computer
I've worked with the LTO tape drives for a long time, although I worked primarily with Tape Library machines, that is, robots that you load dozens of tape into and has more than one drive. The backup software has a master database of all the files and tapes, and does backups, automatically changing the tape when the one in the drive becomes full. As other people have said you need *two* backups of everything, and store one copy in another site, preferably that is climate controlled. You should also periodically verify your backups. In the long term, you should also consider budgeting to change storage mediums every 5-10 years. Just like you are going from Hard Drives to LTO Tapes, in 5 - 10 years, you should be looking at the next great thing...maybe SSDs, or maybe holographic storage...
I've used LTO-4 for 12 years, and suddenly tape drives fail and HPE has forgotten how to repair the old drives, apparently throwing away all the repair manuals. That plus the nightmare of Indian support outsourcing plus only about 2 companies left actually making LTO products, despite originally introduced as a monopoly-free category as DLT had become a Quantum monopoly. So, an old suggestion that I sadly ignored, is to purchase a spare drive (hugely expensive), test it and store it with the off site backups. Once you need the spare, immediately transfer all backups to a new physical format that can be secured with the same measures. Oh, and in case you missed the small print: New LTO versions cannot read tapes from previous versions, except for one overhyped situation. Now I'd need to purchase an otherwise useless old LTO-5 drive to read the old tapes, while also investing in a new format to store new backups and copies of the old ones.
@@johndododoe1411 I totally agree with everything you said, especially the backup drive unit! But I would rotate it in every once in a while since it is bad for equipment to sit unused for long periods of time.
Just don't fall into the trap of routinely cleaning an LTO drive, clean it when the drive requests a cleaning by itself or in case performance takes a sudden dive.
Specifically, the Single Character Display (technical term for that amber 7-segment display) will display "C" when it wants a cleaning tape. Unless there is a serious problem, it will wait for you to eject the tape that's in there first before displaying it. As well, the drive has a brush built into it. Every once in a while, you will notice it will stall for a few minutes while it unloads the tape, brushes the head, reloads the tape, and then goes back to the point it left off. Another thing: If you are writing a lot of small files with LTFS, consider using the sync_type=unmount option, so that it's not spending a bunch of time shoeshining between the index and data partitions.
For someone who only genuinely enjoys the content and knows nothing about the logistics and workings of your results, its really cool to see this. As much as I wish you both were super close to have more time in this channel, life moves on and project priorities change i am glad you keep slowmoguys still going!
Of all the RU-vidrs, your channel has been the one that has been in the back of my mind constantly. as someone who's a tech enthusiast, knowing how much raw data you generate every video and trying to come up with a solution had to come in occasional mental exercise for myself. I knew tape would probably be your archival solution but I didn't know the technology had progressed to the point where the write speeds would actually be faster than most hard drives. Your probably still needing a Linus grade storage server for, effectively, a scratch disk during editing but now you can of-load one your done and can stop having to build servers. Sweet
Thanks for the tech update. It's always interesting seeing storage problems solved, that us mere mortals (in a data size sense of the word) will never face!
Gav has always been so good at making the tech side as interesting as the demonstration side. Cheers Gav! I am from Bristol! :D I live in Louisiana now, I miss the UK :(
I've always found these types of videos interesting, but since I started properly working as a video editor and working with VR video a year ago these are absolutely amazing. That much storage is a dream
That's bonkers! I didn't know such efficient storage even existed! What also surprised me was that 8mm film converter on the shelf there; I didn't expect you of all people to have one of those, lol!
I have 1.04PB of storage which is backed up to large archive drives. I thought about using tapes, but they're likely to degrade at the same speed as a hard drive. The fundamental difference is that with a hard drive you can repair the fading magnetic fluxes. So every few years, just copy off the data, reformat the drive and then rewrite it. This is something that would be more difficult to do with tape. So as someone has pointed out... Don't think of tapes as a very long term backup solution. Not unless you're willing to maintain them too. Also bear in mind that unlike a hard drive, your tape machine is going to wear over time and you also run the risk of tape getting chewed up. These issues are less of a problem to a hard drive as it is a self-contained unit. Moreover, tapes will be much more sensitive to things like moisture and temperature than a HD because the platters in a HD are sealed. You do run the risk of motors failing in spinning discs, but this is more likely through use, rather than sitting idle and certainly less than the degrading of a LTO machine. So always have two drives and make sure they're refreshed at a reasonable interval.
You can repair the magnetic flux as you say, if the rest of your drive still works. Thats the thing right there, these tapes dont have any subsystems that will fail and entomb the data in a dead box. If the tape drive dies, you get a new one. As this is LTO 9 you could argue it may be cheaper to try and move a dead HDD's platters to another compatible drive, if you have someone who has a working drive to transplant them into. Plus the tapes are very immune to fading like old reel to reel audio tape can be because they are made of very different stuff. You are more likley to have problems with tape damage over anything to do with the magnetic fields.
@@dlarge6502 I've used tapes, lots of them and the magnetic fluxes of tape degrade at about the same rate as a HD. It's slightly higher for tapes because tape layers are wound upon each other. HDs die from use, not usually from sitting idle. They die either from motor failure or the controller dies which is why they have a MTBF indication. Each HD is self contained so the MTBF is spread over the number of drives for the archive, but a tape machine is used for all the media, so it's failure rate is significantly higher as is the chance of it chewing or having a r/w failure. The storage requirements in terms of moisture and temperature is also significantly higher for tape. Although admittedly, moving platters between HDs is extremely tricky, it can be done as a last resort. But get a chewed tape and it's game over. The only real benefit a tape gives you over a HD is from shock resistance, if it's a spinning disc and size. This can be eliminated with SSD storage though, obviously at a higher cost. Add to that the random access of a HD coupled with the higher transfer speed, especially from SSD, the HD always ends up on top in terms of long term storage.
@@thewelder3538 Well, at least your second comment made the argument sound more factual and not just a HDD fanboy making stuff up (your first comment gave that impression).
@@johndododoe1411 Agreed, it was a little wishy-washy. Although I do enterprise backups, it's hard to find the level between trying to sound like the ultimate authority on the subject and someone who's just a fan-boy.
I'm just going to echo what other people are saying and strongly consider duplicating your backups and storing them off site somewhere. Redundancy is just as important as backing up the data itself. It doesn't do you much good to back up all your data if it's all sitting in one location waiting to get destroyed in a fire or a tornado. You have a lot of very valuable data and you're now stepping into the world of enterprise level solutions. I would find an off-site backup location in another city, or state if you can mange it. Redundancy, redundancy, redundancy!!!
Former IT gal here. I can't stress hard enough how right all the other IT professionals and data archivists commenting on here are. You need *_at least_* two copies, one stored safely offsite.
"All in the same format, and I only need one device." I would suggest buying three units, one of which is tested and repacked safely, and that you save the data on at least three tapes. And then make Dan keep that third copy of the tapes at his house, in case there's a fire. Also, keep the original hard drives as a last ditch solution. If you could copy the data to clean fresh disks, that'd probably be good too.
Our biggest fear in our computer workshop and data storage room (at a university) was that one day, someone would come in with a degaussing wand and destroy all our backup tapes. We were so paranoid that we had an off-site secure storage area that only 3 people knew of.
That's a bit much don't you think? Wouldn't that storage room only be able to be accessed by verified people anyway? Or was it not locked down. Sorry, not trying to come off as rude just genuinely want to know why.
@@johndenver6769 The uni didn't quite have the budget or sense to provide a secure server room. So we took our own measures. Losing the data of thousands of off-campus distance ed students wouldn't have been a good thing. Not to mention study materials etc. We looked after data for the publishing section as well.
The barcodes were used for larger units which had mechanical arms which auto loaded the tapes into the reader/writer all enclosed into a cabinet which would live in the rack room. I remember those days...
Very helpful! We’re going to need a long term backup solution for our observatory data, and this is exactly what we’ve been looking for. Definitely going to back up with tape! It makes it so easy to back up off site
@@Bruh-zx2mc yup! We’re going to have our NAS store the main sets of data, and then tape on site and off site for long term storage. Our current NAS is 36 tb but I’m sure we will get it closer to 120 tb. We will save up for tape when we can afford it
It's awesome to see how things evolve. It would be cool to see what your production path looks like from recording to ingestion to process to archive. Do you still have the 45 Drives server as well from Linus? Great video!
Love that you now have more cohesive way to store your vids, wondering what happened with the NAS LTT helped you out with. Love all the vids! Cannot stress enough that you should always have AT LEAST 2 copies of all your data (or anybody else reading this comment)
just wish the tape drives weren't so expensive. Or it was easier to rent them or something like that. since archived storage that doesnt need to be accessed you dont need the drive all the time.
The NAS would be great for more short term storage when he needs access to data but his local computer doesn't have enough storage/he might want to reference later. The tapes are great for the long term storage and archiving of his footage.
I remember fixing a pretty big tape library machine a while ago (a tiny gear had cracked), it was super cool to see how it worked with a moving, mechanical arm on a pulley system to move the tapes around and stuff inside
I find it crazy that these machines that cost sometimes as much as a house use any plastic gears in them. I suppose they are expected to be decommissioned and superseded long before the plastic gears degrade, but that's clearly not always the case.
@@jamescollins6085 Hehe, the one I worked on wasn't quite a house in terms of price, but it sure saved the client probably a good few thousand euros. Bought a pack of gears from ebay, used a soldering iron to melt it onto the shaft as it was smooth.. and a few attempts later it worked just fine
Another benefit of tape is how natural is to have offsite and offline backups (the first protects you from disasters like fire or crime and the second one protects you from ransomware)
<a href="#" class="seekto" data-time="46">0:46</a> Some fun facts: 720p at 82,000 frames per second is equivalent to 8K video at 2,278 frames per second, and an hour-long 720p@82,000 video that is actually 1 hour long and not a 90 second slo-mo stretched to 1 hour would be 324TB, and he would only be able to store 2.5 1hr long "720p" videos in his 810TB drive. Even if you somehow found the capacity to store a video that big, good luck 1. finding storage that is fast enough to show 82,000fps 1280x720 video (90GB/s is 12x faster than the fastest consumer-grade NVMe storage that is available today, and that $850,000 1 petabyte 100GB/s storage server that Linus showcased in his video a couple weeks ago (normally $1,000,000 but ignoring the $250,000 GPU cluster with 8x Nvidia A100 GPUs that is irrelevant to the storage portion of it that it includes) would only *theoretically* be able to handle it), 2. and *then*, good luck finding a CPU fast enough to decode (play back) such a video in real time. This is not a perfect comparison because of different codecs of varying quality, but knowing that my M1-powered 13" MacBook Pro can't quite play back 8K@30 VP9 video on RU-vid without buffering, you'd need a CPU 76x faster than an M1 to be able to decode an 8K@2,278 VP9 video uploaded to RU-vid (assuming RU-vid even allows that video to be uploaded). Even if you bought 2 of the just announced $12,000 flagship AMD 96-core Zen 4 Epyc server processor, theoretically it would only take you 40% of the way there (30x faster than M1).
Until I retired, I was heavily involved in the design of truly large scale mission-critical IT systems. I was also the first in the company to exploit LTO (generation 1). It proved to be a good bet on the technology, and we eventually had massive robotic libraries which went through generations of LTO. However, one single copy is not enough. If you are going to use if for archive, then do keep multiple copies. That's a lot of data to lose in one go when a tape fails. Also, one copy should be kept off-premises. Also, be prepared to do a media conversion with new generation of LTO, and it's very good practice to periodically refresh anyway, to prove you can read a tape and then you can deal with a failed tape using one of your alternative copies. This also means a single drive isn't going to do the job either. Note that an LTO can write data a lot faster than it can be read off a single HDD. What you do not want to get into is a situation where you LTO drive is not streaming and you are in write, run out of data, rewind a bit and start writing again. It massively slows throughput and is very bad for the tape mechanism and tapes if it's extreme. LTO drives will throttle down write speed to some extent. It it's a problem, then stage the data to an SSD first before backing up. Of course writing large amounts of data to an SSD will affect it's life too, but at least it would be done once. In our case, we had massive enterprise array storage systems which striped data over RAID sets and could keep up (whilst still keeping systems going), but even then it could still be a problem to keep the LTO drives streaming efficiently. Also, if LTO is used for archival purposes, the tapes must be kept in a temperature and humidity controlled environment free of dust. Nothing too drastic, but something to be careful of. Extreme heat would be bad for them. There is quite a lot more to backup and archive on tape then there might appear to be, and it really requires experience. Of course, if you are keeping tapes and the original disk drives that does limit some of the risks, but this all needs a lot of care.
@@HydroHUN I'd say hard drives experience mechanical failure ("click of death") long before bit rot. The HDD in my IBM Model 30 from 87 still works and that machine was rescued from a damp container at the recycling center
@@sylvester4207 It is semi manual, he inserts the tape drive and it just goes in the background. As far as I understand the file movement is automatic as long as the drive is not full
Part of the reason the BBC wipe over recordings was cost (magnetic tapes cost much much more than they do now), another key reason was contractually in some cases they were not allowed to reuse recordings, there was no reuse infrastructure or licensing agreements back then. Many historical recordings were destroyed just because they were not allowed to reuse it, even on film as opposed to on tape.
Just to chime in here: definitely keep a second storage backup if it's world-ending if you lose the raw. Tapes are pretty hardy but the chance of a file corrupting or at least degrading in that time span is pretty high (though better than a HDD). Would reccomend a second tape backup for any files you definitely need, and a third backup in a different storage medium (HDD, cloud, etc.) for anything absolutely critical. 2 tapes will do you for 30 years, but not if there's a fire where they're stored. Off-site backup in a different medium is airtight for anything critical and gets the best results :) But if it's just nice to have, one tape will keep the majority of your stuff for the 30 years bar any catastrophe
I've somehow managed to cram my whole computer life into 1tb up till now. Finally had to order a 2nd TB last week though, because my old 1TB drive is completely full and I don't have anything disposable to free up space. Managing almost a petabyte of data sounds hard.
wow, LTO tapes take me back used to run multiple systems, with multiloader systems. Haven't touched hardware now in 10 years, with the advent of cloud. But interesting to still see it in full use.
This, this and 100% this! Please make sure you have an off-site backup of your entire library; a theft, fire, flood can wipe out everything in minutes...
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive. So no, an LTT server isn't necessarily the best choice.
Make sure you do regular catalog backups. You should also consider using a tape vault service. That way its offsite for DR redundancy, and they will keep it in a humidity/temp controlled storage to keep them in the best condition so they last for a long time
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Storing on harddrives, would be incredibly expensive.
Only just found out about LTO tapes, and was hoping to get a few LTO 9 drives for my media server. Also saw an article stating the LTO 14 drive will have over a petabyte of compressible data, which seems super cool.
Every time I look into tape I am shocked at the prices of drives. Been sticking to HDD and a HDD dock and basically using the HDDs like tapes. I need to completely revamp my backups though so I can have more automated versioning. Right now it's kinda manual and is just me choosing what drive to put in to update to latest backup.
I'm shocked not only by the prices of the drives, but of the tapes as well. In many cases, you aren't saving much over the cost of a comparable hard drive, and you have the added inconvenience of packing and unpacking all the data every time you want to access any part of it from the tape. I also don't understand the obsession with keeping source material that so many RU-vid creators seem to have when the bitrate that it will be compressed to when streaming is hundreds of times lower than source anyway. I'm sure he could re-encode the footage using the new AV1 encoding format and save mountains of space with no decrease in visual fidelity. Perhaps like with photography, it is necessary to keep the files in their raw form to aid in post-processing. Either way, it is his footage to treat in any way he sees fit.
@@jamescollins6085 The tapes are brand new, this is literally the first time I've seen one outside of a product sheet, thus they will be pricey for a few years. I'm totally shocked that anyone would pay $1000 for a phone too.
CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive. I'm pretty sure, they wouldn't use tapes, if they sucked!
This was basically just an advert, but very interesting and nice to see Gavin is getting more help in the inevitable data hoarding that this job requires. Linus tried, but the Slow Mo Guys are on another level of storage needs.
@@akyhne They gifted him the drive and tapes in exchange for him creating this video in which their brand is mentioned to hundreds of thousands of people. That's literally what an advert is.
A few notes on LTO as someone who's used it extensively for video backup: 1) Generational compatibility -- while they tapes may be guaranteed to last upwards of 30 years, the same definitely isn't true of the drives. A new generation is released roughly every 3-4 years. So LTO-10 is due in about 2 years. And LTO-9 drives will likely still be available and in warranty for a couple of years after that. And LTO-10 will read LTO-9 tapes. But once LTO-11 is on the scene your days are numbered when it comes to being readily able to read LTO-9 cartridges. 2) Tape failure -- they are not perfect. Probably more reliable than 3.5" drives in cold storage, but not 100% reliable. Our backup plans always called for duplicate copies, stored in physically separate locations. On restores we did see failures. Also, while they're a great low-cost high-reliability storage, tapes are slow and annoying in many ways. I've typically preferred to keep media backed up on 3.5" drives also, as a first-instance archive that I can generally access quickly and with little effort.
Didn't you get a 130TB storage server with LinusTechTips? Or was that never used? Edit: Thanks guys for the update. I didn't know it filled up that fast !
This is what Linus should have supplied during your collab, tape is the obvious solution to your problems. When I interned at IBM in a datacenter position, they had a machine with hundreds and hundreds of these tapes. Unit had a robot arm to read the barcodes in the library of tapes and load the appropriate tape on demand. Had to be petabytes of data on them. I remember the 4 tape heads on the high speed recorders were named John, Paul, Ringo and George lol. Someone there had a sense of humor. Then a guy would manually load them from the high speed output to the reading library with all the others, and add the barcodes to their spines.
@@jshklsn Linus is good for general education and entertainment. But his approach o industry standard equipment is usually quite lacklustre and backwards. His use of storage systems is quite painful to watch.
To be clear, I'm on the side of using things for what they are designed. Need to store massive data over quite a few hours? Tape. Need to store data real fast, multi-user? SAN or something local. I just think tape better fits this use case.
I agree it's tragic that so many shows and pieces of footage are lost forever just because some corporation was too lazy to store a copy. I love to watch old entertainment from back in the pre-internet ages. I often wonder if the old lesser known cartoons I used to watch still exist in a vault somewhere, or if they only exist on random people's VHS tapes.
If you're curious about how much it'd cost to store 810TB in the cloud while keeping access to it, you're looking at a monthly fee of about $3,000. As a backup solution (accessible once or twice per year) it's a tenth of that.
DO NOT take the tapes on a plane, the scanners may wipe or corrupt them. When you are copying, you need to use file verification. Nothing worse than thinking you have a file, only to find it was corrupted right off the start. Take the original drives and store them at another location. Also, you may want to create some parity files (.par2) just in case it gets corrupted later or as a later verification of integrity.
Scanners won't do anything to modern tapes. Maybe 30 years ago the motors that drive the conveyor belt might have caused problems for low-density tapes. High density tapes like LTO need a huge amount of flux to cause issues. Scanners won't do it.
I hadn't heard about this tapes, but it sounds a lot like a miracle product. I think it would be best to use them as one more backup besides the HDDs and SSDs.
Good old SCSI! I had a SCSI drive with my DVD burner, back in the day, where your computer was more or less useless, in the time the DVD drive was in use. The SCSI setup had no such problem. That was in 1998. I got the SCSI card for my 30th birthday.
Thank you for sharing this! If I really want to get more into photography and videography, then I should probably invest in these systems of storage. I've been archiving everything I've taken too, and my one external hard drive with 2TB is starting to run out of storage :/
No, we are not going "back". Tape has always been essential for large capacity storage. 20 years ago, I worked for a company, that made the physical negatives for color printing. These were cut by lasers. Each project took around a few gigabytes, which was an enormous amount of data, back then, when you had hundreds of costumer projects. CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive.
It will be interesting, As a former main frame computer engineer form the days of huge tape files, in general a tape reel would be at least a mile long and often twice that. We used to run archive tapes from end to end periodically in fast forward to make sure they wouldn't crosstalk between tape layers with time. Tape can and does deteriorate over time, but good luck. The bigger the tape the more you loose when it goes bad! Add to that access is serial (because it's a tape,) getting to a file at the other end can take an appreciable time where as solid state or HD is much faster to access.
We are not going "backwards". Tape has always been essential for large capacity storage. 20 years ago, I worked for a company, that made the physical negatives for color printing. These were cut by lasers. Each project took around a few gigabytes, which was an enormous amount of data, back then, when you had hundreds of costumer projects. CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive.
So, we're back to where we were forty years ago. Tape on cartridges/cassettes for data storage. Gives the micro-tapes from the original Star Trek new credibility.
No, we are not going "back". Tape has always been essential for large capacity storage. 20 years ago, I worked for a company, that made the physical negatives for color printing. These were cut by lasers. Each project took around a few gigabytes, which was an enormous amount of data, back then, when you had hundreds of costumer projects. CERN (The large hadron collider) stores around 1 petabyte every day on tape. And that's only the essential data. Most data is thrown out. They build their own robot system, to load tapes, when needed for analysis and storage. It's quite impressive. Their storage capacity in the the late 2010s, was over 600 petabytes. Storing on harddrives, would be incredibly expensive.
I just saw a short video on RU-vid, from CERN. They explained, that they are connected to 170 data centers around the globe, to process the data from the LHC. They also have their own computer on site, with over a million cores.
