Okay, I guess my sarcasm level wasn't quite high enough on the Raid Shadow Legends bit-I will not accept sponsorship dollars from any company I don't trust/support (and I don't trust/support Plarium). Lambda is this video's sponsor, and they are awesome. They also have GPUs that work with their computers. I'm still trying to get one working on a Pi :D
ZFS requires as much RAM as you can get. And ZFS compression can chew whatever CPU you have. Also, compressed and uncompressed ARC can make a difference for underpowered systems.
7:53 Good to know that was a goof. Although next time you use that one (or one like it), hold your nose and force a deeper voice-- with your normal speaking voice the bit fell victim to Poe's Law.
Same here. Mine has been running for about 4 years. Skating on thin ice with a single drive. But I just bought a R-pi 4 and a pair of 3T portable drives to build version 2.0.
@@gordonreeder3451 i won a raspi 4 a little while back, so it set it up with a single 2tb hdd that I had for some reason, and ive been using it as a kodi box mostly. Its not been running continuously for very long, i need to restart it relatively often, but its been serving me well for a year or two now
@@Aruneh Makes sense, as he doesn't have advertisements blowing though the wazoo just to deliver his material. Just a simple crate or project package shows up and he tackles it. LTT might as soon well be a large daytime Linus Truman Tech Show.
Take the sponsorships and have fun. You're every bit as fun and interesting as those other big guy channels. You're no Wendell, but those guys don't have a red shirt Jeff, either.
0:34 THANK YOU for the timer. This should be mandatory on YT (and I don't always skip sponsor spots because sometimes people have a genuinely good product, it's just better to know when they end)
This channel, and Explaining Computers are the two channels that have expanded my Pi horizons over the past couple of years, so many projects planned now.
Paused the video to say this. I LOVE this advertisement segue with the timer and everything! I know people can't always have completely relevant adverts, but when they are they are still kind of jarring. This one actually confused me as I wondered what the timer was for because the advert itself was so contextually linked to the content! Top, top work there Mr Geerling
Subject: How to conceive how hardy a raspberry pi can interface with a ton of NAS storage using only 20watts in a compact form factor. Expectations: The board has to last a long time or you want a quick off the shelf price well within budget and high output isn't what your interests are. So, this was a great video! Not only did it show if the top shelf storage devices could work with the raspberry pi as if you were largely concerned about space, modularity and a near unlimited private storage network on a minimalist agenda,, but to consume less than 20 watts. I really enjoyed the review on this RaspberryPi device.
I think one potential real use-case could be as a NAS or as part of a "homelab" for a digital nomad, i.e. someone who lives out of a van or travel trailer. Power consumption is a big deal, especially when you're off the grid. Having so much space but sipping power compared to just about any other implementation makes this a potential solution rather than just a fun experiment.
Agreed. I’ve been spending months at a time in my 30ft class A. I don’t have the luxury of full time unlimited power like I do at home. Everything I use in the RC has to be low power as possible. I’d seriously consider building one of these, maybe 20TB or so.
I loved the sarcasm regarding the next sponsor :D Also, send Anthony a shirt, he’d probably appreciate it considering he seems to like to dabble in Linux stuffs :)
Jeff awesome video! Especially because of all the documentation you keep providing us for these 'niche' use cases. I'd love to hear your take on using old enterprise disposed thin clients to set up a storage server like this one. I know you're the pi guy and that a video would come with natural biases, but would still be awesome to see
I actually used to use such PCs for a few projects, but the problem I had was the chassis was always too small (and expansion too limited) to have too much fun without ending up having something resembling a mini-tower, just without the nice mini-tower all-in-one-case design :(
This looks really cool! I ran my old Pi 4 with Dual HDDs in BTRFS Raid 1. It worked well. Got nearly a gigabit on seq read and writes using nfs. SSH was also not that slow anymore. I think older Pis were heavily limited when any kind of encryption was wrapped and usually were what keept speeds under 3MB/s. These benchmarks were most interesting for me since my most convenient transfers were usually with SSH or a browser. So encryption was the limiting factor then.
If you take the sponsorship, you're only allowed to mention them on videos about RAID or the /etc/shadow file. Their sponsorships show up in some of the strangest places. I guess it's a spray and pray ad campaign? Also, +1 on the idea for videos about ZFS
Jeff, great video! I love it when I see someone trying to push the limits into the insane. This is where we learn and break down faulty thinking. Keep up the good work.
Jeff, where would computing be if there weren't people like you to do all the things that some may call "stupid" or "pointless"? Computing is as much about fun and pushing things as it is about making life easier work wise. So everyone, go and do stupid and pointless things with tech! It's fun and awesome!
Hey this is an awesome rig for an ultra powerful but super power efficient and compact proxmoxpi build! I know at least one RV-owning friends would love to have this to help with his RU-vid travel channel!
Man it’s awesome to see your progress as a technology knowledge advocate, no matter how seemingly “trivial”. Been subscriber since you reviewed some cheap lapel microphones many years ago. Keep the enjoyable content coming!
Yes raid shadow legends sponsorship would be awesome. I wonder if someone from LTT ever helped them out with those drives. I will try to contact them for you red shirt, Jeff.
That Taco board is awesome. You could probably build a decent NAS if you setup like 3-4 RPIs with their own boards then raid them together via network protocols like iSCSI. you would probably need something more powerful than a RPI to act as the SAN head though. But this board could be a building block to building a huge SAN for decently cheap.
The whole preamble about why you did it could be resumed to " does it make sense and should anyone do this? Most likely not. Then why did I do it? Because I wanted to."
$5000 for one person to learn is a bit to much for a single person, unless they have the money to burn, but when you're teaching many people, it almost comes to being reasonable.
cool vid, but i was actually hoping for a more "realistic" setup. a simple m.2 sata drive, 5 spinning disks. can you do a second edition for like "normal" usage?
I'm happy to highlight projects like theirs since they're unique and interesting to me; same thing with pretty much any product I feature-I don't have time to do boring things :D
Worked on something like this a number of years ago. There are a large number of memory crossings (data moved CPU Memory) for SMB. We rewrote/extended bits of the Linux network and file-system stacks so the CPU didn't touch the data unless necessary. This gave a huge performance boost for the slower disks/CPU/memory of that era.
Those SSDs would be crazy to just go out and purchase but The concept of having a low-powered server built out of SSDs you just had. As a video editor there are a lot of times that I could use just having 20 gigs of stuff just easily available LAN capable out of basically spare parts plus a little would be really fantastic. Like a client asks for something random I've done in the past. Or overnighted a hotel and you want to ingest and clean up all your cards and not fill up your laptop. Some of the photography solutions out there for Wi-Fi ingesting: you can drop $1,000 without the usability of a cool board like that. Thank you so much for doing this research definitely going to look forward to purchasing myself one of these boards as soon as they come out.
I'm really curious how this would do on a pi cluster. If the CPU is the primary limitation, would it be feasible to put together a cluster of pis to run the NAS and eliminate some of the bottlenecks on 4k random reads and writes?
Even for 4k random I/O, bandwidth may be an issue. Basically everything has to go over the same PCIe 2.0 x1 link (500 MB/s, full duplex) to and from each Pi CPU, and the 4k random performance is already within an order of magnitude of the bandwidth limit. Random I/O will have some protocol overhead, and plus ZFS RAIDZ also heavily amplifies write bandwidth since it needs to simultaneously write parity. The best case would likely be similar to existing solutions like Ceph; as far as I understand Ceph has a metadata server and for actual traffic directs I/O to the specific nodes containing the data in question.
Sponsorships are definitely less annoying if they somehow fit into the content. But if the alternative is lacking funds to do the project at all... then raid shadow legends it is.
Please dear Raspberry Pi foundation: Hear our prayers, double PCIe Lanes (3x would be perfect for this but double would be plenty already) with Raspberry Pi 5 and give it ECC.... It would kick every "NAS" Manufacturers **** out there even if it had to cost 50 Bucks for the 1GB version.
If you are going to overspec, then this is how you do it. Bit like plugging a ZX81 into a 8k display or driving a Tesla at 0.001MPH. After all, overkill was so last year, Overspec is the new thang and this is the current winner. Unless somebody has a super computer dedicated to turn an LED on or OFF.
"Jeff explains ZFS" would be an incredible video. There are lots of ZFS videos out there but they're all for Linux nerds. I'm sure yours would be a great step in between.
Yeah 99% of the time when I go view a ZFS video (like most Linux videos, too), it seems focused on minutiae that you'd have to already understand a lot of Linux and storage terminology just to get the video. I hope to do a video on it.
@@JeffGeerling Well ZFS is not your regular FS, it's highly specialized and enterprise oriented with much more than just a FS, almost no raid tool or volume manager is for newbies or home users. I'm sure you could simplify it to better explain the concepts to the average Joe, but a lot of advanced storage knowledge IS required to succeed and take advantage of ZFS, otherwise you're better off with the likes of btrfs or xfs + LVM for snapshots and basic functions not enterprise or enthusiast related. But hey, I'd love to see you dig deep into the ZFS rabbbit hole though! And giving it a new perspective or simplifying a little the concepts for mere mortals.
It's too bad that we don't have non SMR 2TB - 4TB 2.5" hard drives. This would be a perfect setup for that with a small NVMe cache drive it would be the perfect compact NAS.
An Anthony & Linus feat. Red Shirt Jeff video is something which will happen eventually. :) What is the thermal camera you are using? I'm thinking about getting one someday and to not break the bank. :D
Hi, Jeff, with all the small projects, we would like to see a paper based "cookbook" compilation of these technical performance journey. Very interesting especially for those individuals who wants to dig deep in performance.
I was going to worry for that beautiful nvme drive not having a heatsink, but... It probably is taking a nap because of the slow speed the pi can handle, so never mind! 😂
"Looker close" Heh :) I love those CM4 boards! Actually I hope to see an Thin Mini ITX board eventually as I have such a case sitting in my storage, even with an integrated power supply (Lunadesign DNK-H - but a WAY older model). Putting a Pi in there would be so neat!
You are going to enjoy the next couple videos, then ;) (Though I'm still waiting on a 'cheap' ITX motherboard for the Pi... the next two are not quite what I'd call inexpensive!)
I'm curious about the performance you got out of this. You weren't including a ZIL in your RAIDZ1 apparently? You probably would have gotten significant performance boost for your writes to the pool if you'd partitioned your Sabrent NVME drive and used that for it.
heh, I know my Dad still runs some 100 Mbps switches in his office. If it works, and you don't need the bandwidth, one nice advantage is you can just use 2P wire and save a bit!
Hard to tell from the video because the headers aren’t labelled, but does it have an RTC with a battery connector? Couple of JST plugs on there that could be candidates, but the Radxa page doesn’t mention it at all.
BTRFS!!!! it uses less cpu than zfs and lets you shrink the raid, and even convert the raid (eg. convert from raid0 to raid5) all while the filesystem remains online
