The primary factor for our continued use of Truenas Core over Scale relates to inconsistent performance observed with ARC, L2ARC, and LOG functionalities under Linux. During our testing phase on the new storage setup, consisting of 45 HDD drives, dual 40Gbe connections, 128GB of RAM, and 3 NVMe drives each with a capacity of 4TB for L2ARC, along with 2 additional NVMe drives each with a capacity of 4TB for LOG, we encountered peculiar behavior under heavy loading conditions, particularly when our workstations were processing large image sequences of .exr files for VFX purpose.
I run a 16tb nvme scale system on a ryzen 5 5600. Definitely overkill, but the asus hyper card made this so easy and I can expand to 9 HDDs with the case I got. Love the videos Tom! You've been a great help with learning to use PFsense, truenas, and proxmox in a home setting, so thanks!
I've been using Core since I built my nas and I have to say it's been rock solid even with multiple vms running. A friend of mine had to use scale for the hardware compatibility and he had some difficulties with the virtualization and the applications. I guess it's a bit of a tradeoff right now.
I have used both Core and Scale, I fine Core to be more robust. Scale has been buggy nothing huge just annoying. My NAS is just that, I run VM on XCP-ng, I feel better to have them separate so I done loose everything when a update or something goes wrong
Running Core as a destination for my Synology backups and has been solid, I did have to migrate the S3 service to their jails because of deprecation of S3 from the base services.
What really gets me is, hardware compatibility is much, much better with Scale, so if you're rolling your own, it's likely the better option regardless of your use case. If you're spending the money for Xi turnkey hardware, Core is undoubtedly more stable and performant, however marginal that may be, and you may as well go that direction with it if you don't need something that's specifically in scale.
"The BSD question comes up a lot." This might have been off the cuff, Tom, as you did the intro, but you said "Why would you deploy TrueNAS Core if BSD … maybe has not as bright of a future as people were hoping for." I'm assuming, and hoping, that you meant TrueNAS on BSD likely has a limited future, not BSD itself. But it does sound a bit like you're saying it's BSD that's going away. 😊 I'm using TrueNAS Core on my home NAS, and will cross-grade to TrueNAS Scale when there is a good reason to do so. For now, there is no such reason that is relevant to my needs.
Im still running Core. I have it as a proxmox VM. If i want applications, containers, or VMs, i run it on proxmox. I just require my TrueNAS to be a NAS. proxmox does the rest better anyway.
@@michaelmcallister711 i've been running it for 2 years now without issue. I honestly can't tell the difference between running TruenNAS Core as a Proxmox VM and when I ran it on bare metal for several years prior. What I can say is that I used to run VMs with TrueNAS Core as a hypervisor (TrueNAS Core uses FreeBSD's Bhyve) when I installed it on bare metal and the performance was much worse compared to running my VMs now on Proxmox.
I am in media production. Any possibility that you can make a video on TrueNAS for storing photos and video footages? That would be great if you can suggest what are the options for that specific purposes.
I failed to configure SMB Shared with complex group policy in TrueNAS-Scale (those option does not exist) So I rolled back to TrueNAS-Core where those policy (group-wise read/write policy with granular control) works just fine.
I've been using core for multiple months with pretty good results. Because of my setup I had to deploy it on windows11 and hyper-v which is not officially recommended. It worked fine until few weeks ago where the host win11 would have strange performance issue with the truenas. Both smb transfers and iperf (tcp only, udp was fine) were capped at less than 0.5Mbps however comms between truenas and other devices on my network were completely fine. It strangely was fine when I detached my pool but the problem would come back after re-importing it. Instead of trying a fresh install of core I decided to give scale a go and I was blown away with the performance. Perhaps linux is just more compatible with hyper-v than bsd but not only I can use my nas again, it performs better. I have some issues with containers but it's not a deal breaker since I have a separate device with proxmox for this reason.
I have always prefered core on my boxes but i would like to go all in on scale. What prevents me is just the huge work in reconfig. I have a bunch of minecraft servers that need huge rework to run well on scale. btw minecraft servers get a huge performance boost from the zfs caching, chunk load times are next to none on truenas hosts :)
I switched to scale early on and never looked back. Yes it feels a little rough around the edges still but I'd argue it's about as stable as core - as long as you stay away from all the extra shenanigans Scale gives you, which obviously you don't, since you installed scale :-D
I've used Linux for over 20 years and never used BSD (because I've never had a reason to). Couple that with the fact that I'm not running truenas bare metal. I mostly have experience and competency to navigate issues. With core, I'd have to go look things up. To put it simply, I know my way around Linux, I don't know much of anything about BSD, so I stick with what I know. This is obviously in a personal setting. If it was for professional situations, then yes, I'd have to learn more about it and give it full consideration.
Depends on many factors. Never going to be a single solution for everything. As a retired software engineer/architect of datacenter-scale technologies, I would never combine storage with virtualization on the same hardware. So Scale offers me nothing I want. On a broader level I would never give preference to a Linux solution for mission critical uses - especially storage. Too much chaos in the Linux model. Before the Oracle beast destroyed Sun I would never have picked a BSD solution over Solaris either. Like everything else it’s a matter of choosing the least undesirable solution for the situation.
With my little experience of last 4 years with storage servers and NAS, I completely agree with you sir. Storage Must be on different hardware box. Rest of the things can be combined in another hardware box. Combined with one or multiple 100 Gbps NIC with LAG. Let me know Sir if you want to add something.
I am just starting with homelabbing so I dont have a lot of experience. Can you explain why it is not recommended to have storage and virtualization on the same machine? Is it more important in regards to a datacenter infrastructure and less important in regards to a homelab?
@@zoglplayer329For a home lab it’s perfectly fine to combine. As is Linux for storage. Just ask yourself how important your data is to you. Personally I carry my prior professional biases with me into my home lab as many people do. It is overkill but makes me more comfortable.
Also when you split things apart it is easier to have 2 of everything, e.g 2 Sans, where the primary san is replicated to a secondary san, maybe off site, so,when things go wrong you have another copy of your data. Same with 2 virtual servers, one fails the other kicks in. Some services like web services can be easily pooled over multiple servers and if one fails it’s no big deal.
That would not be the best way, use ZFS replication. I have numerous videos on replication ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-9gKZOAL7yeE.htmlsi=TcXOY2tBfF8rTzYL
Just upgraded to scale and I do like the look of it but the more I use it the more I start to miss core. Think they need to fix a few things in scale, expecially with apps and vms as you mentioned.
What exactly do you miss? Core does not have Containers or KVM, so again, what exactly do you miss or don’t have? I moved all to Scale. Even the NAS only, no apps, servers. Don’t want to remember 2 GUIs. Yes. Scale Apps/containers can be messy, but they are stabilizing now. KVM is solid and they will continue to add options, features and stability/reliability. Let’s revisit the topic in just one year to see/confirm how far they are, and where FreeBSD is.
Been running Core on my home built NAS for the last 7 years. But I figure when its finally time to upgrade my hardware, I'll probably move over to Scale. Basically, my system is a NAS for my photography work as well as a Plex Server for my home entertainment. I currently have a 32TB (8x4TB) and I'd like to double it in my next system.
I'm running Core in my home lab. Running VMs for Zentyal and windows. Playing with Ubuntu server. Most of my issues are user ignorance because I am still learning. I am also running Plex, Asigra, and I am hoping to get next cloud up. I wish I had more time to dedicate to it.
TrueNAS Core all the way. Just asking a NAS to store files and share them eventually. Don’t need the features like virt or containers. I want it to be rock solid in his storage box role.
I realize when watching this that I am really out of my depths here - but still want to learn. I could not understand the different technical differences - but would like to know which one for a beginner like me who will use it as a NAS - but equilly using quite a few media apps and survaillance station with AI? I think I heard that the core was more app-friendly, but I guess time will tell. Still, seems I will get back to this channel quite often in the future. ;)
Neither or. I still run my proxmox cluster with one as a dedicated storage node. Works awesome with HA for the other nodes using that shared storage. Wouldn't like to be locked to one vendor (no matter if Synology or FreeNAS, TrueNAS or Unraid).
Coming from FreeNAS back in the day, I’m happy that TrueNAS Scale is on Debian. I moved away from FreeNAS to commercial solutions & ZOL because of the instability and constant broken things after updates. I’ve never had great reliability with anything BSD based. Kernel panics were somewhat common every couple months on “stable” BSD builds IMO. I have run Debian since 1998 or so and have always been happy with stability. One negative historically was always hardware & driver support for bleeding edge platforms, but that seems to have been resolved. I recently moved to TrueNAS scale for a couple projects. I am quite happy with the performance and stability. The only thing that I see that could be improved is the cache issue. I’m glad that is “to be resolved soon”.
I think, Core - it's real "Linux-Way", NAS - should be NAS. Scale - it's like "Windows-way", Nero-burning-rom-way - all (useless) stuff in one box. I apologize for this comparation 😅
One thing I haven't seen much mention of other than in the support forums is Scale's treatment of T-10 protected drives. Probably not a big deal for commercial customers putting new drives in the system, but in the homelab world, lots of people are getting used enterprise drives only to find that Scale starts throwing errors with the new drives. They still seem to "work" but considering the proliferation of OCD in the IT world, I suspect I'm not the only one unwilling to live with those ugly exclamations on my dashboard. Yes, I'm aware that many of those T-10 protected drives can be reformatted, but a lot of them can't. I've currently got 16 Toshiba 1.6TB SAS SSDs that are running great on my Core server, but simply are not happy with Scale and I've tried every combination under the sun of SG_FORMAT, SED_UUILS, firmware, controller, to no avail. The only option I haven't exhausted is showing up to Wendell's house with beer and pizza and a sack full of SAS drives, but I'm not wasting that option on something that isn't mission critical. Point being, until Scale starts supporting T-10 protection, or the kernal supporting T-10 protection or whatever it is that changed to cause the problem in the first place, is changed, Core can still be a pretty good option. (If you're happy with a NAS that isn't doing complex VMs or modern apps)
As a home user who just wants to have some simple file backups at my house, and try to de-Google my life a bit, TrueNAS Core is all I need. I'm running it on a really old machine from 11 years ago, and it does all I need. If I upgrade the system I'm using for my server, maybe I'll upgrade to Scale. Until then, Core is perfect.
It's too bad that Gluster is dead. I guess I should read through the GlusterFS mailing list posts more carefully (as I generally don't read them much anymore). I'm not sure about Core vs. Scale, but I would think that Ceph might be an alternative option to Gluster for storage clustering and scaling out. I also haven't tested HA with TrueNAS (neither Core nor Scale), so not sure how that works as well. In the past, I've used Core and that works really well as a NAS OS, better than scale (when Core was still on 12.0-U1.1), but I would think that Scale will likely end up just getting developed faster because it has a larger community behind the bits and pieces that Core uses. Conversely, I've been using Proxmox rather than either Core or Scale, and it is serving my needs well, and has completely displaced either and/or both Core and/or Scale, but I might do yet another 3-way head-to-head test again in the future. Who knows. Thanks.
Ceph doesn't like not having direct control over the block device. You'd be duplicating efforts, presenting a ZVOL as a block device to Ceph, where ZFS does metadata, checksums and all that on the raw device, and Ceph does the same on the ZVOL. I think you'd be better off using Ceph directly on the bare metal.
@@LtdJorge What makes you think that I was layering Ceph on top of ZFS? Nowhere, in my original post, do I even remotely mention ZFS, so why would you assume that??? Can you explain where you got this assumption from?
Truenas Scale is really solid for me. I am using it as a files server. I am running WireGuard, file browser and RustDesk apps. I have one VM running as well. So far so good I am really happy with Truenas Scale.
I'm running TrueNas Scale, mainly because I like Debian Linux on all my PCs. I'm running a Ryzen 3600 with 32gb of 3200 unbuffered ECC ram and 10tb, 4 - 6tb drives striped and mirrored. I'd like to upgrade to 64gb of ram but don't have a use case (possibly the next generation CPU too), this is already total overkill compared to my old Intel N330 Ubuntu Server NAS which never died. I'm looking forward to the next version which is supposed to have the 50% of ram available for ZFS cache issue fixed. I'm starting to look into 10gig ethernet too, but will likely hold out another year for managed switch prices to continue to fall.
As always, timely video again Tom. Still running Core, after coming from Freenas. In the video I see that Core has an “older” version of ZFS compared to Scale? And to what extend is hardware support better/worse in Scale? E.g. 10G network cards. Thanks again for this content!
So I mean like you said if you only want a NAS, like is in the name, then I would say core is probably the better bet. If you want anything more than that I think it is probably worth going with scale. Another way to phrase this would be if you are a home labber and probably running multiple services on a single machine go with scale but if you are a business that is going to have multiple servers in this rack for those services then Core is probably your better bet. Core can do a lot of things but in general those other things are either comparable or better on scale. They are also improving a lot on Scale. The apps, though confusing at times, really are powerful in the capabilities they add to a system. Sure this may not matter to you if you are running your setup on a computer that only has 8 CPU cores or less but the thing is that old server gear is actually pretty cheap right now to the point where having a home lab with 16 or 32 cores isn't all that expensive. Unless you need to have those additional services broken out into their own servers for performance consideration then scale is a great option for a platform that can do it all.
I recently built a new dream server (home), and decided to goto scale, but my old nas is still on core and theres many things that i wish were in scale. Such as having to manually set the ARC to use more than 50% ram in scale, or even being able to assign seperate IPs to app/jails, so that my password manager still works. It was the Apps are what drew me into scale, but after months of trail and error with a lot of things id run into bugs and limitations (such as nginx only running on SSD's?!) so the only 2 apps i added that arent in core, were tailscale and pihole, although adguard works just as well in core, and there are script availble for tailscale in core too, but a out of box solution would be best. The main reason i didnt go back to core though was hardware encoding for plex.
In the end, FreeBSD will continue to lose developers, sponsors and users… Any BSD can be finicky/tricky to get installed on some less common machines so Linux has the compatibility advantage.
I remember when Windows and Visual Basic ruled the popularity contests. It’s safe to say there won’t be “the one” here either. There is a very long list of advantages and Linux wins some and loses others. A better path is for strong competition to continue including entirely new possibilities. In the end I bet on something entirely new. The stagnation is getting very boring.
Well I run Core, but I want to run NextCloud on it and I have been unsuccessful, Tom, you haven't done a video on how to install nextcloud on Core 13, you did it on V12, This is forcing my hand to move to SCALE.
You say you run truenas core over scale because the apps aren’t mature on scale but then say your customers don’t care about apps just file sharing…. That doesn’t make sense
I liked listening to this the challenges you experience are not dissimilar to the real world of hype scaling in cloud solution design for AWS or Azure. 😂
I‘m all in Scale. But why this K8s/Helm crap? Just plain Docker out of the box would be nice (I see no benefit in spinning up a VM solely for a robust Docker experience).
thinking about building a nas, truenas scale looks amazing but only problem is that u cant expand a single vdev, apparently this feature is in the works for zfs but im not into stuff enough to know when to expect it to be finished. can anyone explain?
I would not make my decision based on that feature appearing soon and being tested and ready, etc.. So the basic quick and dirty version is this: say you have zfs equivalent of raid6 (zraid2) with 5 drives, so the equivalent space of 2 drives are used for parity. So you use it, and then run out of space on that vdev for your pool (pools made of vdevs, vdevs made of devices). You can't just throw another single disk into it to now make it a zraid2 with 6 drives. You would have to add another set of 5 drives to make another vdev to add to the pool. Or do something risky and replace one drive at a time and resilver, but that is just tempting fate. The easiest way (but least space efficient) is to use mirrors (mirrors can be 2 or more ie 3 way mirror each drive has the exact same data on it). That way you only need to add another mirror at a time to increase your pool which is usually far less disks than a zraid (minimum 3).
As the other person that responded explained, you can simply add vdevs. The other expansion options include replacing disks with bigger disks, and RaidZ(1/2/3) do, as of at least recently in a release, support expansion. I have tested this on test systems with zero issues. I have not had a need to run this on any of my larger arrays that I have my actual data on, but it looks promising. RAID is not a backup, same as said on the shirt, so in the worst case, you rebuild the pool and restore from backup. Never ideal, but it appears to be working well with initial personal testing. Time will tell, and bug fixes will continue to roll out as all software does.
thanks for the responses, as someone with a very limitted budget i think i will simply wait until i can afford to build a single vdev that is large enough to last me a long time. as for the raidz expansion feature, i will await its arrival. hopefully not too long :D
I'm fine with core (and it's still arguably better for pure NAS performance which is what I use it for) but see the writing on the wall and have deployed a test vm on proxmox to start getting used to scale. I'll keep using core as long as it exists though, kind of sad to see how things are going with FreeBSD as a long time user.
im a home user i use mine as a home server for my video editing storage, and plex server, and a temp backup server Using SMB I'm using TrueNAS Core at mo I'm thinking of changing to Scale but not sure that is 100% right for me at mo but upgrading my motherboard and ram is my best bet at mo as I'm using Gigabite S-Series Ultra Durable 2 GA-EP35-DS3 with Intel 2 Duo CPU E8500 @ 3.16GHz (2 cores) and 8GB ECC maxed-out lol
I have been out of the game for a wile and I was doing good until I tried to install crafty-4 so I could do a Minecraft server for the kiddos and I can not for the life of me figure out how to get that damn web ui password lol. In truenas
Because they want to increase their revenue, not decrease it! While ix adds significant value it’s not rocket science. Offend their Core customers and they will find themselves with a new competitor. Home lab users are not a revenue generator.
Thanks for your videos, I always come back to the channel for more info!!! I'm using a Scale, which works great! I'm listing the components of my build-up below as a guide for anyone new to this: CPU: AMD Ryzen 7 5800X processor MOBO: ASRock B550 Phantom Gaming-ITX/ax motherboard RAM: Kingston KSM26ED8/16ME DDR4-16 GB memory module clocked at 2666 MHz, featuring unbuffered ECC with timings of CL19-1.2 V HD Controller: LSI Logic Controller Card bearing the model LSI00301 SAS 9207-8i, providing connectivity for up to 8 ports SATA Cables: CableCreation's Mini SAS to 4 SATA Cable, equipped with a 36-pin SFF 8087 connector on the host/controller end and 7-pin SATA connectors on the target/backplane end, compatible with RAID controllers such as PERC H310, with a length of 1 meter or 3.3 feet HDD: Toshiba N300 6TB NAS 3.5-Inch Internal Hard Drive, employing CMR technology, SATA interface with 6 GB/s transfer rate, spinning at 7200 RPM, and boasting a 256 MB cache under the model HDWG460XZSTA Cache: Intel's Optane Memory M.2 2280 module, offering a capacity of 32GB, utilizing PCIe NVMe 3.0 x2 interface standards, with the model MEMPEK1W032GAXT.
