UPDATE EDIT:
Man it is crazy to watch the dashboard and console at the time. Even with no HDD’s spinning, and as much RAM as I can give the Scale VM, services just slowly takes over the RAM, until the console shows kernel panic.

core was solid for so long with everything i threw at it.

it runs out of memory after services soaks up all the RAM, ZFS cache is choked down to 3gb out of 16.

  • xeon E3 1265LV2
  • Asus p8z77-v-deluxe
  • 32GB DDR3
  • hba passed through to truenas running a mirror pool

VM for truenas is running on the local proxmox SSD.

  • proxmox 9.1.1
  • TrueNAS scale 25.10.0.1 but i tried a 24 version also

once the install starts crashing, the VM will still crash after booting up without the HBA card

I’ve seen a few posts with other people having the out of memory issues (OOM) but almost every reply says it will be fixed in the next update, which is older than what we’ve got now.

it did run okay enough JUST long enough to make the mistake of updating the ZFS flags, so now i can’t roll back to core.

does scale have this issue because it’s virtualized? would it run better on bare metal?

anyone tried xigmaNAS? freeBSD based again at least.

Unraid looks okay, but paywall?

open media vault?

any advice or discussion is appreciated!

  • SapphironZA@sh.itjust.worksEnglish
    1·
    1 month ago

    My exprience with Trunas has been that ZFS does not like virtual disks. Especially when the Proxmox host also uses ZFS. Two layers of ZFS arc caching creates some memory issues. Setting the Host datasets to Metadata only may help.

    But the most reliable method would be doing hardware passthrough of physical disks to the VM. It gets you most of the bare metal reliability benefits without having to commit the entire hardware box to one OS.

    You may also want to disable memory ballooning in your VMs. It works well when you have lots of small VMs, but if you have a few large ones, it can cause issues if you overallocate Ram to VMs, beyond what the OS has available. I suspect it could also be interferring the zfs arc as well.

    Lastly, check that your VM is set to use the “host” Cpu type. Freenas would likely benefit from having access to more CPU functions.

    • Possibly linux@lemmy.zipEnglish
      1·
      1 month ago

      To add to that:

      Make sure you passthough a pcie device like a data controller. Just passing though disks doesn’t preserve metadata which will lead to errors and data loss

  • tvcvt@lemmy.mlEnglish
    0·
    1 month ago

    I’m not sure what it is, but Scale has never thrilled me. I’ve tested it a couple times and I just didn’t get along well with it. I’ve tested know Jim Salter (practicalzfs.com) has frequently recommended XigmaNAS as a strong (albeit less pretty) alternative to TrueNAS. I did some tests with that as well and it seemed perfectly fine. In the end I decided that when I migrate off of Core this winter, it’ll be to a bare metal FreeBSD system. I’m using it as an excuse to better learn that ecosystem and to bone up on ansible, which I’m using to define all of my settings.

    • Possibly linux@lemmy.zipEnglish
      0·
      1 month ago

      BSD isn’t really being maintained. It gets contributions once and a while but the vast majority of development happens on Linux.

      • tvcvt@lemmy.mlEnglish
        1·
        1 month ago

        That’s certainly true in terms of TrueNAS Core, but FreeBSD itself is quite active (15.0-RELEASE dropped this month), as are the others BSDs.

        • Possibly linux@lemmy.zipEnglish
          1·
          1 month ago

          All I know is that iX systems said that they might drop core since BSD isn’t nearly as well maintained as Linux.

          • tvcvt@lemmy.mlEnglish
            1·
            1 month ago

            If I remember correctly, that was largely in consideration of the large corpus of docker-packaged projects that could be used as a pre-built app ecosystem. That makes a lot of sense for anyone who really wants an appliance-like all-in-one system with minimal setup.

  • PeriodicallyPedantic@lemmy.caEnglish
    0·
    1 month ago

    Not practical advice, but since you’re throwing out alternatives, one that I’ve had my eye on is hexOS.
    It’s a trueNAS scale (fork?) by a bunch of ex-unraid devs with the goal of making trueNAS as easy as unraid.

    It still has a few more months of beta, though, so I haven’t tried it. Also, like unraid, it has a paywall.

    • Possibly linux@lemmy.zipEnglish
      0·
      1 month ago

      I’d avoid it as it is currently being developed by a startup. Startups are high risk businesses which means they could easily go under.