In enterprise network-attached storage and modern private cloud virtualization, memory allocation dictates the boundary between deterministic microsecond I/O performance and catastrophic kernel instability. For over a decade, system administrators running TrueNAS CORE relied on FreeBSD’s native OpenZFS integration, where the Adaptive Replacement Cache (ARC) dynamically claimed up to 90 percent of host physical RAM and yielded memory back to user space applications with rapid agility. However, as organizations migrate production storage fleets to TrueNAS SCALE—built on Debian Linux and the Linux OpenZFS kernel module—administrators routinely encounter an unexpected operational shock. By default, OpenZFS on Linux hard-caps the maximum ARC size (zfs_arc_max) to exactly 50 percent of total installed physical RAM. A high-density storage server equipped with 128 gigabytes of error-correcting code (ECC) memory suddenly displays 64 gigabytes sitting apparently idle in basic monitoring dashboards, while random access reads to active databases or virtual machines suffer severe cache miss penalties. Conversely, administrators deploying containerized K3s applications or KVM virtual machines alongside ZFS storage frequently witness out-of-memory kernel panics when the Linux page cache and ZFS ARC memory reclaimers collide during high memory pressure. Here is Jay Tech’s definitive systems architecture guide to mastering TrueNAS SCALE ZFS ARC memory sizing, Linux kernel memory accounting, kernel module tuning, and enterprise virtualization budgeting.

Why Does TrueNAS SCALE Cap ZFS ARC at 50% of RAM by Default?

To understand why this conservative allocation exists, one must analyze the divergent architectural philosophies between the FreeBSD and Linux memory managers. In TrueNAS CORE (FreeBSD), OpenZFS is tightly coupled with the kernel’s virtual memory subsystem. The FreeBSD page daemon communicates directly with the ZFS ARC reclaimer. If an aggressive user process demands 32 gigabytes of heap memory, the FreeBSD kernel immediately reclaims clean ARC pages in a synchronous, zero-copy transaction with minimal latency. Consequently, FreeBSD can safely dedicate 85 to 90 percent of host physical memory to ZFS ARC without risking system instability.

In contrast, TrueNAS SCALE operates on Debian Linux, where OpenZFS runs as an out-of-tree kernel module. Linux manages its own native page cache and treats OpenZFS ARC memory as kernel slab allocations tracked through the Spl (Solaris Portability Layer). Linux kernel memory management relies on registered shrinkers (mm/vmscan.c) to request cache drops from external modules during memory starvation. When a guest KVM virtual machine or Docker container suddenly requests a massive memory allocation, the Linux kernel shrinker sends an asynchronous signal to the ZFS ARC eviction thread (arc_evict()). Evicting gigabytes of compressed ZFS metadata and data buffers takes measurable milliseconds. If memory demand outpaces ARC eviction velocity, the Linux kernel Out-Of-Memory (OOM) killer triggers instantly, aggressively terminating critical host processes or guest virtual machines before the ARC has released its buffers. To prevent this destructive behavior in hybrid storage-compute environments, the OpenZFS on Linux development team hardcoded zfs_arc_max to default to half of system memory.

ZFS ARC Architecture: Linux Slab vs FreeBSD Unified Virtual Memory

The TrueNAS SCALE Memory Budgeting Calculus

Arbitrarily setting zfs_arc_max to 90 percent on a TrueNAS SCALE server running virtual machines or container workloads is a guaranteed path to production downtime. System engineers must calculate memory distribution using a deterministic physical budgeting formula:

Host_Physical_RAM = RAM_Base_OS + RAM_VM_Pinned + RAM_Apps_K3s + RAM_Metadata + RAM_Data_ARC

Let us break down each memory vector for an enterprise 128GB TrueNAS SCALE storage head:

• Base Operating System (RAM_Base_OS): Allocate a static 6GB to 8GB for the Debian base system, Nginx web interface, OpenSSH, middleware daemons, and Python control plane. High-speed 10GbE, 25GbE, or 100GbE networking interfaces utilizing jumbo frames require substantial kernel ring buffers (sk_buff allocations) that live outside ZFS ARC.
• Pinned Virtual Machines (RAM_VM_Pinned): If your TrueNAS SCALE node hosts KVM virtual machines, guest memory is allocated via anonymous hugepages or standard guest RAM blocks. Because Linux cannot dynamically reclaim guest VM memory without balloon drivers (which introduce latency), every megabyte allocated to a VM must be permanently subtracted from your ARC ceiling. A virtualized Windows Server domain controller requiring 16GB must be strictly reserved.
• Containerized Apps (RAM_Apps_K3s): Docker and Kubernetes applications (Nextcloud, Plex, PostgreSQL, Redis) consume dynamic heap memory. Allocate an un-tunable margin of 8GB to 16GB depending on running container density.
• ZFS Metadata Footprint (RAM_Metadata): The most critical component of ARC is metadata caching (directory structures, inode tables, indirect block pointers, and deduplication tables if active). A pool containing 100 terabytes of small files requires between 8GB and 16GB of metadata caching to prevent disk-bound tree traversals. Metadata hits should always approach 99 percent.
• ZFS Data ARC (RAM_Data_ARC): The remaining RAM buffer acts as the Most Recently Used (MRU) and Most Frequently Used (MFU) data cache. On a dedicated storage head with 128GB of RAM and zero guest VMs, setting zfs_arc_max to 96GB (75% of total RAM) leaves a healthy 32GB safety margin for OS buffers, dirty write syncs, and network throughput bursts.

Step-by-Step Implementation: Tuning ZFS ARC Parameters in TrueNAS SCALE

Executing ARC optimization requires inspecting live kernel parameters, testing dynamic changes at runtime, and baking persistent configurations into the TrueNAS SCALE boot pipeline. Follow this engineered deployment sequence:

Step 1: Inspect Current ARC Sizing and Cache Hit Ratios

Open a root SSH terminal on your TrueNAS SCALE host and evaluate active ARC consumption using the native Python diagnostic utility:

# Query live ARC operational statistics
arc_summary

# Inspect current byte-level limits directly from sysfs
cat /sys/module/zfs/parameters/zfs_arc_max
cat /sys/module/zfs/parameters/zfs_arc_min
cat /sys/module/zfs/parameters/zfs_arc_sys_free

Pay close attention to the Cache Hit Rate section in the arc_summary output. If your total ARC hit rate sits below 85 percent and the arc_size equals arc_max, your working dataset exceeds current cache limits and storage throughput is being throttled by physical disk latency.

Step 2: Dynamically Apply New ARC Maximums Without Rebooting

Before making changes persistent, test system stability by calculating your desired byte limit and writing directly to the kernel module sysfs interface. For example, to allocate 96 gigabytes (96 * 1024 * 1024 * 1024 = 103,079,215,104 bytes) on a 128GB system:

# Dynamically expand zfs_arc_max to 96GB
echo 103079215104 > /sys/module/zfs/parameters/zfs_arc_max

# Verify the updated limit is actively enforced
cat /sys/module/zfs/parameters/zfs_arc_max

Monitor system memory pressure over the next two hours using htop and vmstat 1 30. Ensure that free host memory remains above your defined safety margin during peak read and write operations.

Step 3: Establish Persistent Module Tuning Across System Updates

TrueNAS SCALE utilizes an immutable, appliance-style root filesystem that updates via complete system image deployments. Modifying /etc/modprobe.d/zfs.conf manually will be overwritten during major OS updates. The canonical, production-grade method to persist ARC limits is through the TrueNAS SCALE Web UI:

1. Log in to the TrueNAS SCALE Web Interface with administrative credentials.
2. Navigate to System Settings > Advanced > Init/Shutdown Scripts.
3. Click Add to create a new startup hook.
4. Set Description to Tune ZFS ARC Max to 96GB.
5. Set Type to Command.
6. Enter the following shell command into the execution field:
   echo 103079215104 > /sys/module/zfs/parameters/zfs_arc_max
7. Set When to Post Init (ensuring the OpenZFS kernel module is fully loaded before execution).
8. Set Enabled to checked and click Save.

This configuration executes automatically on every host boot cycle, guaranteeing that your storage cluster re-establishes optimal memory allocation without manual intervention.