Configuring ZFS Cache for High Speed IO

1. How can I make my ZFS faster?
2. How much ZFS cache do I need?
3. How does ZFS cache work?
4. How do you increase the size of a ZFS pool?
5. Why is ZFS bad?
6. How many drives do I need for ZFS?
7. Why does ZFS use so much RAM?
8. What is Zpool cache?
9. What is ZFS log?
10. Should I use ZFS compression?
11. What is ZFS sync?
12. What is ARC ZFS?

How can I make my ZFS faster?

How to Improve ZFS performance

1. A Good 64-bit CPU + Lots of Memory.
2. Tweaking the Boot Loader.
3. Use Disks with the Same Specifications.
4. Use a Powerful Power Supply.
5. Enable Compression.
6. Identify the Bottleneck.
7. Keep Your ZFS up to Date.
8. Understand How the ZFS Caching Works.

How much ZFS cache do I need?

Sizing it at ~10s * maximum write speed is all you need. If you have a 10gbit network, 16G of mirrored SLOG would be more than enough. And at least it doesn't negatively impact performance... just probably won't get used. L2ARC on the other hand consumes memory that would be used for ARC.

How does ZFS cache work?

ZFS allows for tiered caching of data through the use of memory. ... The ARC and its extension, the L2ARC are straight up read caches. They exist to speed up reads in the server so that the system doesn't need to go searching through slow spinning disks every time it needs to find data.

How do you increase the size of a ZFS pool?

So the bottom line is ,you can't reduce the zpool size but it can be increased on fly by adding new LUNS or DISKS to the zpool.
...
RAIDZ Zpool:

1. Check the existing RAIDZ pool status. ...
2. Add a new LUN/Disk to the oracle-RZ to extend the zpool size. ...
3. Check the zpool status.

Why is ZFS bad?

If you're looking for the absolute-fastest uncached and uncompressed performance, ZFS is unlikely to satisfy. While not much slower than alternatives like ext4 or xfs, it is generally at least a bit slower—again, if you design your benchmark workload to exclude the benefits of caching or compression.

How many drives do I need for ZFS?

The current rule of thumb when making a ZFS raid is: MIRROR (raid1) used with two(2) to four(4) disks or more. RAIDZ-1 (raid5) used with five(5) disks or more.

Why does ZFS use so much RAM?

Apparently it's the deduplication feature that requires lots of RAM See Oracle documentation. ZFS only support block-level deduplication, and if dedup is turned on, you will need approximately 320 Bytes per block per core. This makes the resulting RAM consumption a bit tricky.

What is Zpool cache?

cache file. Whenever a pool is imported on the system it will be added to the /etc/zfs/zpool. ... This file stores pool configuration information, such as the device names and pool state. If this file exists when running the zpool import command then it will be used to determine the list of pools available for import.

What is ZFS log?

ZIL stands for ZFS Intent Log. The purpose of the ZIL in ZFS is to log synchronous operations to disk before it is written to your array. That synchronous part essentially is how you can be sure that an operation is completed and the write is safe on persistent storage instead of cached in volatile memory.

Should I use ZFS compression?

An absolutely killer feature of ZFS is the ability to add compression with little hassle. As we turn into 2018, there is an obvious new year's resolution: use ZFS compression. Combined with sparse volumes (ZFS thin provisioning) this is a must-do option to get more performance and better disk space utilization.

What is ZFS sync?

ZFS handles synchronous writes in a special way—it temporarily but immediately stores them in the ZIL, before writing them out permanently along with asynchronous writes later.

What is ARC ZFS?

Background in a nutshell. The "ARC" is the ZFS main memory cache (in DRAM), which can be accessed with sub microsecond latency. ... The L2ARC sits in-between, extending the main memory cache using fast storage devices, such as flash memory based SSDs (solid state disks).