Add option to prefer a WAL write if the write is below a size threshold,
even if we could avoid it. This lets you trade some write-amp (by
journaling data to rocksdb) for latency in cases where the WAL device is
much faster than the main device.
This affects:
- writes to new extents locations below min_alloc_size
- writes to unallocated space below min_alloc_size
- "big" writes above min_alloc_size that are below the prefer_wal_size
threshold.
Note that it's applied to individual blobs, not the entirety of the write,
so if your have a larger write torn into two pieces/blobs that are below
the threshold then they will both go through the wal.
Set different defaults for HDD and SSD, since this makes more sense for HDD
where seeks are expensive.