The idea of using SSD disk to accelerate file system on rotational media was around for some time already.
However using SSD for caching may not require
complicated solutions like
bcache that we tried
Among all mainstream file systems only ext4 on rotational media can be accelerated using SSD disk safely and effectively. Its full performance potential can be unleashed by using a unique feature — external journal. Having said this it is worth noticing that xfs also can use external journal but only in theory because xfs can not be mounted if external journal device is lost neither it can be reconfigured to convert journal back to local or to change its size.
First step to optimise ext4 performance is to use
journal_async_commit mount option. This will automatically enable
journal_checksum feature as well.
journal_async_commit alone gives nice performance boost and it is
essential to use it with external journal.
Test: iozone (SYNC,O_DIRECT)
Iozone: Performance Test of File I/O Version $Revision: 3.397 $ Compiled for 64 bit mode. Build: linux-AMD64 Machine = Linux 3.7-trunk-amd64 #1 SMP Debian 3.7.3-1~experimental.1 x SYNC Mode. SYNC Mode. Include close in write timing Include fsync in write timing O_DIRECT feature enabled File size set to 4194304 KB Record Size 256 KB Command line used: iozone -M -o -c -e -t3 -T -I -s 4g -r 256k -i0 -i2 -i6 -i8 -i9 -i11 Output is in Kbytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 Kbytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. Throughput test with 3 threads Each thread writes a 4194304 Kbyte file in 256 Kbyte records
The table below shows results of tests where unbuffered writing was performed using direct access (O_DIRECT) and fsync() after every operation.
|file system||effective mount options||Test execution time (less is better)|
|ext4 (external journal_data)||rw,relatime,nodelalloc,journal_checksum,journal_async_commit||
|ext4 (external journal_data_ordered)||rw,relatime,journal_checksum,journal_async_commit||
* Don't trust NILFS2 benchmarks. This test was run on freshly-created file system. In real-world scenario the result probably will be at least 2 times worse due to background cleaning.
The first line (slowest) is a default ext4 behaviour (journal_checksum does not have measurable effect on performance).
xfs and btrfs are given for comparison.
journal_async_commit gives nice performance boost and puts ext4 ahead
The most interesting ext4 benchmarks follow cleanerless nilfs2 measurement.
The last two measurements are taken with ext4 external journal placed to 1906 MiB (3903488 sectors) partition on SSD disk.
journal_data mode first saves all data to journal then flushes it to
main rotational device. Basically all data is written twice, first to
journal and then to file system — that's why system time is 5 times greater
than in last test where
journal_data_ordered mode was used.
The very last result demonstrates fantastic performance boost — over 3 times faster than ext4 default (first test).
Full ext4 performance potential can be unleashed with the careful use of external journal. This is safe and effective method utilising no additional software and achieving performance comparable to less reliable SSD caching solutions.