SQUASHFS 4.3 - A squashed read-only filesystem for Linux Copyright 2002-2014 Phillip Lougher <phillip@lougher.demon.co.uk> Released under the GPL licence (version 2 or later). Welcome to Squashfs 4.3. This is the first release in over 3 years, and there are substantial improvements to stability, new compression options and compressors, speed optimisations, and new options for Mksquashfs/Unsquashfs. This is a tools only release, support for Squashfs filesystems is in mainline (2.6.29 and later). Changes in Squashfs-tools 4.3 ----------------------------- 1. Stability improvements. Better checking of user input for out of range/invalid values. Better handling of corrupted Squashfs filesystems (Mksquashfs append mode, and Unsquashfs). Better handling of buffer overflow/underflow. 2. GZIP compressor now supports compression options, allowing different compression levels to be used. 3. Rewritten LZO compressor with compression options, allowing different LZO algorithms and different compression levels to be used. 4. New LZ4 compressor (note not yet in mainline kernel) 5. Better default memory usage for Mksquashfs. Mksquashfs by default now uses 25% of physical memory. 6. Duplicate checking in Mksquashfs further optimised. With certain "problem filesystems" greater than 2x performance improvement. Filesystems with a lot of duplicates should see at least 10-20% speed improvement. 7. The -stat option in Unsquashfs now displays the compression options used to generate the original filesystem. Previously -stat only displayed the compression algorithm used. 8. The file being compressed/uncompressed in Mksquashfs/Unsquashfs is now displayed if CTRL-\ (SIGQUIT from keyboard) typed. 9. The status of the internal queues/caches in Mksquashfs/Unsquashfs is now displayed if CTRL-\ (SIGQUIT from keyboard) is typed twice within one second. Normally only useful for "power users", but it can be used to discover if there's any bottlenecks affecting performance (the bottleneck will normally be the compressors/fragment compressors). 10. Miscellaneous new options for Mksquashfs/Unsquashfs to fine tune behaviour. 11. Fixes for CVE-2012-4024 and CVE-2012-4025. Compatiblity ------------ Mksquashfs 4.3 generates 4.0 filesystems. These filesystems are fully compatible/interchangable with filesystems generated by Mksquashfs 4.0 and are mountable on 2.6.29 and later kernels. Compressors ----------- New compression options and compressors are now supported. The new options and compressors are: 1. gzip -Xcompression-level <compression-level> <compression-level> should be 1 .. 9 (default 9) -Xwindow-size <window-size> <window-size> should be 8 .. 15 (default 15) -Xstrategy strategy1,strategy2,...,strategyN Compress using strategy1,strategy2,...,strategyN in turn and choose the best compression. Available strategies: default, filtered, huffman_only, run_length_encoded and fixed 2. lzo -Xalgorithm <algorithm> Where <algorithm> is one of: lzo1x_1 lzo1x_1_11 lzo1x_1_12 lzo1x_1_15 lzo1x_999 (default) -Xcompression-level <compression-level> <compression-level> should be 1 .. 9 (default 8) Only applies to lzo1x_999 algorithm 3. lz4 -Xhc Compress using LZ4 High Compression The compression specific options are, obviously, specific to the compressor in question, and you should read the compressor documentation and check their web sites to understand their behaviour. In general the defaults used by Mksquashfs for each compressor are optimised to give the best performance for each compressor, where what constitutes best depends on the compressor. For gzip/xz best means highest compression (trying multiple filters/strategies can improve compression, but this is extremely expensive computationally, and hence, not suitable for the defaults), for LZO/LZ4 best means a tradeoff between compression and (de)-compression overhead (LZO/LZ4 by definition are intended for weaker processors). New Mksquashfs options ---------------------- 1. -mem <size> Set the amount of memory used by Mksquashfs to <size> bytes. G/M and K post-fixes are supported. By default Mksquashfs uses 25% of the physical memory. Increasing this with the -mem option can increase performance (note it does not have any effect on compression). Reducing it can prevent thrashing if the system is busy and there is not 25% of physical memory free (again, note it does not have any effect on compression). 2. -exit-on-error By default Mksquashfs treats certain errors as benign, if these errors occur Mksquashfs prints the error on the console but continues. These errors are typically failure to read a file from the source filesystem. This is deliberate, in many cases users prefer Mksquashfs to flag the error but continue rather than abort what may be hours of compression. But there are times where failure to read any file is considered critical, and users (especially in the case of automated scripts where the errors output to the console may be missed) prefer Mksquashfs to exit. The new -exit-on-error option can be used in this scenario. This option makes Mksquashfs treat all benign errors as fatal. 3. -progress By default if -info is specified, the progress bar is disabled as it gets in the way. Occasionally you might want the progress bar enabled whilst -info is enabled. This option forces Mksquashfs to output the progress bar when -info is specified. 4. -Xhelp Display the usage text for the currently selected compressor. New Unsquashfs options ---------------------- 1. -u[ser-xattrs] Only write user xattrs. This forces Unsquashfs to ignore system xattrs. This is useful when Unsquashing a filesystem as a non-root user, and the filesystem contains system xattrs which are only writable by root. Major bugs fixed ---------------- 1. If Mksquashfs ran out of space in the destination filesystem, this would not cause Mksquashfs to immediately abort, and Mksquashfs would continue to process the source filesystem. Mksquashfs now immediately aborts on out of space in the destination filesystem. 2. Unsquashfs ignored the maximum number of open files limit, and if that was lower than the default limit for Linux, it would run out of file descriptors. Unsquashfs now limits the number of open files to the limit currently in force (e.g. specified by setrlimit). 3. If huge numbers of dynamic pseudo files were specified, Mksquashfs could exceed the maximum number of open files limit. This was because Mksquashfs created all the dynamic file processes up front before commencing source filesystem reading and compression. Mksquashfs now creates the dynamic file processes on demand whilst reading and compressing the source filesystem, thus limiting the number of dynamic pseudo file processes in existence at any one time. 4. When outputting Unsquashfs used to set the permissions of directories as it recursively descended. This in hindsight had an obvious oversight, if a directory had only read permission (or was otherwise restricted), then Unsquashfs would fail to write its contents when descending into it. Fixed by setting directory permissions as Unsquashfs recursively unwinds.