Adaptive Filesystem Compression for General Purpose Systems

Thumbnail Image

Persistent link to this item

View Statistics

Journal Title

Journal ISSN

Volume Title


Adaptive Filesystem Compression for General Purpose Systems

Published Date




Thesis or Dissertation


File systems have utilized compression to increase the amount of storage available to the operating system. However, compression comes with the risk of increased latency and wasted computation when compressing incompressible data. We used Adaptive Compression (AC) to increase disk write speeds using compression, while not falling victim to incompressible data. Our AC system decides which algorithm to use from a list of available compression algorithms based on a model that queries CPU usage, disk bandwidth, current write queue length and estimates compressibility using “bytecounting,” an entropy-like statistic. Our model learns over time by building a table of estimated outcomes for each state and averaging previous outcomes for that state. We integrated our AC system into ZFSs write pipeline to reduce overall write times. In benchmarks with several standard datasets, our solution increased average write speeds by up to 48% compared to the baseline. For worst case (incompressible) data, our system decreased write speeds by 5%, due to system overhead. Compared to a previous ZFS AC system, our system does not perform statistically worse on any dataset and writes datasets with many small compressible files up to 49% faster.


University of Minnesota M.S. thesis.June 2018. Major: Computer Science. Advisor: Peter Peterson. 1 computer file (PDF); vi, 55 pages.

Related to



Series/Report Number

Funding information

Isbn identifier

Doi identifier

Previously Published Citation

Suggested citation

Beaulieu, Jonathan. (2018). Adaptive Filesystem Compression for General Purpose Systems. Retrieved from the University Digital Conservancy,

Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.