Adaptive Filesystem Compression for General Purpose Systems
2018-06
Loading...
View/Download File
Persistent link to this item
Statistics
View StatisticsJournal Title
Journal ISSN
Volume Title
Title
Adaptive Filesystem Compression for General Purpose Systems
Authors
Published Date
2018-06
Publisher
Type
Thesis or Dissertation
Abstract
File systems have utilized compression to increase the amount of storage available to the operating system. However, compression comes with the risk of increased latency and wasted computation when compressing incompressible data. We used Adaptive Compression (AC) to increase disk write speeds using compression, while not falling victim to incompressible data. Our AC system decides which algorithm to use from a list of available compression algorithms based on a model that queries CPU usage, disk bandwidth, current write queue length and estimates compressibility using “bytecounting,” an entropy-like statistic. Our model learns over time by building a table of estimated outcomes for each state and averaging previous outcomes for that state. We integrated our AC system into ZFSs write pipeline to reduce overall write times. In benchmarks with several standard datasets, our solution increased average write speeds by up to 48% compared to the baseline. For worst case (incompressible) data, our system decreased write speeds by 5%, due to system overhead. Compared to a previous ZFS AC system, our system does not perform statistically worse on any dataset and writes datasets with many small compressible files up to 49% faster.
Keywords
Description
University of Minnesota M.S. thesis.June 2018. Major: Computer Science. Advisor: Peter Peterson. 1 computer file (PDF); vi, 55 pages.
Related to
Replaces
License
Series/Report Number
Funding information
Isbn identifier
Doi identifier
Previously Published Citation
Suggested citation
Beaulieu, Jonathan. (2018). Adaptive Filesystem Compression for General Purpose Systems. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/200153.
Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.