Browsing by Subject "SMR"

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    The Applications of Workload Characterization in The World of Massive Data Storage
    (2015-08) He, Weiping
    The digital world is expanding exponentially because of the growth of various applications in domains including scientific fields, enterprise environment and internet services. Importantly, these applications have drastically different storage requirements including parallel I/O performance and storage capacity. Various technologies have been developed in order to better satisfy different storage requirements. I/O middleware software, parallel file systems and storage arrays are developed to improve I/O performance by increasing I/O parallelism at different levels. New storage media and data recording technologies such as shingled magnetic recording (SMR) are also developed to increase the storage capacity. This work focuses on improving existing technologies and designing new schemes based on I/O workload characterizations in corresponding storage environments. The contributions of this work can be summarized into four pieces, two on improving parallel I/O performance and two on increasing storage capacity. First, we design a comprehensive parallel I/O workload characterization and generation framework (called PIONEER) which can be used to synthesize a particular parallel I/O workload with desired I/O characteristics or precisely emulate a High Performance Computing (HPC) application of interest. Second, we propose a non-intrusive I/O middleware (called IO-Engine) to automatically improve a given parallel I/O workload in Lustre which is a widely used HPC or parallel I/O system. IO-Engine can explore the correlations between different software layers in the deep I/O path, as well as workload patterns at runtime to transparently transform the workload patterns and tune related I/O parameters in the system. Third, we design several novel static address mapping schemes for shingled write disks (SWDs) to minimize the write amplification overhead in hard drives adopting SMR technology. Fourth, we propose a track-level shingled translation layer (T-STL) for SWDs with hybrid update strategy (in-place update plus out-of-place update). T-STL uses dynamic address mapping scheme and performs garbage collection operations by migrating selected disk tracks. This scheme can provider larger storage capacity and better overall performance with the same effective storage percentages when compared to the static address mapping schemes.
  • Thumbnail Image
    Item
    Storage System Designs with Emerging Storage Technologies
    (2020-01) Zhang, Baoquan
    In modern data centers, the volume of data has grown to an enormous size with an incredible speed due to the flourish of the Internet, mobile network, and the Internet of Things (IoT). Storage systems play a critical role under different scenarios, e.g., machine learning pipeline, interactive data analysis, data storage service, etc. Applications have to meet with very high requirements in the aspects of performance, capacity, and reliability. However, the I/O performance of storage systems suffer from the long-latency of storage devices. Besides, the data area density of storage devices has reached a bottleneck. Thus, it becomes difficult to increase the capacity of storage systems further. At last, silent data corruption happens more frequently than we expect. Traditional methods, e.g., replica, erasure code, etc., are not sufficient to ensure data reliability anymore. To address these challenges of performance, capacity, and data reliability in storage systems, storage vendors have proposed new storage technologies/devices. Firstly, Non-Volatile Memory (NVM) is a persistent memory that provides memory-speed data persistence and byte-addressable data accesses. Secondly, Shingled Magnetic Recording (SMR) is a promising layout to increase data area density further with existing magnetic recording technologies. At last, T10 Protection Information (T10-PI) drives are proposed against data corruption. However, current storage systems need to be optimized or even redesigned to leverage the advantages of these new storage technologies/devices. This thesis introduces four complemented research topics targeting designing new storage systems using NVM, SMR drives, PI-capable drives, and hybrid systems with NVM and storage devices, respectively. NVLSM is a key-value store using Log-Structured Merge Tree (LSM-Tree) on NVM systems. Idler is a mechanism to control the I/O workload to minimize the tail response time of an SMR drive. Idler artificially induces idle cleanings to avoid expensive blocking cleanings. DIX-aware RAID improves the data integrity in Linux software RAID using T10-PI against any data corruption during data transmission and persistence. Finally, PMDB is a new key-value store on systems with both NVM and traditional storage devices to achieve performance and capacity simultaneously.