The CLyDE Project

Vision and Goal

CLyDE (2011-2015) was a project funded by the Danish Council for Independent ResearchıTechnology and Production Sciences. CLyDE was a collaboration between IT University of Copenhagen and  the SMIS project at INRIA. The goal of the project was to address the challenges caused by the advent of flash devices as replacement for hard disks as secondary storage of choice for data intensive systems. In the application, we define the following goal:

Our goal is to explore how database systems can collaborate with the software embedded in flash devices, through so-called cross-layer optimisations, in order to improve overall performance. Our approach is based on the insight that flash devices can be programmed to provide predictable, high performance as long as the database system respects a set of well-defined constraints

The premise of the CLyDE project was that the strict layering that has been so successful for designing database systems on top of magnetic disks is no longer applicable with SSDs based on existing observation that SSD performance varies a lot and in unpredictable ways. What was a slightly unconventional take on SSD performance characterisation has now become an accepted view in the storage industry.

People

Philippe Bonnet (ITU),Luc Bouganim (INRIA), Niv Dayan (ITU), Matias Bjørling (ITU).

Achievements

CLyDe’s main achievements are the following:

  • Matias Bjørling and Niv Dayan defended their PhDs in August 2015. Matias’ thesis is titled “Operating System Support for High-Performance Solid State Drives”. Niv’s thesis is titled “Modelling and Managing SSD Write Amplification”. After graduation, Matias joined CNEX Labs as Linux kernel developer and Niv joined the Data Systems Lab at Harvard University as a postdoc.
  • Matias made two significant contributions to the Linux kernel in the context of CLyDe. First, he contributed in 2013, together with Jens Axboe, to the Multi-queue Block Layer. Second, he proposed in 2015 to incorporate support for open channel SSDs in the Linux kernel. This framework for host-based SSD programming is being integrated upstream. Matias work on the Multi-queue Block Layer is described in a paper published at Systor in 2013. His work on Open Channel SSDs is being described in a new paper.
  • Matias Bjørling’s work on open channel SSDs is having a significant impact on the storage industry.  A range of vendors are implementing support for open channel SSDs. Support for key-value stores and database systems is under development. Most of these efforts are open source and available on github.
  • Niv Dayan developed a simulator to study the design space and experimenting with cross layer design involving  the software embedded on a flash device (the so-called Flash Translation Layer). The simulator is called EagleTree. It is available on GitHub. It was described in a demo paper, presented at VLDB in 2013.
  • The IT University of Copenhagen has joined the RapidIO industry consortium and contributes.RapidIO focuses on the convergence of networking and storage technologies. ITU contributes to the storage group together with IIT Madras and a range of companies.

Publications

Presentations

  • CLyDE Mid-Flight: What We have Learnt about the SSD-Based I/O Stack.
    Presented at EPFL, INRIA Montpellier, Spring 2014 (CLyDE-EPFL14)
  • Host-SSD Co-Design: Lessons Learnt and Perspectives
    Presented at 1st EMC Flash University Forum, Fall 2014 (EMC-flashuni14)
  • CLyDE Project: Lessons Learnt and Perspectives
    Presented to the board of the Technology and Production Sciences Committee of the Danish Independent Research Council, October 2014 (CLyDE-FTP)
Advertisements