Technical Report MSC-2015-06

Title: RAIDP: ReplicAtion with Intra-Disk Parity
Authors: Eitan Rosenfeld
Supervisors: Dan Tsafrir, Michael Factor
PDFCurrently accessibly only within the Technion network
Abstract: Distributed storage systems often triplicate data to reduce the risk of permanent data loss, thereby providing failure tolerance for at least two simultaneous disk failures at the price of 2/3 of the capacity. To reduce this price, some systems utilize erasure coding. But this optimization is usually only applied to cold data, because erasure coding might hinder performance for warm data.

We propose RAIDP—a new point in the distributed storage design space between replication and erasure coding. RAIDP largely retains the benefits of replication, trading off some performance for substantially reduced storage and networking overheads. RAIDP maintains only two replicas, rather than three or more. It increases durability by utilizing small disk “add-ons” for storing intra-disk erasure codes that are local to the server and fail independently from the disk. By carefully laying out the data, the add-ons allow RAIDP to recover from simultaneous disk failures. (Add-ons can be stacked to withstand an arbitrary number of failures.) We implement RAIDP in HDFS, which triplicates by default. We show that RAIDP performs within 0%–24% of the baseline while halving the storage and networking overheads and providing similar durability.

CopyrightThe above paper is copyright by the Technion, Author(s), or others. Please contact the author(s) for more information

Remark: Any link to this technical report should be to this page (, rather than to the URL of the PDF files directly. The latter URLs may change without notice.

To the list of the MSC technical reports of 2015
To the main CS technical reports page

Computer science department, Technion