ceClub: CORFU: A Shared-Log Design for Network-Attached Flash
- Dahlia Malkhi (Microsoft Research, Silicon Valley)
- Wednesday, 27.6.2012, 11:30
- Room 337-8 Taub Bld.
CORFU* is a novel storage cluster design that pools a farm of flash units and exposes it to clients as a single, global shared-log. CORFU utilizes flash to break the seeming tradeoff between consistency and performance, providing both strong consistency guarantees and high throughput. Our implementation is carried mostly by a client-side library, thus relieving the service from any IO bottlenecks; a CORFU cluster of 40 flash units can drive roughly 1 million 4-KByte IOPS.
A shared log is a powerful abstraction, which empowers high-performance, in-memory distributed applications that derive coordination and reliability from a common source. We demonstrate this through TOFU*, a distributed key-value store built over CORFU, which supports multi-key transactions.
Our client-heavy design relieves the need to place expensive machines and controllers between flash and clients. To demonstrate the feasibility of directly attached CORFU storage units to a network, we built a prototype FPGA-based flash unit which attaches directly to a 1Gbps NIC, and consumes an order of magnitude less energy than a PC-attached SSD.
* CORFU stands for Clusters of Raw Flash Units; it is also the name of a Greek resort island near Paxos.
* TOFU stands for Transactions over CORFU.
CORFU team: Mahesh Balakrishnan, John Davis, Dahlia Malkhi, Vijayan Prabhakaran, Ted Wobber
Thanks: Phil Bernstein, Ken Eguro, Jeremy Elson, Ed Nightingale, Colin Reid, Michael Wei, Ming Wu