A Performance Evaluation of Erasure Coding Libraries for Cloud-Based Data Stores
| Résumé |
Erasure codes have been widely used over the last decade to
implement reliable data stores. They offer interesting trade-offs
between efficiency, reliability, and storage overhead. Indeed, a
distributed data store holding encoded data blocks can tolerate the
failure of multiple nodes while requiring only a fraction of the
space necessary for plain replication, albeit at an increased
encoding and decoding cost. There exists nowadays a number of
libraries implementing several variations of erasure codes, which
notably differ in terms of complexity and implementation-specific
optimizations. Seven years ago, Plank et al. [14] have conducted a comprehensive performance evaluation of open-source erasure coding libraries available at the time to compare their raw performance and measure the impact of different parameter configurations. In the present experimental study, we take a fresh perspective at the state of the art of erasure coding libraries. Not only do we cover a wider set of libraries running on modern hardware, but we also consider their efficiency when used in realistic settings for cloud-based storage, namely when deployed across several nodes in a data centre. Our measurements therefore account for the end-to-end costs of data accesses over several distributed nodes, including the encoding and decoding costs, and shed light on the performance one can expect from the various libraries when deployed in a real system. Our results reveal important differences in the efficiency of the different libraries, notably due to the type of coding algorithm and the use of hardware-specific optimizations. |
| Mots-clés |
|
| Citation | D. Burihabwa, et al., "A Performance Evaluation of Erasure Coding Libraries for Cloud-Based Data Stores," in DAIS'16 (Distributed Applications and Interoperable Systems), Heraklion, Crete, Greece, 2016, p. 160-173. |
| Type | Actes de congrès (Anglais) |
| Nom de la conférence | DAIS'16 (Distributed Applications and Interoperable Systems) (Heraklion, Crete, Greece) |
| Date de la conférence | 5-6-2016 |
| Editeur commercial | Springer |
| Pages | 160-173 |
| URL | http://2016.discotec.org/index.php?MG=20&Mid=6&sub1=1 |
| Liée au projet | SafeCloud: Secure and Resilient Cloud Architecture |