Next generation erasure coding methods for cloud storage
Responsable du projet Veronica Estrada Galinanes
Partenaire Ethan Miller
Résumé This project proposes the study of next generation erasure coding methods to preserve data in cloud storage systems efficiently. Cloud computing is built with less expensive hardware. Software and hardware failures may cause data loss. The storage of redundant data is essential to preserve digital data. Replication is a de-facto standard to create redundancy, e.g. triplication keeps three replicas in distinct places. Google, Facebook and many other storage systems use triplication. Currently, research and industry efforts are focused on reducing the storage overhead. As a result, erasure coding like Reed-Solomon codes are a popular alternative. None of both approaches can practically tolerate a large amount of simultaneous failures as they consume plenty of resources. Significant trade-offs among the storage overhead, network bandwidth, disk I/O constitute a limitation on a system’s fault-tolerance. As a result, the failure tolerance is low. For instance, triplication tolerates 2 failures, and Reed-Solomon in a common setting used by Facebook tolerates 4 failures. The main question that this project tries to address is: How can we improve the reliability of storage systems while using few resources? Increasing the fault tolerance brings multiple benefits. Notably, it helps for long-term retention of data. In addition, it may facilitate datacenter maintenance and is a deterrent against malicious attacks such tampering or data censorship. The hypothesis is that the creation of interdependencies between old and new content inserted in a system can be used to disperse redundant data across a large amount of devices efficiently.
Mots-clés reliability, archival storage, long-term retention data, erasure codes, cloud storage, data entanglement, fault tolerance, codes d'enchevêtrement, tolérance aux pannes, archivage d'information, fiabilité
Type de projet Recherche appliquée
Domaine de recherche Computer Science
Source de financement SNF Doc Mobility
Etat Terminé
Début de projet 1-2-2016
Fin du projet 31-7-2016
Budget alloué $36100
Contact Veronica Estrada Galinanes