Login
Next generation erasure coding methods for cloud storage
Project responsable Veronica Estrada Galinanes
   
Project partner Ethan Miller
   
Abstract This project proposes the study of next generation erasure coding methods to preserve data in cloud storage systems efficiently. Cloud computing is built with less expensive hardware. Software and hardware failures may cause data loss. The storage of redundant data is essential to preserve digital data. Replication is a de-facto standard to create redundancy, e.g. triplication keeps three replicas in distinct places. Google, Facebook and many other storage systems use triplication. Currently, research and industry efforts are focused on reducing the storage overhead. As a result, erasure coding like Reed-Solomon codes are a popular alternative. None of both approaches can practically tolerate a large amount of simultaneous failures as they consume plenty of resources. Significant trade-offs among the storage overhead, network bandwidth, disk I/O constitute a limitation on a systemís fault-tolerance. As a result, the failure tolerance is low. For instance, triplication tolerates 2 failures, and Reed-Solomon in a common setting used by Facebook tolerates 4 failures. The main question that this project tries to address is: How can we improve the reliability of storage systems while using few resources? Increasing the fault tolerance brings multiple benefits. Notably, it helps for long-term retention of data. In addition, it may facilitate datacenter maintenance and is a deterrent against malicious attacks such tampering or data censorship. The hypothesis is that the creation of interdependencies between old and new content inserted in a system can be used to disperse redundant data across a large amount of devices efficiently.
   
Keywords reliability, archival storage, long-term retention data, erasure codes, cloud storage, data entanglement, fault tolerance, codes d'enchevêtrement, tolérance aux pannes, archivage d'information, fiabilité
   
Type of project Applied research project
Research area Computer Science
Method of financing SNF Doc Mobility
Status Completed
Start of project 1-2-2016
End of project 31-7-2016
Overall budget $36100
Contact Veronica Estrada Galinanes