WukaStore: Scalable, Configurable and Reliable Data Storage on Hybrid Volunteered Cloud and Desktop Systems

• Published in: IEEE transactions on big data Vol. 8; no. 1; pp. 85 - 98
• Main Authors: Tang, Bing, Fedak, Gilles
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01-02-2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Big Data; Cloud computing; Cloud storage; data availability; data durability; Data management; Data storage; desktop grid; Durability; Encoding; Fault tolerance; Maintenance engineering; Middleware; Peer-to-peer computing; peer-to-peer storage system; Performance evaluation; User requirements; User satisfaction; volunteered cloud
• ISSN: 2332-7790; 2372-2096
• DOI: 10.1109/TBDATA.2017.2758791

In this paper, we propose a hybrid storage framework WukaStore, which offers a scalable, configurable and reliable big data storage service by integrating stable storage (such as Cloud storage or durable storage utilities) and volatile storage (such as idle storage harnessed from desktop PCs over the Internet). By configuring different storage strategies, WukaStore delivers cost-effective storage service to satisfy users' requirements. We present trace-driven simulations to study the impact of different strategies and how to ensure high availability and durability in WukaStore. We present the prototype implementation of WukaStore based on BitDew, an open source middleware for Big Data management. We conduct performance evaluations to validate the effectiveness of WukaStore, through the deployment of WukaStore in the French Grid'5000 experimental platform. In particular, we also present a case study where WukaStore is deployed in a hybrid environment taking advantage of storage provided by Amazon S3, Dropbox and a local Desktop Grid at the same time. Our evaluation results show that WukaStore provides users with an alternative method to store their big data on hybrid volunteered cloud and desktop systems and to decrease the cost of data storage, while providing great data access performance, scalability as well as reliability.