Long-term availability prediction for groups of volunteer resources

• Published in: Journal of parallel and distributed computing Vol. 72; no. 2; pp. 281 - 296
• Main Authors: Lázaro, Daniel, Kondo, Derrick, Marquès, Joan Manuel
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01-02-2012
• Subjects: Availability; Availability prediction; Collection; Computation; Internet; Intranets; Platforms; Reliability; Service deployment; Trace analysis; Volatility; Volunteer computing; Workflow; Trace analysis; Volunteer computing; Service deployment; Availability prediction; Reliability
• ISSN: 0743-7315; 1096-0848
• DOI: 10.1016/j.jpdc.2011.10.007

Volunteer computing uses the free resources in Internet and Intranet environments for large-scale computation and storage. Currently, 70 applications use over 12 PetaFLOPS of computing power from such platforms. However, these platforms are currently limited to embarrassingly parallel applications. In an effort to broaden the set of applications that can leverage volunteer computing, we focus on the problem of predicting if a group of resources will be continuously available for a relatively long time period. Ensuring the collective availability of volunteer resources is challenging due to their inherent volatility and autonomy. Collective availability is important for enabling parallel applications and workflows on volunteer computing platforms. We evaluate our predictive methods using real availability traces gathered from hundreds of thousands of hosts from the SETI@home volunteer computing project. We show our prediction methods can guarantee reliably the availability of collections of volunteer resources. We show that this is particularly useful for service deployments over volunteer computing environments.
► Providing collective availability for long-lived services on distributed resources. ► We analyze temporal structure of host CPU availability in real SETI@Home traces. ► Prediction of long-term availability by capturing weekly patterns. ► Attain collective availability in service deployment using availability prediction. ► Achieve availability with lower communication costs than short-term prediction.