CoRE: Cooperative End-to-End Traffic Redundancy Elimination for Reducing Cloud Bandwidth Cost

• Published in: IEEE transactions on parallel and distributed systems Vol. 28; no. 2; pp. 446 - 461
• Main Authors: Lei Yu, Haiying Shen, Sapra, Karan, Lin Ye, Zhipeng Cai
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-02-2017; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptive algorithms; Bandwidth; bandwidth cost; Bandwidths; Cloud computing; Efficiency; Elasticity; Electronic mail; end-to-end; network bandwidth; Receivers; Redundancy; Servers; Short term; Traffic redundancy elimination
• ISSN: 1045-9219; 1558-2183
• DOI: 10.1109/TPDS.2016.2578928

The pay-as-you-go service model impels cloud customers to reduce the usage cost of bandwidth. Traffic Redundancy Elimination (TRE) has been shown to be an effective solution for reducing bandwidth costs, and thus has recently captured significant attention in the cloud environment. By studying the TRE techniques in a trace driven approach, we found that both short-term (time span of seconds) and long-term (time span of hours or days) data redundancy can concurrently appear in the traffic, and solely using either sender-based TRE or receiver-based TRE cannot simultaneously capture both types of traffic redundancy. Also, the efficiency of existing receiver-based TRE solution is susceptible to the data changes compared to the historical data in the cache. In this paper, we propose a Cooperative end-to-end TRE solution (CoRE) that can detect and remove both short-term and long-term redundancy through a two-layer TRE design with cooperative operations between layers. An adaptive prediction algorithm is further proposed to improve TRE efficiency through dynamically adjusting the prediction window size based on the hit ratio of historical predictions. Besides, we enhance CoRE to adapt to different traffic redundancy characteristics of cloud applications to improve its operation cost. Extensive evaluation with several real traces show that CoRE is capable of effectively identifying both short-term and long-term redundancy with low additional cost while ensuring TRE efficiency from data changes.