Lightweight Reinforcement Learning for Energy Efficient Communications in Wireless Sensor Networks

• Published in: IEEE access Vol. 7; pp. 29355 - 29364
• Main Authors: Savaglio, Claudio, Pace, Pasquale, Aloi, Gianluca, Liotta, Antonio, Fortino, Giancarlo
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adaptive systems; Artificial intelligence; Computational modeling; Energy consumption; energy-efficient network; Informatics; Machine learning; Media Access Protocol; medium access control; Process parameters; Reinforcement learning; Wireless communications; Wireless networks; Wireless sensor network; Wireless sensor networks
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2019.2902371

High-density communications in wireless sensor networks (WSNs) demand for new approaches to meet stringent energy and spectrum requirements. We turn to reinforcement learning, a prominent method in artificial intelligence, to design an energy-preserving MAC protocol, with the aim to extend the network lifetime. Our QL-MAC protocol is derived from Q-learning, which iteratively tweaks the MAC parameters through a trial-and-error process to converge to a low energy state. This has a dual benefit of 1) solving this minimization problem without the need of predetermining the system model and 2) providing a self-adaptive protocol to topological and other external changes. QL-MAC self-adjusts the WSN node duty-cycle, reducing energy consumption without detrimental effects on the other network parameters. This is achieved by adjusting the radio sleeping and active periods based on traffic predictions and transmission state of neighboring nodes. Our findings are corroborated by an extensive set of experiments carried out on off-the-shelf devices, alongside large-scale simulations.