A fuzzy reinforcement learning approach to power control in wireless transmitters

• Published in: IEEE transactions on systems, man and cybernetics. Part B, Cybernetics Vol. 35; no. 4; pp. 768 - 778
• Main Authors: Vengerov, D., Bambos, N., Berenji, H.R.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01-08-2005
• Subjects: Actor-critic algorithms; Algorithms; Artificial Intelligence; Cell Phone; Channels; Communication system traffic control; Computer Simulation; Convergence; Dynamic programming; Fuzzy; Fuzzy control; Fuzzy Logic; fuzzy reinforcement learning; Fuzzy set theory; Interference; Learning; Models, Statistical; Pattern Recognition, Automated - methods; Power control; Reinforcement; Stochastic processes; Transmitters; Wireless networks; wireless power control
• ISSN: 1083-4419; 1941-0492
• DOI: 10.1109/TSMCB.2005.846001

We address the issue of power-controlled shared channel access in wireless networks supporting packetized data traffic. We formulate this problem using the dynamic programming framework and present a new distributed fuzzy reinforcement learning algorithm (ACFRL-2) capable of adequately solving a class of problems to which the power control problem belongs. Our experimental results show that the algorithm converges almost deterministically to a neighborhood of optimal parameter values, as opposed to a very noisy stochastic convergence of earlier algorithms. The main tradeoff facing a transmitter is to balance its current power level with future backlog in the presence of stochastically changing interference. Simulation experiments demonstrate that the ACFRL-2 algorithm achieves significant performance gains over the standard power control approach used in CDMA2000. Such a large improvement is explained by the fact that ACFRL-2 allows transmitters to learn implicit coordination policies, which back off under stressful channel conditions as opposed to engaging in escalating "power wars".