Age of Information Minimization Through Reinforcement Learning in IoT Enabled Cognitive Radio Networks
Ragupathi C
Department of Electrical and Electronic Engineering, P. T. Lee Chengalvaraya Naicker College of Engineering and Technology, Veliyur, Tamil Nadu, India.
The rapid growth of Internet of Things (IoT) applications has led to an unprecedented demand for efficient wireless communication, particularly in scenarios where devices compete for limited spectrum resources. Cognitive Radio Networks (CRNs) provide a promising solution by enabling dynamic spectrum access, yet maintaining timely information updates remains a challenge. Age of Information (AoI), a metric that measures the freshness of status updates, has become a key performance indicator in such networks. This paper proposes a reinforcement learning (RL)-based scheduling framework designed to minimize AoI in IoT-enabled CRNs. The framework models the dynamic environment of spectrum availability and device activity, and uses an adaptive RL agent to optimize transmission scheduling decisions in real time. A mathematical model of the AoI minimization problem is elaborated and after that, the proposed learning algorithm is designed. Large-scale simulations are done in Python in Google Colab and the performance of these simulations is compared with four state-of-the-art scheduling strategies. Findings demonstrate that the suggested RL-based system is able to generate lower AoI with different network scales, channel characteristics, and traffic dynamics. The results indicate the promise of RL-based solutions to the creation of scalable and reactive IoT communication systems.
Keywords
Cognitive Radio Networks, IoT, Reinforcement Learning, Age of Information, Resource Allocation.
J. Sun, S. Zhang, C. Yang, and L. Huang, “Age of Information Minimization for Radio Frequency Energy-Harvesting Cognitive Radio Networks,” Entropy, vol. 24, no. 5, p. 596, Apr. 2022, doi: 10.3390/e24050596.
T. He, Y. Peng, Y. Liu, and H. Song, “AoI-Oriented Resource Allocation for NOMA-Based Wireless Powered Cognitive Radio Networks Based on Multi-Agent Deep Reinforcement Learning,” IEEE Access, vol. 12, pp. 69738–69752, 2024, doi: 10.1109/access.2024.3401624.
Chen, Y., Cai, Y., & Wang, M. (2023). Age of Information for Short-Packet Two-Way Relay System in Cognitive IoT Network. Journal of Electronics & Information Technology, 45(12), 3077–3084. https://doi.org/10.11999/JEIT221377.
Haldorai, B. L. R, S. Murugan, and M. Balakrishnan, “Nature Inspired Optimizers and Their Importance for AI: An Inclusive Analysis,” EAI/Springer Innovations in Communication and Computing, pp. 407–427, 2024, doi: 10.1007/978-3-031-53972-5_21.
K. V. V. Prasad and P. T. Rao, “Reinforcement learning-based cooperative sensing in cognitive radio networks for primary user detection,” International Journal of Information and Computer Security, vol. 19, no. 1/2, p. 34, 2022, doi: 10.1504/ijics.2022.126752.
H. Anandakumar and K. Umamaheswari, “A bio-inspired swarm intelligence technique for social aware cognitive radio handovers,” Computers Electrical Engineering, vol. 71, pp. 925–937, Oct. 2018, doi: 10.1016/j.compeleceng.2017.09.016.
B. F. Lo and I. F. Akyildiz, “Reinforcement learning for cooperative sensing gain in cognitive radio ad hoc networks,” Wireless Networks, vol. 19, no. 6, pp. 1237–1250, Dec. 2012, doi: 10.1007/s11276-012-0530-4.
I. F. Akyildiz, W.-Y. Lee, M. C. Vuran, and S. Mohanty, “NeXt generation/dynamic spectrum access/cognitive radio wireless networks: A survey,” Computer Networks, vol. 50, no. 13, pp. 2127–2159, Sep. 2006, doi: 10.1016/j.comnet.2006.05.001.
I. F. Akyildiz, B. F. Lo, and R. Balakrishnan, “Cooperative spectrum sensing in cognitive radio networks: A survey,” Physical Communication, vol. 4, no. 1, pp. 40–62, Mar. 2011, doi: 10.1016/j.phycom.2010.12.003.
M. Narender Pavan, “Deep Reinforcement Learning based channel allocation (DRLCA) in Cognitive Radio Networks,” Journal of Electrical Systems, vol. 20, no. 6s, pp. 914–926, Apr. 2024, doi: 10.52783/jes.2774.
Oksanen, J., Lundén, J., & Koivunen, V. (2021). Reinforcement Learning Based Sensing Policy Optimization for Energy Efficient Cognitive Radio Networks. Scientific World Journal, 2011, 209810. https://doi.org/10.1155/2011/209810.
Dastangoo, S., Fossa, C., & Gwon, Y. (2025). Multiagent Reinforcement Learning in Cooperative Anti-Jamming Games. Journal of Supercomputing, 71(1), 1–24. https://doi.org/10.1007/s11227-015-1465-0.
S. Ayub, N. Singh, Md. Z. Hussain, M. Ashraf, D. K. Singh, and A. Haldorai, “Hybrid approach to implement multi‐robotic navigation system using neural network, fuzzy logic, and bio‐inspired optimization methodologies,” Computational Intelligence, vol. 39, no. 4, pp. 592–606, Sep. 2022, doi: 10.1111/coin.12547.
G. Gokilakrishnan, V. M, R. Anushanjali, R. Subbiah, and A. H, “Analysis of the Requirement and Architecture for the Internet of Drones,” 2023 9th International Conference on Advanced Computing and Communication Systems (ICACCS), Mar. 2023, doi: 10.1109/icaccs57279.2023.10112779.
Yuan, X., & Zhang, L. (2025). Fairness in Cognitive Radio Networks: Models, Measurement Methods, Applications, and Future Research Directions. Journal of Network and Computer Applications, 73, 1–17. https://doi.org/10.1016/j.jnca.2016.08.004.
Zhang, Y., & Liu, Y. (2020). A Survey on Reinforcement Learning in Spectrum-Aware Routing in Cognitive Radio Networks. International Journal of Wireless Information Networks, 27(3), 1–16. https://doi.org/10.1007/s10776-019-00404-5.
Wu, J., Zhang, Y., & Zhang, L. (2021). Large-Scale Reinforcement Learning Scheduling for Age of Information in 5G Uplink. IEEE Internet of Things Journal, 8(5), 3895–3906. https://doi.org/10.1109/JIOT.2020.3033142.
Hao, Z., Jia, X., & Xu, J. (2021). Correlated Age of Information and Energy Efficiency in Heterogeneous Cognitive Radio Internet of Things Systems. arXiv preprint arXiv:2105.05051. https://doi.org/10.48550/arXiv.2105.05051.
Zhang, Y., & Cai, Y. (2024). Aging in Hybrid Radio-Optical Internet of Things Networks. ACM Transactions on Sensor Networks, 20(1), 1–22. https://doi.org/10.1145/3459876.
J. Amin, M. Sharif, A. Haldorai, M. Yasmin, and R. S. Nayak, “Brain tumor detection and classification using machine learning: a comprehensive survey,” Complex & Intelligent Systems, vol. 8, no. 4, pp. 3161–3183, Nov. 2021, doi: 10.1007/s40747-021-00563-y.
K.-L. A. Yau, G.-S. Poh, S. F. Chien, and H. A. A. Al-Rawi, “Application of Reinforcement Learning in Cognitive Radio Networks: Models and Algorithms,” The Scientific World Journal, vol. 2014, pp. 1–23, 2014, doi: 10.1155/2014/209810.
H. Anandakumar and K. Umamaheswari, “Supervised machine learning techniques in cognitive radio networks during cooperative spectrum handovers,” Cluster Computing, vol. 20, no. 2, pp. 1505–1515, Mar. 2017, doi: 10.1007/s10586-017-0798-3.
CRediT Author Statement
The author reviewed the results and approved the final version of the manuscript.
Conceptualization: Ragupathi C, Sathish Kumar Ravichandran and Sivasamy A;
Methodology: Sathish Kumar Ravichandran;
Software: Ragupathi C, Deepa R;
Data Curation: Sivasamy A;
Writing- Original Draft Preparation: Ragupathi C and Sivasamy A;
Visualization: Sivasamy A;
Investigation: Sathish Kumar Ravichandran;
Supervision: Sivasamy A;
Validation: Deepa R;
Writing- Reviewing and Editing: Sivasamy A and Deepa R; All authors reviewed the results and approved the final version of the manuscript.
Acknowledgements
We would like to thank Reviewers for taking the time and effort necessary to review the manuscript. We sincerely appreciate all valuable comments and suggestions, which helped us to improve the quality of the manuscript.
Funding
No funding was received to assist with the preparation of this manuscript.
Ethics declarations
Conflict of interest
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Availability of data and materials
Data sharing is not applicable to this article as no new data were created or analysed in this study.
Author information
Contributions
All authors have equal contribution in the paper and all authors have read and agreed to the published version of the manuscript.
Corresponding author
Ragupathi C
Department of Electrical and Electronic Engineering, P. T. Lee Chengalvaraya Naicker College of Engineering and Technology, Veliyur, Tamil Nadu, India.
Open Access This article is licensed under a Creative Commons Attribution NoDerivs is a more restrictive license. It allows you to redistribute the material commercially or non-commercially but the user cannot make any changes whatsoever to the original, i.e. no derivatives of the original work. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-nd/4.0/
Cite this article
Ragupathi C, Sathish Kumar Ravichandran, Sivasamy A and Deepa R, “Age of Information Minimization Through Reinforcement Learning in IoT Enabled Cognitive Radio Networks”, Journal of Machine and Computing, vol.5, no.4, pp. 2772-2787, October 2025, doi: 10.53759/7669/jmc202505211.
