A Survey of Tools and Techniques of Big Data Management in Cloud Environments
Main Article Content
Abstract
Big data has emerged as a critical asset for organizations due to the rapid growth of data generated from social media, Internet of Things (IoT) devices, business transactions, and digital services. Managing such massive volumes of data requires scalable, efficient, and reliable computing infrastructures. Cloud computing provides a flexible platform for storing, processing, and analyzing big data through distributed resources and on-demand services. This survey paper presents an overview of big data management in cloud environments, focusing on the key tools and techniques used for data storage, processing, analytics, and security. The study discusses widely adopted tools such as Hadoop, MapReduce, Spark, HBase, and NoSQL databases, along with techniques including data mining, machine learning, predictive analytics, optimization methods, and text analytics. Furthermore, the paper highlights major challenges related to scalability, resource management, data privacy, and security in cloud-based big data systems. A comparative review of recent research works is also provided to identify current trends and future directions. The survey concludes that the integration of advanced analytics techniques with cloud computing can significantly improve the efficiency, reliability, and security of big data management.
Downloads
Article Details
Section
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) License. Authors retain the copyright of their work and grant the Journal of Global Research in Multidisciplinary Studies (JGRMS) the right of first publication. This license permits unrestricted use, distribution, adaptation, and reproduction in any medium or format, provided the original author(s), source, and publication are properly credited. Users may copy, redistribute, remix, transform, and build upon the published material for any purpose, including commercial use, in accordance with the terms of the CC BY 4.0 License.
How to Cite
References
[1] B. Sotomayor, R. S. Montero, I. M. Llorente, and I. Foster, “Virtual Infrastructure Management in Private and Hybrid Clouds,” IEEE Internet Comput., vol. 13, no. 5, pp. 14–22, Sep. 2009, doi: 10.1109/MIC.2009.119.
[2] T. H. Noor, Q. Z. Sheng, S. Zeadally, and J. Yu, “Trust management of services in cloud environments,” ACM Comput. Surv., vol. 46, no. 1, pp. 1–30, Oct. 2013, doi: 10.1145/2522968.2522980.
[3] A. Joon, B. K. R. Janumpally, A. Gogineni, and P. Chatterjee, “Efficient Large-Scale Intrusion Identification and Prevention in Distributed Cloud Networks Using Artificial Intelligence,” in 2025 5th International Conference on Intelligent Technologies (CONIT), HUBBALI, India: IEEE, Jun. 2025, pp. 1–8. doi: 10.1109/CONIT65521.2025.11167760.
[4] D. A. B. Fernandes, L. F. B. Soares, J. V. Gomes, M. M. Freire, and P. R. M. Inácio, “Security issues in cloud environments: a survey,” Int. J. Inf. Secur., vol. 13, no. 2, pp. 113–170, Apr. 2014, doi: 10.1007/s10207-013-0208-7.
[5] B. Suvarnamukhi and M. Seshashayee, “Big Data Concepts and Techniques in Data Processing,” Int. J. Comput. Sci. Eng., vol. 6, no. 10, pp. 712–714, Oct. 2018, doi: 10.26438/ijcse/v6i10.712714.
[6] A. Oussous, F.-Z. Benjelloun, A. Ait Lahcen, and S. Belfkih, “Big Data technologies: A survey,” J. King Saud Univ. - Comput. Inf. Sci., vol. 30, no. 4, pp. 431–448, Oct. 2018, doi: 10.1016/j.jksuci.2017.06.001.
[7] A. Siddiqa et al., “A survey of big data management: Taxonomy and state-of-the-art,” J. Netw. Comput. Appl., vol. 71, pp. 151–166, Aug. 2016, doi: 10.1016/j.jnca.2016.04.008.
[8] P. Neves and J. Bernardino, “Big Data in the Cloud: A Survey,” Open J. Big Data, vol. 1, pp. 3–20, 2015.
[9] A. Joon, S. A. Pahune, S. S. S. Neeli, and B. Boddu, “A Robust Artificial Intelligence-Based Financial Forecasting System to Enhance Accuracy with Machine Learning Algorithms,” in Proceedings of the Second International Conference on Advanced Computing and Systems, W. Ding, A. Chakrabarti, M. Chakraborty, and S. Chakraborty, Eds., Cham: Springer Nature Switzerland, 2026, pp. 522–533.
[10] S. K. Madishetty and G. N. M. V. Kumar, “Intelligent Surgical Tables: Real-Time Patient Monitoring and Autonomous Adjustment Using IoT, Cloud Computing, and Cybersecurity,” J. Comput. Anal. Appl., vol. 31, no. 02, pp. 803–809, Feb, Feb. 2023, doi: 10.48047/jocaaa.2023.31.02.50.
[11] T. Cherian, “A Study And Survey Of Big Data Using Data Mining Techniques,” Int. J. Eng. Sci. Res. Technol., 2019.
[12] S. Kumar and M. Singh, “Big data analytics for healthcare industry: impact, applications, and tools,” Big Data Min. Anal., vol. 2, no. 1, pp. 48–57, Mar. 2019, doi: 10.26599/BDMA.2018.9020031.
[13] S. Aslam and M. A. Shah, “Load balancing algorithms in cloud computing: A survey of modern techniques,” in 2015 National Software Engineering Conference (NSEC), IEEE, Dec. 2015, pp. 30–35. doi: 10.1109/NSEC.2015.7396341.
[14] D. A. Shafiq, N. Z. Jhanjhi, and A. Abdullah, “Load balancing techniques in cloud computing environment: A review,” J. King Saud Univ. - Comput. Inf. Sci., vol. 34, no. 7, pp. 3910–3933, Jul. 2022, doi: 10.1016/j.jksuci.2021.02.007.
[15] B. Saraladevi, N. Pazhaniraja, P. V. Paul, M. S. S. Basha, and P. Dhavachelvan, “Big Data and Hadoop-a Study in Security Perspective,” Procedia Comput. Sci., vol. 50, pp. 596–601, 2015, doi: 10.1016/j.procs.2015.04.091.
[16] A. Fahad et al., “A Survey of Clustering Algorithms for Big Data: Taxonomy and Empirical Analysis,” IEEE Trans. Emerg. Top. Comput., vol. 2, no. 3, pp. 267–279, Sep. 2014, doi: 10.1109/TETC.2014.2330519.
[17] A. K. Sandhu, “Big data with cloud computing: Discussions and challenges,” Big Data Min. Anal., vol. 5, no. 1, pp. 32–40, Mar. 2022, doi: 10.26599/BDMA.2021.9020016.
[18] V. K. Sharma, “Cloud Computing & IoT: 5G Focused IoT with Cloud Solutions,” Int. J. AI, BigData, Comput. Manag. Stud., vol. 6, no. 3, pp. 21–25, July, 2025, doi: 10.63282/3050-9416.IJAIBDCMS-V6I3P103.
[19] R. Krebs, C. Momm, and S. Kounev, “Metrics and techniques for quantifying performance isolation in cloud environments,” Sci. Comput. Program., vol. 90, pp. 116–134, Sep. 2014, doi: 10.1016/j.scico.2013.08.003.
[20] R. K. Kanneganti, “Security and Compliance Issues in Cloud-Based Deployments of Content and Workflow Management Systems,” Eastasouth J. Inf. Syst. Comput. Sci., vol. 2, no. 01, pp. 131–138, Aug. 2024, doi: 10.58812/esiscs.v2i01.1122.
[21] Tanmaykumar Shah, “Cloud-Based Data Warehousing for Marketing Agility: Lessons from FinTech Migrations to Snowflake and AWS,” Int. J. Adv. Res. Sci. Commun. Technol., vol. 4, no. 4, March, pp. 642–652, Mar. 2024, doi: 10.48175/IJARSCT-16000B.
[22] S. Verma, “Survey Paper of Cloud Computing,” vol. 11, no. 01, pp. 431–433, 2022.
[23] M. R. C. Mukkolakkal, “1.5 Million Messages Per Second on 3 Machines: Benchmarking and Latency Optimization of Apache Pulsar at Enterprise Scale,” in 22nd International Conference on Network and Service Management, 2026, pp. 1–4, Mar. doi: https://doi.org/10.48550/arXiv.2603.29113.
[24] C. Xie, J. Gao, and C. Tao, “Big Data Validation Case Study,” in 2017 IEEE Third International Conference on Big Data Computing Service and Applications (BigDataService), IEEE, Apr. 2017, pp. 281–286. doi: 10.1109/BigDataService.2017.44.
[25] H. Venkatesh, S. D. Perur, and N. Jalihal, “A Study on Use of Big Data in Cloud Computing Environment,” Int. J. Comput. Sci. Inf. Technol., 2015.
[26] K. Singh and R. Kaur, “Hadoop: Addressing challenges of Big Data,” in 2014 IEEE International Advance Computing Conference (IACC), IEEE, Feb. 2014, pp. 686–689. doi: 10.1109/IAdCC.2014.6779407.
[27] K. Skracic and I. Bodrusic, “A Big Data solution for troubleshooting mobile network performance problems,” in 2017 40th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), IEEE, May 2017, pp. 472–477. doi: 10.23919/MIPRO.2017.7973471.
[28] I. Soric, D. Dinjar, M. Stajcer, and D. Orescanin, “Efficient social network analysis in big data architectures,” in 2017 40th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), IEEE, May 2017, pp. 1397–1400. doi: 10.23919/MIPRO.2017.7973640.
[29] M. Kari, “A comparative performance study of hashing algorithms: MD5,SHA-256, and SHA-512 using Java : 2 NEEDED,” Int. J. Curr. Eng. Technol., vol. 13, no. 06, pp. 619–626, Nov/Dec, 2023, doi: https://doi.org/10.14741/ijcet/v.13.6.14.
[30] J. P. Verma, B. Patel, and A. Patel, “Big Data Analysis: Recommendation System with Hadoop Framework,” in 2015 IEEE International Conference on Computational Intelligence & Communication Technology, IEEE, Feb. 2015, pp. 92–97. doi: 10.1109/CICT.2015.86.
[31] W. Almeida, L. Monteiro, A. Lima, R. Rodrigues, and F. Escobar, “Survey on Trends in Big Data: Data Management, Integration and Cloud Computing Environment,” 2019.
[32] N. Khan et al., “Big Data: Survey, Technologies, Opportunities, and Challenges,” Sci. World J., vol. 2014, pp. 1–18, 2014, doi: 10.1155/2014/712826.
[33] Mr. Jay Bharat Mehta, “Ai-Driven Test Engineering For Cloud-Native Systems,” Int. J. Data Sci. IoT Manag. Syst., vol. 5, no. 1, Jan. 2026, doi: 10.64751/ijdim.2026.v5i1.297.
[34] S. Zhang, “Research on the Construction of Information System of Enterprise Financial Management in the Big Data Environment,” in 2025 International Conference on Information Technology, Communication Ecosystem and Management (ITCEM), IEEE, Dec. 2025, pp. 189–193. doi: 10.1109/ITCEM68692.2025.00043.
[35] J. Fan, F. Han, and H. Liu, “Challenges of Big Data analysis,” Natl. Sci. Rev., vol. 1, no. 2, pp. 293–314, Jun. 2014, doi: 10.1093/nsr/nwt032.
[36] M. Chen, S. Mao, and Y. Liu, “Big data: A survey. Mobile Networks and Applications,” Mob. Networks Appl., 2014.
[37] A. Gandomi and M. Haider, “Beyond the hype: Big data concepts, methods, and analytics,” Int. J. Inf. Manage., vol. 35, no. 2, pp. 137–144, Apr. 2015, doi: 10.1016/j.ijinfomgt.2014.10.007.
[38] S. Devarakonda, R. LINGAM, and V. Challa, “Confidence-Gated RAG for Adaptive Retrieval in Sequential Agents,” in ICLR 2026 Workshop on Logical Reasoning of Large Language Models, ICLR, 2026, pp. 01–10, Apr.
[39] R. D. Patil and O. S. Jadhav, “Some Contribution of Statistical Techniques in Big Data : A Review,” vol. 4, no. April, pp. 293–303, 2016.
[40] M. N. Paralath, “Bridging Astronomical Imagery and Text with Advanced Multi-Modal AI,” in 2025 IEEE 7th International Conference on Computing, Communication and Automation (ICCCA), Greater Noida, India: IEEE, Nov. 2025, pp. 1–8. doi: 10.1109/ICCCA66364.2025.11325649.
[41] B. Zhang, P. He, and N. Jin, “A Multi-scale Big Data Analytics and Interactive Visualization Framework for IoT Smart Home Energy Efficiency Using Unity,” in 2026 6th International Symposium on Computer Technology and Information Science (ISCTIS), IEEE, May 2026, pp. 1359–1365. doi: 10.1109/ISCTIS70043.2026.11572251.
[42] P. N. Deshmukh and V. H. Deshmukh, “Securing Big Data Storage in a Cloud Environment,” in 2025 3rd DMIHER International Conference on Artificial Intelligence in Healthcare, Education and Industry (IDICAIHEI), IEEE, Nov. 2025, pp. 1–5. doi: 10.1109/IDICAIHEI65991.2025.11378052.
[43] S. Yang, Q. Xia, and B. Zhu, “Exploration of Data Reuse Patterns Based on Scientific Data Lifecycle in Big Data Environment,” in 2024 9th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA), IEEE, Apr. 2024, pp. 280–284. doi: 10.1109/ICCCBDA61447.2024.10569511.
[44] Y. Yang, “Practice and Exploration of Multidimensional Resource Scheduling and Optimization in Cloud Computing Big Data Environment,” in 2024 International Conference on Interactive Intelligent Systems and Techniques (IIST), IEEE, Mar. 2024, pp. 398–402. doi: 10.1109/IIST62526.2024.00113.
[45] S. Wu, B. Cui, and Y. Zhang, “Application of Real-Time Big Data Technology in Complex Power Engineering Management,” in 2023 8th International Conference on Data Science in Cyberspace (DSC), IEEE, Aug. 2023, pp. 526–532. doi: 10.1109/DSC59305.2023.00082.
[46] X. Miao, “Application of Cloud Computing in Information Management System in the Era of Big Data,” in 2022 2nd International Conference on Computational Modeling, Simulation and Data Analysis (CMSDA), IEEE, Dec. 2022, pp. 307–310. doi: 10.1109/CMSDA58069.2022.00063.