Home / Current Issue / Paper 1710320
Modern Data Warehousing Architectures for Real-Time Business Decision Making
Subject area: Science,Engineering and Technology · Area of research: Data Warehousing
Abstract
In an age where information is growing at a faster rate and the need to make business decisions in real-time is an absolute must, enterprises are increasingly turning to sophisticated data warehousing solutions as a means of facilitating real-time business decisions. This paper examines how data warehousing as an idea has evolved, first using the batch-oriented architecture, to hybrids and cloud-native architectures that support real-time analytics. Based on a systematic review of scholarly publications, whitepapers provided by industry, and the results of industry benchmarking, the paper will evaluate the architecture in terms of latency, scalability, cost-efficiency, and the need to make decisions. This has shown that cloud-native microservice warehouses with streaming architecture, such as Apache Kafka, and ELT support, achieve superior latency-performance costs compared to legacy systems in low-latency queries at scale. A case study of a mid-sized retail enterprise further shows how real-time analytics can potentially optimize inventory and improve customer responsiveness. We end with a summary of best-practice guidelines for selecting and deploying modern data warehouse architectures in various business scenarios, and emergent trends in serverless warehousing, datamesh, and ML-based data warehouse query optimization. The knowledge presented in this article is intended to help practitioners and researchers to implement the real-time BI in their business in order to generate lasting and high impacts.
Keywords
Data Warehousing, Real-Time Analytics, Cloud Computing, ETL, Business Intelligence
References
[1] Azeem, M. (2024). The relationship between warehousing, transportation and halal food exports: an empirical case. South Asian Journal of Social Review, 3(1), 1–14. https://doi.org/10.57044/sajsr.2024.3.1.2413
[2] Albin Ahmed, A., Shaahid, A., Alnasser, F., Alfaddagh, S., Binagag, S., & Alqahtani, D. (2024). Android Ransomware Detection Using Supervised Machine Learning Techniques Based on Traffic Analysis. Sensors, 24(1). https://doi.org/10.3390/s24010189
[3] An, K., Sun, X., Song, Y., Lu, Y., & Shangguan, Q. (2024). A DenseNet-based feature weighting convolutional network recognition model and its application in industrial part classification. IET Image Processing, 18(3), 589–601. https://doi.org/10.1049/ipr2.12971
[4] Ben Mrad, A., & Hnich, B. (2024). Intelligent attribution modeling for enhanced digital marketing performance. Intelligent Systems with Applications, 21. https://doi.org/10.1016/j.iswa.2024.200337
[5] Banik, S., Das, A., Das, B. K., & Islam, N. (2024). Numerical simulation and performance optimization of a lead-free inorganic perovskite solar cell using SCAPS-1D. Heliyon, 10(1). https://doi.org/10.1016/j.heliyon.2024.e23985
[6] Cherepenin, V. A., Smyk, N. O., & Vorob’ev, S. P. (2024). Integration of cloud, fog, and edge technologies for the optimization of high-load systems. Программные Системы и Вычислительные Методы, (1), 1–9. https://doi.org/10.7256/2454-0714.2024.1.69900
[7] Chen, G., Wu, S., Deng, Y., Jiao, J., & Zhang, Q. (2024). VLEO Satellite Constellation Design for Regional Aviation and Marine Coverage. IEEE Transactions on Network Science and Engineering, 11(1), 1188–1201. https://doi.org/10.1109/TNSE.2023.3321600
[8] Dr. Alok Singh Chauhan, Dr. B., & Dr. Mukta Makhija, D. A. A. (2024). Industry 4.0 and Indian Manufacturing Companies: Barriers towards sustainability. Journal of Informatics Education and Research, 4(1). https://doi.org/10.52783/jier.v4i1.646
[9] Dr. Alok Singh Chauhan, Dr. B., & Dr. Mukta Makhija, D. A. A. (2024). Industry 4.0 and Indian Manufacturing Companies: Barriers towards sustainability. Journal of Informatics Education and Research, 4(1). https://doi.org/10.52783/jier.v4i1.646
[10] Ejaz, Umair & Sarkar, Ankur. (2022). Intelligent Multi-Agent Robotics for Warehouse Automation and Resilient Supply Chain Infrastructure.
[11] Hsu, B. M., Wang, T. C., Shu, M. H., & Wu, C. W. (2025). Optimum Design of an Efficient Variables Sampling System for Validating Process Yield with Six-Sigma Quality Requirement and Creation of a Cloud-Computing Tool. European J. of Industrial Engineering, 19(1). https://doi.org/10.1504/ejie.2025.10059660
[12] Ganesan, I., Ponnuviji, N. P., Kumar, A. S., Nithya, M., Jambulingam, U., & Lalitha, S. D. (2024). Real-time event detection and predictive analytics using IoT and deep learning. In Industry Applications of Thrust Manufacturing: Convergence with Real-Time Data and AI (pp. 1–41). IGI Global. https://doi.org/10.4018/979-8-3693-4276-3.ch001
[13] Guo, P., Xiong, J., Wang, Y., Meng, X., & Qian, L. (2024). Intelligent Scheduling for Group Distributed Manufacturing Systems: Harnessing Deep Reinforcement Learning in Cloud-Edge Cooperation. IEEE Transactions on Emerging Topics in Computational Intelligence, 8(2), 1687–1698. https://doi.org/10.1109/TETCI.2024.3354111
[14] Gitinavard, H., Mohagheghi, V., Mousavi, S. M., & Makui, A. (2024). A new bi-stage interactive possibilistic programming model for perishable logistics distribution systems under uncertainty. Expert Systems with Applications, 238. https://doi.org/10.1016/j.eswa.2023.122121
[15] Gatov, E., Sennik, S., Goldfarb, A., Gans, J., Stein, J., Agrawal, A., & Rosella, L. (2024). Examining the Relationship Between Workplace Industry and COVID-19 Infection A Cross-sectional Study of Canada’s Largest Rapid Antigen Screening Program. Journal of Occupational and Environmental Medicine, 66(2), E68–E76. https://doi.org/10.1097/JOM.0000000000003028
[16] Khuat, T. T., Bassett, R., Otte, E., Grevis-James, A., & Gabrys, B. (2024, March 1). Applications of machine learning in antibody discovery, process development, manufacturing and formulation: Current trends, challenges, and opportunities. Computers and Chemical Engineering. Elsevier Ltd. https://doi.org/10.1016/j.compchemeng.2024.108585
[17] Maziviero, F. V., Melo, D. M. A., Medeiros, R. L. B. A., Oliveira, Â. A. S., Macedo, H. P., Braga, R. M., & Morgado, E. (2024, February 1). Advancements and Prospects in Perovskite Solar Cells: From Hybrid to All-Inorganic Materials. Nanomaterials. Multidisciplinary Digital Publishing Institute (MDPI). https://doi.org/10.3390/nano14040332
[18] Murphy, D. P. (2023). The Future of Robots in Medicine. In Robotics in Physical Medicine and Rehabilitation (pp. 179–193). Elsevier. https://doi.org/10.1016/B978-0-323-87865-4.00012-1
[19] Mohasseb, A. M. A. (2024). The Impact of Big Data Predictive Analytics on Firm Performance: The Role of Cloud ERP and Business Intelligence Integration. المجلة العلمية للدراسات والبحوث المالية والتجارية, 5(1), 917–947. https://doi.org/10.21608/cfdj.2024.329290
[20] Muldoon, J., Cant, C., Wu, B., & Graham, M. (2024). A typology of artificial intelligence data work. Big Data and Society, 11(1). https://doi.org/10.1177/20539517241232632
[21] Ompong, D., & Clements, M. (2024). Optimization of formamidinium-based perovskite solar cell using SCAPS-1D. Results in Optics, 14. https://doi.org/10.1016/j.rio.2024.100611
[22] Tungal, A., Singh, K., Singh, P., & Simintiras, A. C. (2024). Smart IoT-Enabled Cloud-Fuzzy System for Remote Monitoring of Infected Patients. In IFIP Advances in Information and Communication Technology (Vol. 698 AICT, pp. 381–392). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-50192-0_32
[23] Verma, A., Goyal, A., & Kumara, S. (2024). Machine learning-assisted collection of reduced sensor data for improved analytics pipeline. In Procedia CIRP (Vol. 121, pp. 150–155). Elsevier B.V. https://doi.org/10.1016/j.procir.2023.09.242
[24] Wang, X., Zhang, C., Liu, T., Qin, S., Lin, Z., Shi, C., … Wang, Y. (2024). Efficient Inverted Perovskite Photovoltaics Through Surface State Manipulation. Small, 20(32). https://doi.org/10.1002/smll.202311673
[25] Yakubu, A. S., Kassim, A. M., & Husin, M. H. (2025). Moderating the role of trust toward intention to adopt cloud computing by academic libraries in North-Eastern Nigerian. Journal of Librarianship and Information Science, 57(2), 551–567. https://doi.org/10.1177/09610006231222623
[26] Zhu, Z. (2024). Summarizing Data Warehousing, Data Lakes, and Data Lakehouses. In Data Warehousing, Data Lakes and Data Lakehouses. SAGE Publications, Inc. https://doi.org/10.4135/9781071937587
How to cite this paper
@article{1710320,
author = {Jyothi Swaroop Myneni},
title = {Modern Data Warehousing Architectures for Real-Time Business Decision Making},
journal = {Iconic Research And Engineering Journals},
year = {2024},
volume = {8},
number = {3},
pages = {970-978},
issn = {2456-8880},
url = {https://www.irejournals.com/formatedpaper/1710320.pdf},
abstract = {In an age where information is growing at a faster rate and the need to make business decisions in real-time is an absolute must, enterprises are increasingly turning to sophisticated data warehousing solutions as a means of facilitating real-time business decisions. This paper examines how data warehousing as an idea has evolved, first using the batch-oriented architecture, to hybrids and cloud-native architectures that support real-time analytics. Based on a systematic review of scholarly publications, whitepapers provided by industry, and the results of industry benchmarking, the paper will evaluate the architecture in terms of latency, scalability, cost-efficiency, and the need to make decisions. This has shown that cloud-native microservice warehouses with streaming architecture, such as Apache Kafka, and ELT support, achieve superior latency-performance costs compared to legacy systems in low-latency queries at scale. A case study of a mid-sized retail enterprise further shows how real-time analytics can potentially optimize inventory and improve customer responsiveness. We end with a summary of best-practice guidelines for selecting and deploying modern data warehouse architectures in various business scenarios, and emergent trends in serverless warehousing, datamesh, and ML-based data warehouse query optimization. The knowledge presented in this article is intended to help practitioners and researchers to implement the real-time BI in their business in order to generate lasting and high impacts.},
keywords = {Data Warehousing, Real-Time Analytics, Cloud Computing, ETL, Business Intelligence},
month = {September},
}