Evaluating Programming Tools for Scalable and Efficient Data Science Applications

References

[1] Alhelou, H. H., Hamedani-Golshan, M. E., Njenda, T. C., & Siano, P. (2019, February 20). A survey on power system blackout and cascading events: Research motivations and challenges. Energies. MDPI AG. https://doi.org/10.3390/en12040682

[2] Asnawi, A. L., Ahmad, A., Azmin, N. F. M., Ismail, K., Jusoh, A. Z., Ibrahim, S. N., & Mohd Ramli, H. A. (2019). The needs of collaborative tool for practicing pair programming in educational setting. International Journal of Interactive Mobile Technologies, 13(7), 17–30. https://doi.org/10.3991/ijim.v13i07.10722

[3] Bachiller-Burgos, P., Barbecho, I., Calderita, L. V., Bustos, P., & Manso, L. J. (2020). LearnBlock: A Robot-Agnostic Educational Programming Tool. IEEE Access, 8, 30012–30026. https://doi.org/10.1109/ACCESS.2020.2972410

[4] Bezanson, J., Edelman, A., Karpinski, S., & Shah, V. B. (2017). Julia: A fresh approach to numerical computing. SIAM Review, 59(1), 65–98. https://doi.org/10.1137/141000671

[5] Blank, J., & Deb, K. (2020). Pymoo: Multi-Objective Optimization in Python. IEEE Access, 8, 89497–89509. https://doi.org/10.1109/ACCESS.2020.2990567

[6] Fairbrother, J., Nemeth, C., Rischard, M., Brea, J., & Pinder, T. (2022). GaussianProcesses.jl: A Nonparametric Bayes Package for the Julia Language. Journal of Statistical Software, 102. https://doi.org/10.18637/jss.v102.i01

[7] Gao, K., Mei, G., Piccialli, F., Cuomo, S., Tu, J., & Huo, Z. (2020, August 1). Julia language in machine learning: Algorithms, applications, and open issues. Computer Science Review. Elsevier Ireland Ltd. https://doi.org/10.1016/j.cosrev.2020.100254

[8] Gramfort, A., Luessi, M., Larson, E., Engemann, D. A., Strohmeier, D., Brodbeck, C., … Hämäläinen, M. (2013). MEG and EEG data analysis with MNE-Python. Frontiers in Neuroscience, (7 DEC). https://doi.org/10.3389/fnins.2013.00267

[9] Hall, K. R., Anantharaman, R., Landau, V. A., Clark, M., Dickson, B. G., Jones, A., … Shah, V. B. (2021). Circuitscape in julia: Empowering dynamic approaches to connectivity assessment. Land, 10(3). https://doi.org/10.3390/land10030301

[10] Hines, M. L., Davison, A. P., & Muller, E. (2009). NEURON and Python. Frontiers in Neuroinformatics, 3(JAN). https://doi.org/10.3389/neuro.11.001.2009

[11] Impram, S., Varbak Nese, S., & Oral, B. (2020, September 1). Challenges of renewable energy penetration on power system flexibility: A survey. Energy Strategy Reviews. Elsevier Ltd. https://doi.org/10.1016/j.esr.2020.100539

[12] Khan, D., Jung, L. T., & Hashmani, M. A. (2021, October 2). Systematic literature review of challenges in blockchain scalability. Applied Sciences (Switzerland). MDPI. https://doi.org/10.3390/app11209372

[13] Nadeem, F., Hussain, S. M. S., Tiwari, P. K., Goswami, A. K., & Ustun, T. S. (2019). Comparative review of energy storage systems, their roles, and impacts on future power systems. IEEE Access. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ACCESS.2018.2888497

[14] Nikulchev, E., Ilin, D., Kolyasnikov, P., Belov, V., Zakharov, I., & Malykh, S. (2018). Programming technologies for the development of web-based platform for digital psychological tools. International Journal of Advanced Computer Science and Applications, 9(8), 34–45. https://doi.org/10.14569/ijacsa.2018.090806

[15] Nocoń, A., & Paszek, S. (2023, February 1). A Comprehensive Review of Power System Stabilizers. Energies. MDPI. https://doi.org/10.3390/en16041945

[16] Parsons, D., & Haden, P. (2006). Parson’s programming puzzles: A fun and effective learning tool for first programming courses. Conferences in Research and Practice in Information Technology Series, 52, 157–163.

[17] Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., … Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.

[18] Roald, L. A., Pozo, D., Papavasiliou, A., Molzahn, D. K., Kazempour, J., & Conejo, A. (2023). Power systems optimization under uncertainty: A review of methods and applications. Electric Power Systems Research, 214. https://doi.org/10.1016/j.epsr.2022.108725

[19] Richer, G., Pister, A., Abdelaal, M., Fekete, J. D., Sedlmair, M., & Weiskopf, D. (2024). Scalability in Visualization. IEEE Transactions on Visualization and Computer Graphics, 30(7), 3314–3330. https://doi.org/10.1109/TVCG.2022.3231230

[20] Svec, D., Tichopad, A., Novosadova, V., Pfaffl, M. W., & Kubista, M. (2015). How good is a PCR efficiency estimate: Recommendations for precise and robust qPCR efficiency assessments. Biomolecular Detection and Quantification, 3, 9–16. https://doi.org/10.1016/j.bdq.2015.01.005

[21] Swathi, P., & Venkatesan, M. (2021). Scalability improvement and analysis of permissioned-blockchain. ICT Express, 7(3), 283–289. https://doi.org/10.1016/j.icte.2021.08.015

[22] Tippmann, S. (2015, January 1). Programming tools: Adventures with R. Nature. Nature Publishing Group. https://doi.org/10.1038/517109a

[23] Ucbas, Y., Eleyan, A., Hammoudeh, M., & Alohaly, M. (2023). Performance and Scalability Analysis of Ethereum and Hyperledger Fabric. IEEE Access, 11, 67156–67167. https://doi.org/10.1109/ACCESS.2023.3291618

[24] Van Der Walt, S., Schönberger, J. L., Nunez-Iglesias, J., Boulogne, F., Warner, J. D., Yager, N., … Yu, T. (2014). Scikit-image: Image processing in python. PeerJ, 2014(1). https://doi.org/10.7717/peerj.453

[25] Xinogalos, S., & Tryfou, M. M. (2021). Using Greenfoot as a tool for serious games programming education and development. International Journal of Serious Games, 8(2), 67–86. https://doi.org/10.17083/ijsg.v8i2.425

[26] Yilmaz, R., & Karaoglan Yilmaz, F. G. (2023). The effect of generative artificial intelligence (AI)-based tool use on students’ computational thinking skills, programming self-efficacy and motivation. Computers and Education: Artificial Intelligence, 4. https://doi.org/10.1016/j.caeai.2023.100147

[27] Zhou, Q., Huang, H., Zheng, Z., & Bian, J. (2020). Solutions to Scalability of Blockchain: a Survey. IEEE Access, 8, 16440–16455. https://doi.org/10.1109/aCCESS.2020.2967218

[28] Ejaz, Umair & Islam, S A Mohaiminul & Sarkar, Ankur & Imashev, Aidar. (2024). Federated Learning for Secure and Privacy-Preserving Medical Collaboration Across Multi-Cloud Healthcare Systems. IOSR Journal of Mechanical and Civil Engineering. 21. 36-44. https://doi.org/10.9790/1684-2105023644.

@article{1710319,
      author = {Jyothi Swaroop Myneni},
      title = {Evaluating Programming Tools for Scalable and Efficient Data Science Applications},
      journal = {Iconic Research And Engineering Journals},
      year = {2024},
      volume = {8},
      number = {2},
      pages = {1131-1145},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1710319.pdf},
      abstract = {The explosion in the number of information-centric applications has markedly affected research and industrial practices in a variety of areas, and the power system domain is one of the hardest hit. Smart grid, smart power meters, and vast numbers of sensors have produced astronomical volumes of data, and any system tasked with parsing that information will require highly scalable and efficient computation. Conventional programming solutions are well-suited to small-scale problems, but they are not always able to support large-scale power system analytics with regard to both computation and scalability. The selection of programming tools gains importance so that timely insights are achieved, offering the effective use of resources and providing reliable system performance. This paper compares four common programming platforms, Python, R, Julia, and C++, in terms of their suitability for large-scale data science applications in power system technology. It will use benchmarking approaches that combine synthetic and real-world datasets of the smart grid. The main benchmarks, such as the execution rate, scalability in the distributed architectures, memory consumption, and power system workflow adaptability, were analysed on a high-performance computer cluster. The datasets have been found to be small-scale (10 GB) to large-scale (1TB) to represent the varied operating conditions. Simulated workloads were assigned as short-time load forecasting, anomaly detection, and real-time monitoring, which are the keystones of the current power system analytics. The empirical results indicate that Python, particularly with the use of distributed frameworks like Apache Spark or Dask, is a good middle ground in terms of scalability, usability, and interoperability with machine learning packages, and may therefore be an appropriate selection when forecasting and real-time monitoring of the system are desired. Julia shows impressive efficiency; although Julia is slightly slower than C++, it has a high-level syntax that qualifies it for time-sensitive applications like fault detection and predictive maintenance. C++ remains the undisputed king within the realm of raw computing speed, and is particularly popular in applications dependent on latency and simulation-intensive, although its sharp learning curve and the cost to maintain those skills are quite high. },
      keywords = {Programming tools, scalability, efficiency, power systems, data science, Python, Julia, R, C++.},
      month = {August},
  }