Human-in-the-Loop Machine Learning Systems

References

[1] Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., & García-Sánchez, F. (2023). Human-in-the-loop machine learning: A state of the art. Artificial Intelligence Review, 56, 3005–3054. https://doi.org/10.1007/s10462-022-10246-w https://link.springer.com/article/10.1007/s10462-022-10246-w

[2] Wu, X., Zhu, Q., & Zhang, Y. (2022). A survey of human-in-the-loop for machine learning. Journal of Systems and Software / Computer Science (survey). https://www.sciencedirect.com/science/article/abs/pii/S0167739X22001790 https://www.sciencedirect.com/science/article/abs/pii/S0167739X22001790

[3] Li, D., Wang, J., & Zhou, Y. (2024). A survey on deep active learning: recent advances and open issues. arXiv. https://arxiv.org/abs/2405.00334https://arxiv.org/pdf/2405.00334

[4] Cacciarelli, D., & de Rijke, M. (2024). Active learning for data streams: a survey. Machine Learning (JMLR / Springer). https://doi.org/10.1007/s10994-023-06454-2 https://link.springer.com/article/10.1007/s10994-023-06454-2

[5] Lee, H., Phatale, S., Mansoor, H., Mesnard, T., Ferret, J., Lu, K. R., Bishop, C., Hall, E., Carbune, V., Rastogi, A., & Prakash, S. (2024). RLAIF vs. RLHF: Scaling reinforcement learning from human feedback with AI feedback. Proceedings of the 41st International Conference on Machine Learning (ICML). https://proceedings.mlr.press/v235/lee24t.html https://proceedings.mlr.press/v235/lee24t.html

[6] Munos, R., Valko, M., Calandriello, D., Gheshlaghi Azar, M., Rowland, M., & Piot, B. (2024). Nash learning from human feedback. Proceedings of ICML 2024. https://proceedings.mlr.press/v235/munos24a.html https://proceedings.mlr.press/v235/munos24a.html

[7] Retzlaff, C. O., Das, S., Wayllace, C., Mousavi, P., Afshari, M., Yang, T., Saranti, A., & Angerschmid, A. (2024). Human-in-the-Loop reinforcement learning: A survey and position on requirements, challenges, and opportunities. Journal of Artificial Intelligence Research, 79, 359–415. https://irll.ca/files/publications/jair.pdfhttps://irll.ca/files/publications/jair.pdf

[8] Lambert, N. (2024). Reinforcement learning from human feedback: A short introduction and practical guide (book / technical report). https://rlhfbook.com/book.pdf https://rlhfbook.com/book.pdf

[9] González Barman, K. (2025). Reinforcement learning from human feedback in large language models: epistemic and ethical perspectives. AI & Society. https://link.springer.com/article/10.1007/s13347-025-00861-0https://link.springer.com/article/10.1007/s13347-025-00861-0

[10] Senoner, J., et al. (2024). Explainable AI improves task performance in human–AI collaboration. Scientific Reports, 14, Article 82501. https://doi.org/10.1038/s41598-024-82501-9https://www.nature.com/articles/s41598-024-82501-9

[11] Yuan, H., et al. (2024). Human-in-the-loop machine learning for healthcare: progress, gaps, and opportunities. MedComm / Med4. https://onlinelibrary.wiley.com/doi/10.1002/med4.70 https://onlinelibrary.wiley.com/doi/10.1002/med4.70

[12] Hysmith, H., Foadian, E., Padhy, S. P., Kalinin, S. V., Moore, R. G., Ovchinnikova, O. S., & Ahmadi, M. (2024). The future of self-driving laboratories: from human in the loop interactive AI to gamification. Digital Discovery, 3, 621–636. https://doi.org/10.1039/D4DD00040Dhttps://pubs.rsc.org/en/content/articlelanding/2024/dd/d4dd00040d

[13] Jiang, T., et al. (2024). Human-AI interaction research agenda: a user-centered perspective. Patterns / Cell Press. https://www.sciencedirect.com/science/article/pii/S2543925124000147 https://www.sciencedirect.com/science/article/pii/S2543925124000147

[14] Shakeri Hossein Abad, Z., et al. (2022). Crowdsourcing for machine learning in public health: applications, opportunities, and challenges. Journal of Medical Internet Research (JMIR), e28749. https://www.jmir.org/2022/1/e28749/ https://www.jmir.org/2022/1/e28749/

[15] Kittur, A., et al. (2021). Quality control for crowdsourced labeling: approaches and best practices. ACM Computing Surveys / conference papers. (Case studies and methodologies). https://dl.acm.org/doi/10.1145/3464432.3464436 https://dl.acm.org/doi/10.1145/3464432.3464436

[16] Holz, T., & Holz, A. (2023). Design patterns for machine learning based systems with human-in-the-loop. arXiv. https://arxiv.org/abs/2312.00582 https://arxiv.org/abs/2312.00582

[17] Amaliah, N. R., Tjahjono, B., & Palade, V. (2025). Human-in-the-Loop XAI for predictive maintenance: a systematic review of interactive systems. Electronics, 14(17), 3384. https://doi.org/10.3390/electronics14173384 https://www.mdpi.com/2079-9292/14/17/3384

[18] Paleja, R., Munje, M., Chang, K., Jensen, R., & Gombolay, M. (2024). Designs for enabling collaboration in human-machine teaming via interactive and explainable systems. arXiv. https://arxiv.org/abs/2406.05003 https://arxiv.org/abs/2406.05003

[19] Pilaciński, A., Christ, L., Iossifidis, I., Adler, P., Miro, M., Kuhlenkötter, B., & Klaes, C. (2024). Human in the collaborative loop: Integrating human activity recognition and non-invasive brain-machine interfaces to control collaborative robots. Frontiers in Neurorobotics, 18. https://doi.org/10.3389/fnbot.2024.1383089 https://www.frontiersin.org/articles/10.3389/fnbot.2024.1383089/full

[20] Sen, S., & Stumpf, S. (2022). Interactive machine learning: a state of the art review. ACM / Research surveys. https://www.researchgate.net/publication/361981610_Interactive_Machine_Learning_A_State_of_the_Art_Review https://www.researchgate.net/publication/361981610_Interactive_Machine_Learning_A_State_of_the_Art_Review

[21] Amershi, S., et al. (2021). Humans in the loop: human-centered machine learning design patterns and guidelines. CHI / HCI literature. (Foundational design guidance for HITL systems). https://dl.acm.org/doi/10.1145/3411764.3445322 https://dl.acm.org/doi/10.1145/3411764.3445322

[22] Zhang, Y., & Wang, S. (2023). Privacy-preserving human-in-the-loop learning: methods and challenges. IEEE Transactions on Knowledge and Data Engineering. https://ieeexplore.ieee.org/document/XXXXX (searchable via IEEE Xplore) https://ieeexplore.ieee.org/

[23] Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J., & Dennison, D. (2015). Hidden technical debt in machine learning systems. Communications of the ACM, 61(10), 56–65. (Important systems perspective that remains highly relevant to HITL practices.) https://doi.org/10.1145/2827594 https://dl.acm.org/doi/10.1145/2827594

[24] Abid, A., Farooqi, M., & Zou, J. (2021). Large-scale human evaluation for aligning language models: methodology and best practices. arXiv. https://arxiv.org/abs/2109.03389 https://arxiv.org/abs/2109.03389

[25] FT Tech / Industry analysis (2025). DeepSeek’s approach: reducing human feedback costs in RL training (industry perspective). Financial Times. https://www.ft.com/content/ea803121-196f-4c61-ab70-93b38043836ehttps://www.ft.com/content/ea803121-196f-4c61-ab70-93b38043836e

@article{1711956,
      author = {Geetha Aradhyula},
      title = {Human-in-the-Loop Machine Learning Systems},
      journal = {Iconic Research And Engineering Journals},
      year = {2024},
      volume = {7},
      number = {7},
      pages = {746-753},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1711956.pdf},
      abstract = {Human-in-the-loop (HITL) machine learning systems is a paradigm change of the previous, fully automated based models to collaborative intelligence, where human expertise meets with machine efficiency to deliver higher performance. Human input features in these systems are implemented at different levels of the machine learning pipeline such as data collection, annotation, feature engineering, model training, evaluation, and post-deployment monitoring to make sure that learning processes are directed by domain knowledge, moral consciousness, and contextual awareness. HITL systems include human feedback to overcome the major issues of data scarcity, label noise, algorithmic bias, and interpretability. In active learning systems, human beings give precise corrective labels or feedback to doubtful model predictions so that the system can learn efficiently with minimal data. Such systems are especially useful in areas with high stakes, such as healthcare, finance, security, and autonomous systems because this dynamic interaction does not only help to make models more accurate, but also more transparent and trusting. HITL methods have been further enabled by recent breakthroughs in explainable AI, interactive visualization, and reinforcement learning from human feedback (RLHF) to make machines act in ways that are consistent with human values and goals. In addition, the introduction of collaborative systems of crowd-sourcing data annotation and model auditing indicates the scalability and flexibility of HITL systems in practice. Finally, human-in-the-loop machine learning will help to close the boundary between artificial and human intelligence, providing adaptive, ethical, and responsible AI solutions. It also puts humans not as observers but as agents in an ongoing learning process, which keeps AI systems resilient, equitable, and sensitive to the needs of the ever-changing society.},
      month = {January},
      doi = {https://doi.org/10.64388/IREV9I5-1711956}
  }