Architecting AI-First Applications: Software Development Patterns for LLM-Integrated Systems at Scale

References

[1] Brooks, F. P. (1987). No silver bullet: Essence and accidents of software engineering. IEEE Computer, 20(4), 10–19.

[2] Simon, H. A. (1996). The Sciences of the Artificial (3rd ed.). MIT Press.

[3] Kleppmann, M. (2017). Designing Data-Intensive Applications. O’Reilly Media.

[4] Richards, M., & Ford, N. (2020). Fundamentals of Software Architecture. O’Reilly Media.

[5] Bass, L., Clements, P., & Kazman, R. (2021). Software Architecture in Practice (4th ed.). Addison-Wesley.

[6] Dean, J., & Barroso, L. A. (2013). The tail at scale. Communications of the ACM, 56(2), 74–80.

[7] Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., … Zimmermann, T. (2019). Software engineering for machine learning: A case study. Proceedings of the 41st International Conference on Software Engineering (ICSE), 291–300.

[8] Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., … Dennison, D. (2015). Hidden technical debt in machine learning systems. Advances in Neural Information Processing Systems (NeurIPS), 28, 2503–2511.

[9] Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., … Liang, P. (2021). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258.

[10] Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., … Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems (NeurIPS), 33, 1877–1901.

[11] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems (NeurIPS), 30, 5998–6008.

[12] Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., … Riedel, S. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems (NeurIPS), 33, 9459–9474.

[13] Zaharia, M., Chen, A., Davidson, A., Ghodsi, A., Hong, S., Konwinski, A., … Stoica, I. (2018). Accelerating the machine learning lifecycle with MLflow. IEEE Data Engineering Bulletin, 41(4), 39–45.

[14] Breck, E., Cai, S., Nielsen, E., Salib, M., & Sculley, D. (2017). The ML test score: A rubric for ML production readiness. Proceedings of the IEEE International Conference on Big Data, 1123–1132.

[15] Hellerstein, J. L., Diao, Y., Parekh, S., & Tilbury, D. M. (2004). Feedback Control of Computing Systems. Wiley-IEEE Press.

[16] Ozkaya, I., Kazman, R., & Klein, M. (2016). Managing Technical Debt: Reducing Friction in Software Development. Addison-Wesley.

[17] Mitchell, T. M. (1997). Machine Learning. McGraw-Hill.

[18] Kreps, J. (2014). Questioning the lambda architecture. O’Reilly Radar.

[19] Wieringa, R. (2014). Design Science Methodology for Information Systems and Software Engineering. Springer.

[20] Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016). Concrete problems in AI safety. arXiv preprint arXiv:1606.06565.

@article{1714655,
      author = {Umut Gumeli},
      title = {Architecting AI-First Applications: Software Development Patterns for LLM-Integrated Systems at Scale},
      journal = {Iconic Research And Engineering Journals},
      year = {2024},
      volume = {7},
      number = {12},
      pages = {689-699},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1714655.pdf},
      abstract = {The rapid adoption of large language models (LLMs) has fundamentally altered the landscape of software development. While early applications treated AI capabilities as isolated features or external services, a growing class of systems now place LLMs at the core of application behavior. These AI-first applications rely on probabilistic reasoning, dynamic context construction, and adaptive execution flows that challenge traditional software architecture assumptions.
This paper argues that architecting AI-first applications requires a rethinking of software development patterns rather than incremental adaptation of existing models. LLM-integrated systems differ from conventional software in their non-deterministic behavior, variable cost profiles, and tight coupling between data, inference, and user interaction. Treating LLMs as interchangeable libraries or black-box APIs obscures these characteristics and leads to brittle, inefficient, and unscalable systems. The study examines architectural challenges unique to LLM-integrated systems, including context management, reliability under uncertainty, latency variability, and observability of AI behavior. It proposes a set of software development patterns that address these challenges, emphasizing separation of intent and execution, orchestration-based control flows, and infrastructure-aware design. Rather than focusing on specific models or vendors, the paper adopts a system-centric perspective applicable across evolving AI platforms. The contributions of this work are threefold. First, it distinguishes AI-first applications from AI-enabled systems and clarifies the architectural implications of this distinction. Second, it articulates core design principles and patterns for integrating LLMs into scalable software systems. Third, it analyzes how AI-first architectures reshape the software development lifecycle, from testing and deployment to monitoring and governance. By grounding AI integration in software engineering fundamentals, this paper provides a foundation for building robust, scalable, and responsible AI-first applications.},
      keywords = {AI-First Software Development; Large Language Models; LLM-Integrated Systems; Scalable AI Architectures; Intelligent Applications; Modern Software Engineering},
      month = {June},
      doi = {https://doi.org/10.64388/IREV7I12-1714655}
  }