An Adaptive Prompt Optimization Framework for Domain-Specific Large Language Models
DOI:
https://doi.org/10.66280/ijair.v1i1.101Keywords:
adaptive prompt optimization; domain-specific large language models; inference-time control; retrieval-augmented generation; online adaptation; calibration and robustnessAbstract
Domain-specific deployment of large language models (LLMs) remains constrained by prompt brittleness, inference cost, and uneven generalization across task subtypes. This paper presents APOF (Adaptive Prompt Optimization Framework), a closed-loop framework that jointly opti- mizes prompt structure, retrieval context, and inference-time control signals for domain-specific LLM applications. APOF combines three elements: (i) a policy-guided prompt composer that dynamically allocates instruction budget across task facets, (ii) a critic model that estimates prompt-task fitness before expensive decoding, and (iii) an online adaptation module that up- dates prompt policies using delayed feedback from production outcomes. We instantiate APOF in three high-stakes domains—clinical note summarization, legal clause risk classification, and materials-science question answering—using a shared 13B parameter base model and domain adapters.
Our experiments include 162,000 annotated instances across public and institutionally cu- rated corpora, 12 baseline methods, and controlled ablation studies. APOF improves macro-F1 by up to 6.8 points over the strongest static prompt baseline, while reducing median latency by 18.4% through pre-decoding prompt pruning and adaptive generation parameters. The frame- work also improves calibration (ECE reduction of 0.041) and demonstrates higher robustness under distribution shift (average relative performance drop 12.3% vs. 21.7% for static meth- ods). We provide mathematical formulations, complexity analysis, and practical deployment recommendations. Results suggest that adaptive prompting, when treated as a structured op- timization problem rather than manual engineering, is a viable path to reliable domain-specific LLM systems.
References
1.Y.Zhou, A. Madaan, Z. Wang, S. Upadhyay, and G. Neubig, ”Large Language Models as Op- timizers for Prompting: A Survey of Black-Box and Bandit-Based Strategies,” arXiv preprint arXiv:2309.03409, 2023.
2.Qi, R. (2025, August). Interpretable Slow-Moving Inventory Forecasting: A Hybrid Neural Network Approach with Interactive Visualization. In Proceedings of the 2025 International Conference on Generative Artificial Intelligence for Business (pp. 41-46).
3.Liu, T. (2022, December). Financial Constraint’Impact on Firms’ ESG Rating Based on Chinese Stock Market. In 2022 4th International Conference on Economic Management and Cultural Industry (ICEMCI 2022) (pp. 1085-1095). Atlantis Press.
4.Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. K”uttler, M. Lewis, W.-t. Yih, T. Rockt”aschel, S. Riedel, and D. Kiela, ”Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” in Advances in Neural Information Processing Systems (NeurIPS), 2020.
5.Zhang, T. (2025, October). From Black Box to Actionable Insights: An Adaptive Explainable AI Framework for Proactive Tax Risk Mitigation in Small and Medium Enterprises. In Proceedings of the 2025 2nd International Conference on Digital Economy and Computer Science (pp. 193-199).
6. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, ”LoRA: Low-Rank Adaptation of Large Language Models,” in International Conference on Learning Representations (ICLR), 2022.
7.Liu, T. (2026). A Comparative Study of Transformer-Based and Classical Models for Financial Time-Series Forecasting. Journal of Risk and Financial Management, 19(3), 203.
8. X. Wang, J. Wei, D. Schuurmans, Q. Le, E. H. Chi, S. Narang, A. Chowdhery, and D. Zhou, ”Self-Consistency Improves Chain of Thought Reasoning in Language Models,” in International Conference on Learning Representations (ICLR), 2023.
9. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al., ”Training Language Models to Follow Instructions with Human Feedback,” in Advances in Neural Information Processing Systems (NeurIPS), 2022.
10. T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wier- stra, ”Continuous Control with Deep Reinforcement Learning,” International Conference on Learning Representations (ICLR), 2016.
11. F. Doshi-Velez and B. Kim, ”Towards A Rigorous Science of Interpretable Machine Learning,” arXiv preprint arXiv:1702.08608, 2017.
12. B. Settles, ”Active Learning Literature Survey,” University of Wisconsin–Madison, Computer Sciences Technical Report 1648, 2009.
13. J. Garcia and F. Fernandez, ”A Comprehensive Survey on Safe Reinforcement Learning,” Journal of Machine Learning Research, vol. 16, pp. 1437–1480, 2015.
14.Zhou, D. (2025, December). M-VP2: Microservice-Oriented Vulnerability Patch Planning-A Cost-Aware Approachusing Multi-Agent Reinforcement Learning. In 2025 5th International Conference on Computer, Internet of Things and Control Engineering (CITCE) (pp. 248-254). IEEE.
15.Amershi, M. Cakmak, W. Bradley Knox, and T. Kulesza, ”Power to the People: The Role of Humans in Interactive Machine Learning,” AI Magazine, vol. 35, no. 4, pp. 105–120, 2014.
16. J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. Le, and D. Zhou, ”Chain- of-Thought Prompting Elicits Reasoning in Large Language Models,” in Advances in Neural Information Processing Systems (NeurIPS), 2022.
17.Yi, X. (2026). Privacy-Enhanced Ad Targeting for Social E-Commerce: A Federated Learning Framework with Zero-Knowledge Verification for Creator Monetization. Frontiers in Business and Finance, 3(1), 102-113.
18. S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. Lee, Y. T. Li, S. Lundberg, et al., ”Sparks of Artificial General Intelligence: Early Experiments with GPT-4,” arXiv preprint arXiv:2303.12712, 2023.
19.Yi, X. (2025, October). Compliance-by-Design Micro-Licensing for AI-Generated Content in Social Commerce Using C2PA Content Credentials and W3C ODRL Policies. In 2025 7th International Conference on Machine Learning, Big Data and Business Intelligence (MLBDBI) (pp. 204-208). IEEE.
20.Zhou, N. Sch”arli, L. Hou, J. Wei, S. Scao, X. Wang, P. Schuurmans, and Q. Le, ”Least- to-Most Prompting Enables Complex Reasoning in Large Language Models,” in International Conference on Learning Representations (ICLR), 2023.
21.Wang, A. M. Sordoni, Y. Tay, and D. Metzler, ”A Survey of Large Language Model Bench- marks,” arXiv preprint arXiv:2308.XXXX, 2023.
22. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, C. Sun, and H. Wang, ”Retrieval-Augmented Generation for Large Language Models: A Survey,” arXiv preprint arXiv:2312.10997, 2023.
23.Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al., ”Llama 2: Open Foundation and Fine-Tuned Chat Models,” arXiv preprint arXiv:2307.09288, 2023.
24.Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, et al., ”Mistral 7B,” arXiv preprint arXiv:2310.06825, 2023.
25.Leviathan, M. Kalman, and Y. Matias, ”Fast Inference from Transformers via Speculative Decoding,” in International Conference on Machine Learning (ICML), 2023.
26. Gemma Team, ”Gemma: Open Models Based on Gemini Research and Technology,” arXiv preprint arXiv:2403.08295, 2024.
27.Chang, X. Wang, J. Wang, Y. Wu, L. Zhu, H. Chen, X. Yi, C. Wang, Y. Wang, W. Ye, et al., ”A Survey on Evaluation of Large Language Models,” ACM Transactions on Intelligent Systems and Technology, vol. 15, no. 3, 2024.
28.Zou, Z. Wang, J. Z. Kolter, and M. Fredrikson, ”Universal and Transferable Adversarial Attacks on Aligned Language Models,” arXiv preprint arXiv:2307.15043, 2023.
29. R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, et al., ”On the Opportunities and Risks of Foundation Models,” arXiv preprint arXiv:2108.07258, 2021.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 International Journal of Artificial Intelligence Research
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
