★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★★Open research. Global perspectives.
editor@theamericanjournals.comPublisher Home ↗
Engineering and Technology OPEN ACCESS

Human-in-the-Loop Frameworks in Automated Decision Systems: A Systematic Analysis of Design Patterns, Performance Characteristics, and Deployment Considerations

Srinivasarao Daruna
Senior Software Dev Engineer, USA
DOI Crossref DOI 10.37547/tajet/Volume08Issue02-03
tajet 2026
VOL. 8 / NO. 2 FEBRUARY
FULL TEXT PP. 17-25
DOWNLOAD PDF

View all articles in this issue ↗

VOLUME 8
ISSUE 2
YEAR 2026
PAGES 17-25
RESEARCH OVERVIEW

Abstract

The article examines Human-in-the-Loop (HITL) architectures for automated decision-making systems deployed in enterprise operations and regulated domains. The topic’s relevance follows from the rapid adoption of high-capacity models alongside stricter requirements for accountability, traceability, explainability, and risk control. The paper’s novelty lies in formalizing a taxonomy of intervention modes and linking engineering choices to operational metrics rather than to model accuracy alone. The study identifies four recurring intervention patterns—pre-emptive review, confidence-based routing, asynchronous audit, and exception handling—and specifies their placement within the decision pipeline. The analytical basis relies on a comparative synthesis of documented production deployments in finance, healthcare, and corporate operations, focusing on throughput, decision quality, latency, and per-case processing cost. The results indicate a non-linear trade-off between automation rate and decision quality and show that optimal thresholding depends on risk asymmetry and governance constraints. Practical recommendations address uncertainty calibration, reviewer interface design, and closed-loop feedback capture for continuous improvement. The overall objective is to provide a deployment-oriented framework for selecting HITL patterns and tuning escalation thresholds in high-stakes settings.

RESEARCH TOPICS

Keywords

human-in-the-loop automated decision-making uncertainty calibration confidence-based routing auditability explainable AI governance active learning operational metrics high-stakes systems

Downloads

Download data is not yet available.
SOURCES & SCHOLARSHIP

References

  1. Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016). Concrete problems in AI safety. arXiv preprint arXiv:1606.06565.
  2. Bansal, G., Nushi, B., Kamar, E., Lasecki, W. S., Weld, D. S., & Horvitz, E. (2019). Beyond accuracy: The role of mental models in human-AI team performance. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 7, 2–11.
  3. Bradshaw, J. M., Hoffman, R. R., Woods, D. D., & Johnson, M. (2013). The seven deadly myths of autonomous systems. IEEE Intelligent Systems, 28(3), 54–61.
  4. Dodge, J., Liao, Q. V., Zhang, Y., Bellamy, R. K., & Dugan, C. (2019). Explaining models: An empirical study of how explanations impact fairness judgment. Proceedings of the 24th International Conference on Intelligent User Interfaces, 275–285.
  5. Donmez, P., & Carbonell, J. G. (2008). Proactive learning: Cost-sensitive active learning with multiple imperfect oracles. Proceedings of the 17th ACM Conference on Information and Knowledge Management, 619–628.
  6. European Commission. (2021). Proposal for a regulation laying down harmonised rules on artificial intelligence. COM/2021/206 final.
  7. Green, B., & Chen, Y. (2019). The principles and limits of algorithm-in-the-loop decision making. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), 1–24.
  8. Horvitz, E. (1999). Principles of mixed-initiative user interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 159–166.
  9. Laprie, J. C. (1995). Dependable computing: Concepts, limits, challenges. Proceedings of the 25th IEEE International Symposium on Fault-Tolerant Computing, 42–54.
  10. Parasuraman, R., & Manzey, D. H. (2010). Complacency and bias in human use of automation: An attentional integration. Human Factors, 52(3), 381–410.
  11. Scerri, P., Pynadath, D. V., & Tambe, M. (2002). Towards adjustable autonomy for the real world. Journal of Artificial Intelligence Research, 17, 171–228.
  12. Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019). Fairness and abstraction in sociotechnical systems. Proceedings of the Conference on Fairness, Accountability, and Transparency, 59–68.
  13. Settles, B. (2009). Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin–Madison.
  14. Sheng, V. S., Provost, F., & Ipeirotis, P. (2008). Get another label? Improving data quality and data mining using multiple, noisy labelers. Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 614–622.
  15. Varshney, K. R., & Alemzadeh, H. (2017). On the safety of machine learning: Cyber-physical systems, decision sciences, and data products. Big Data, 5(3), 246–255.
  16. Welinder, P., Branson, S., Perona, P., & Belongie, S. (2010). The multidimensional wisdom of crowds. Advances in Neural Information Processing Systems 23, 2424–2432.
  17. Zhang, Y., Liao, Q. V., & Bellamy, R. K. (2020). Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 295–305.