Journal of Accounting and Management Vision

Journal of Accounting and Management Vision

An Intelligent Multi-Agent Reconciliation System Based on Large Language Models: From Heterogeneous Document Matching to Real-Time Financial Statement Validation

Document Type : Original Article

Authors
Department of Accounting, LIAN INSTITUTE OF HIGHER EDUCATION of Bushehr, Iran
Abstract
Objective: This study aims to design, implement, and evaluate an intelligent multi-agent reconciliation system powered by Large Language Models (LLMs) with a Retrieval-Augmented Generation (RAG) architecture. The primary objective is to automate the reconciliation of heterogeneous financial documents across purchasing, sales, inventory, receivables, payables, and bank reconciliation cycles so that discrepancies are identified and resolved at source, thereby enhancing the accuracy and reliability of financial statements.

Methods: The research adopts an applied-developmental approach with an experimental simulation strategy. The proposed architecture comprises six specialized agents (document perception, entity matching with the "augment-both-sides" approach, intelligent classification, payment matching, inventory control, and validation and period-closing). Implementation used the LangGraph framework, vector databases, and open-source models (LLaMA 3.1 and Gemma) running locally. The statistical population included approximately 500 real invoices and financial documents from companies listed on the Tehran Stock Exchange together with simulated data. Performance was assessed using precision, recall, F1-score, Top-3 hit rate, processing time, reduction in closing-cycle duration, reduction in human error, and audit-log analysis. For qualitative validation, the generated audit logs were evaluated by five independent auditors with an average of 15 years of experience.

Results: In operational deployment on approximately 200 accounting invoices, the proposed system achieved a Top-3 hit rate of 97%, substantially outperforming rule-based matching (76.3%) and RPA-based matching (78.9%). System precision reached 94.8% (a 20.6 percentage-point improvement over RPA). Chart of Accounts classification accuracy was 90.8%. The mean financial closing cycle decreased from 11 days to 3.6 days (a 67.3% reduction) and the human error rate fell from 12.4% to 3.9% (a 68.5% reduction). Independent auditors assigned the audit logs an overall score of 4.5 out of 5, and replayability was confirmed in 98% of cases.

Conclusion: The combination of RAG architecture and multi-agent systems based on Large Language Models provides an efficient operational solution to long-standing financial document reconciliation challenges. The system enhances the reliability of financial statements by substantially reducing time and human error and, through transparent and tamper-evident audit logs, supports a paradigm shift in the role of accountants from transaction controllers to algorithmic governors. In emerging markets, the system can serve as a strategic instrument for improving corporate governance and internal controls. Theoretically, the findings simultaneously advance the objectives of agency theory and internal control theory.
Keywords
Subjects