A solution to Loan Application Drop off Problem with RAG AI

📊 Executive Summary

This blog explores the development of an AI assisted expense calculator designed to support loan applicants in preparing their documentation, a common pain point in the mortgage process.

As the cost of living continues to rise, many Australians are seeking to refinance their mortgages. In this climate, lenders that offer streamlined, informative, and user friendly application processes will have a competitive advantage. One of the most challenging steps for customers is accurately completing monthly expense forms. This tool addresses that challenge by using a Retrieval Augmented Generation (RAG) chatbot to guide users through the process, improving both the customer experience and the quality of submissions received by banks.

The working prototype leverages RAG to tailor responses to the user's context while applying strict guardrails to keep conversations on topic and free from personal data handling.

This post walks through the real world business problem, the thinking behind the solution, and how it was translated into a functioning AI powered prototype offering insights for technologists, product designers, and fintech professionals alike.

🏦 The Business Problem

This solution is aimed at prospective home buyers and refinancing applicants navigating the often complex mortgage application process.

One of the most frustrating aspects for applicants is completing the monthly expense section of the loan application. A key pain point is not knowing how to categorise specific expenses for example, whether health insurance should be listed under insurance, medical, or another category. These small uncertainties create significant friction.

When unsure, applicants are faced with two poor choices: guess (risking errors that could hurt their chances of loan approval), or stop and seek clarification. In many cases, this leads to abandoned applications either temporarily while they try to get help, or permanently as they switch to another lender with a simpler or clearer process.

Lenders lose out when this happens. Reducing friction at this stage not only improves the customer experience but also increases application completion rates.

By introducing an AI powered chatbot that can answer simple, specific questions in real time such as "Which category should I put health insurance in?" we can address a clear user need and reduce application drop off.

🔄 Translating the Problem into a Technical Solution

At its core, the business problem is about reducing application abandonment by ensuring prospective borrowers have support when they get stuck particularly when it comes to categorising their expenses. Right now, many applicants drop off simply because they don't have anyone to answer simple questions in real time.

The proposed solution is to introduce a chatbot that acts as a digital assistant during the application process. This addresses the core issue: no human available, no progress made.

However, from a business perspective, any AI assistant must meet strict requirements:

• 🔒 It must not collect or process personal identifiable information (PII)
• 🎯 It must stay strictly within the scope of expense guidance
• ✅ It must provide accurate, relevant, and up to date information

To meet these needs, the technical solution takes the form of a Retrieval Augmented Generation (RAG) chatbot, backed by a curated knowledge base containing definitions, examples, edge cases, FAQs, and guidance for expense categories.

RAG enables the chatbot to provide responses that are tailored and context aware, while still grounded in a verified document base. To ensure compliance and relevance, guardrails are implemented to keep the conversation focused, prevent unsafe queries, and maintain a high standard of accuracy and privacy.

By aligning technical design with business expectations, the solution supports both the end user and the organisation enhancing the customer experience while maintaining compliance and control.

💻 The Technical Solution

To bring the concept to life, I developed a working prototype of a Retrieval Augmented Generation (RAG) chatbot designed to assist with expense related queries during the loan application process. The implementation follows a modular design, integrating open source tools and AWS services.

🏗️ Architecture Overview

The system is built using a modular architecture that separates the concerns of the frontend and backend:

• 🌐 Frontend: A clean, responsive web interface prototype, where users interact with the chatbot.
• ⚙️ Backend: A Python based API service that receives user inputs, connects to AWS Bedrock to generate responses, and enforces guardrails to ensure safe and relevant interactions.

📚 Knowledge Base (RAG)

At the heart of the chatbot is a Retrieval Augmented Generation (RAG) pipeline that improves response accuracy by grounding the language model's outputs in a structured knowledge base.

To support this, I created a synthetic dataset tailored to mortgage related expense inquiries. The dataset was designed to reflect realistic user questions and edge cases while remaining free of personal or proprietary data. It includes:

• 📋 Expense category definitions
• 📝 Examples, edge cases, and FAQs

🛡️ Guardrails and Safety

To ensure compliance and maintain user trust, several safety layers were implemented:

• 🛑 No PII Handling: The chatbot is configured to reject or ignore inputs containing personal identifiable information.
• 🎯 Scoped Responses: The assistant is restricted to questions specifically related to expense calculations. Off topic queries are gently deflected.
• 📝 Prompt Engineering: A well defined system prompt keeps the model focused, avoids speculation, and ensures it sticks to verified knowledge.

🤖 LLM & API Integration

The chatbot leverages Claude via AWS Bedrock to generate responses:

• ☁️ Model Provider: AWS Bedrock (Claude by Anthropic)
• 🔌 API Access: Managed using the boto3 library in Python
• 🔄 Prompt Flow: Each user query is enriched with context retrieved via LangChain and passed to the LLM along with guardrails and system instructions

🧩 Challenges and Lessons Learned

Key Challenges

Building this prototype surfaced several challenges both technical and strategic that required careful trade offs and rapid decision making:

🚀 Future Enhancements

While the prototype demonstrates a working concept, several enhancements are planned to improve usability, robustness, and business value:

💭 Reflections

Building this prototype was a valuable exercise in balancing technical feasibility with user experience. It reinforced the importance of focusing on the core problem, getting to a working prototype quickly, and learning from real interactions.

I also gained a deeper appreciation for the nuances of retrieval-augmented generation especially how much the quality of the knowledge base impacts the usefulness of the model. Finally, it highlighted how guardrails and responsible AI practices aren't just technical concerns, but key to building trust with users.