In a previous role, I built a fully automated Intelligent Document Processing (IDP) system on Amazon Web Services (AWS). The system enabled Irish citizens to submit Freedom of Information–style requests to check whether their personal details (name, address, etc.) were stored in a large insurance claims database maintained by my employer.
High-Level Workflow
The process followed this schedule:
- Client solicitors sent an email to a specific Outlook address requesting a database search. The email had to include the client’s first name, last name, date of birth, and current address.
- A search could only proceed with corroborating documents attached to the email to verify the client’s identity.
- Attachments were typically PDFs containing scanned images of passports, driving licences, or other official documents. We also received MS Word documents or PDFs of bank statements and utility bills.
- Once daily, the IDP system processed these emails: it extracted and classified attachments, then extracted any relevant Personally Identifiable Information (PII) and sent it to the claims database for a match.
- If claim details were found, the system generated a password-protected PDF with those details and emailed it to the solicitor, along with a separate email containing the unlock password. If no results were found, the solicitor received a simple notification email.
The process was fully automated except for a Human-In-The-Loop (HITL) stage, where users performed a final cross‑check to confirm that the PII extracted from different documents matched.
This new system proved approximately 90% more efficient than the manual process it replaced.
In this article, I’ll walk through the steps to implement a slightly simplified version of that system on AWS.
Note: I am a user of AWS but have no affiliation or association with the company.
The original system used a corporate Outlook email account. Since I no longer have access to one, I’ll use my personal Gmail as the email server for this demo. I’ll assume each email contains only one attachment, which is always an image file (JPG or PNG) of a passport, driving licence, or bank statement. The final output of the process will be a results file stored on Amazon S3, showing the document classification and extracted PII values.
Due to the length of the code snippets, a link to the GitHub repository is provided at the end of this article. There you’ll find all Lambda functions, Step Functions definitions, IAM permissions, and everything else required for this project.
Prerequisites: Email Integration
This article focuses on the AWS side, so I’ll assume your email client is already set up to allow an AWS Lambda function to read your emails.
For Gmail, the setup involves:
- Creating or selecting a Google Cloud project
- Enabling the Gmail API
- Enabling OAuth and completing a one‑time Google consent flow
After completing these steps, store the returned credentials and refresh tokens securely in a service like AWS Secrets Manager.
Goal: Extract PII from Attachments
From each email attachment, the system extracts the following PII fields:
Document Type, First Name, Last Name, Date of Birth, and Address
Our Test Gmail Attachment Images
I asked Codex to create three obviously fake images: a passport, a driver’s licence, and a bank statement. Although fake, they contain the same types of information as real documents. Below are those images.
(In a real deployment, you would use actual scanned documents or high‑quality synthetic data for validation.)