Skip to main content

Process Receipts with AI

YAFFA can read receipts, invoices, and email confirmations, extract the relevant information with the help of an AI provider, and create a draft transaction for you, which you can review and record. This feature can speed up the process of entering transactions into the system, but nothing is recorded without your review.

Documents can reach YAFFA in three ways:

  • you upload them manually in the application,
  • you forward an email to an address monitored by your YAFFA instance,
  • YAFFA imports them from a Google Drive folder.
warning

This is one of the most advanced features of YAFFA and requires some setup.

  • You need to bring your own API key for a supported AI provider (currently OpenAI or Google Gemini), and set up billing for your account with them.
  • YAFFA tries to clean up received emails from styling and other unnecessary information before communicating with the AI provider, but the content of the email or document is still sent to them.
  • Experiment with this feature and keep an eye on your usage to avoid unexpected costs.
  • Receiving emails requires an account with one of the supported email providers, and full control over the domain of the email address you want to use, to set up the required DNS records.

AI provider​

The AI provider is configured by each user, in the AI settings of the application (not in the .env file). Select the provider and the model, enter your API key, and use the connection test to verify it. In the same place, you can turn the feature on or off, and fine-tune how documents are processed, for example the language used for OCR and how duplicates are detected.

Upload limits​

Manual uploads can be limited with the following optional settings in the .env file. The values below are the defaults.

AI_DOCUMENT_MAX_FILES_PER_SUBMISSION=3
AI_DOCUMENT_MAX_FILE_SIZE_MB=20
AI_DOCUMENT_ALLOWED_TYPES=pdf,jpg,jpeg,png,txt

Images and scanned documents​

Text-based PDFs, plain text files, and emails are read directly. To process images, YAFFA needs a way to read the text of them: either the Tesseract OCR engine, or a vision-capable AI model of your provider. If neither is available, processing of the document fails, and you are notified.

Tesseract is optional. You can either install it on the server (binary mode), or, in the Docker setup, use the Tesseract service from the Docker Compose file (HTTP mode: uncomment the tesseract service and the related depends_on lines in the file).

TESSERACT_ENABLED=TRUE
# 'binary' (local executable) or 'http' (Docker sidecar)
TESSERACT_MODE=binary
TESSERACT_PATH=/usr/bin/tesseract
# Used only in HTTP mode. In Docker, use the name of the service
TESSERACT_HTTP_HOST=localhost
TESSERACT_HTTP_PORT=8888
TESSERACT_HTTP_TIMEOUT=30

Receive receipts in email​

This guide below is an example of how to set this up with SendGrid. You can use any other supported email provider, but the setup might differ slightly.

  1. Register an account with SendGrid. You can use the free tier for testing.
  2. Set up an MX record for your domain to point to SendGrid. This is necessary for receiving emails. See SendGrid's guide on setting up an MX record for the detailed steps.
  3. Head over to SendGrid at https://app.sendgrid.com/settings/parse and add a new record matching your domain.
  • The receiving domain should be your domain, e.g., your-yaffa-domain.com
  • The destination URL should be https://laravel-mailbox:yourpassword@your-yaffa-domain.com/mailbox/sendgrid
    • The URL should be accessible from the internet, so make sure your YAFFA server is reachable from the internet.
    • The password is a secret you can choose, and you will need to set it in the .env file of YAFFA.
  • Make sure to check the "POST the raw, full MIME message" option.
  1. In your YAFFA .env file, set the following settings:
MAILBOX_DRIVER=sendgrid
MAILBOX_HTTP_PASSWORD=yourpassword
# The email address where incoming receipts are sent, and which is monitored by the Mailbox service
# YAFFA will process emails sent to THIS address from YOUR EMAIL ADDRESS registered in YAFFA
INCOMING_RECEIPTS_EMAIL=
  1. Configure your AI provider in the AI settings of YAFFA, as described above.
  2. Reload the configuration of your deployment using the appropriate Artisan command.
php artisan config:cache

You can now send an email with a receipt to the email address you have set up in the .env file. YAFFA will process the email and create a draft transaction for you to review. The email should be sent from the email address you have registered in YAFFA. In response, you will receive an email with the draft transaction and a link to review and record it in YAFFA.

Import documents from Google Drive​

YAFFA can monitor a Google Drive folder, and import the files you drop into it. The connection uses a Google service account, which you configure in the AI settings:

  1. In the Google Cloud console, create a project, enable the Google Drive API, and create a service account with a JSON key.
  2. Share the Drive folder with the email address of the service account (the client_email value in the key file).
  3. In the Google Drive section of the AI settings, upload the JSON key, select the folder, and test the connection.
  4. Choose how often the folder is checked. You can also start a synchronization manually.

You can also select what happens with a file after a successful import: delete it, move it to the trash, move it to a "processed" folder, or rename it with a processed_ prefix. Deleting or trashing only works for files owned by the service account, so the last two options are the reliable choices for a folder shared from a personal Google account.

Document retention​

Processed documents and their files can accumulate over time. Each user can set a retention period in the "Document Retention" section of their AI settings, to have YAFFA clean up automatically:

  • Finalized documents older than the set number of days (between 1 and 3650) are deleted daily, together with their stored files and received emails.
  • The transaction created from a document is kept. It only loses the link to its source document. This is also true when you delete a document manually.
  • Documents that are not finalized are never deleted automatically. Instead, the user receives one reminder email per day, linking to the filtered document list, until the documents are finalized or deleted.
  • Leaving the setting blank (the default) keeps everything.

The cleanup is a scheduled task that needs a running queue worker, and the reminders need working outgoing email. You can also run it on demand with the "Old AI document cleanup" action of the maintenance page, which is disabled until a retention period is set.

caution

Deleted documents cannot be restored. Back up your database and storage before enabling the retention period on an existing instance.

Google Drive imports without a post-import action

If no post-import action is set, the imported files stay in the monitored folder. After their documents are deleted by the cleanup, a manual full synchronization can import those files again as duplicates. YAFFA shows a warning when you enable a retention period in such a setup. Configure a post-import action (move or rename the processed files) first.