Using a Power Apps app with AI Builder models to Capture a Vendor Invoice

In my previous posts, I used AI Builder in Power Apps to set up models for form processing and category classification respectively. In this post I will show the app, I have built with Power Apps that brings those two capabilities together. In the next post, I will dissect the app to show how to leverage the AI Builder models inside an app.

In the last post of this series, we will take a quick look at how I use Power Automate to submit the invoice to Dynamics 365 Finance.

As you can see from the screenshots, my use case has a slightly agricultural flavour.

Screen 1: Landing Page

When I start the app, I am presented with a landing page. As you can see, the app is quite simple, but obviously the landing page allows me to add more items to the menu, if required.

From the landing page, I select Process Invoice.

Screen 2: Select Cost Centre

In my example, I am asking the user to select the cost centre where the cost on the invoice needs to go. In this case I select John Deere Tractor 5100R and click on the <NEXT> icon on the line.

Screen 3: Main Invoice Processing Screen

From this screen the user is able to see the selected cost centre and the captured invoice image before submitting it to Dynamics 365 Finance. However, first, the user must capture the invoice by clicking on the camera icon.

Screen 4: Capture Invoice Screen

I now come into the screen where I can capture the invoice. This is done by clicking on the Capture Invoice button. Since I am doing this from an iPhone, I am asked to take a photo or select an existing photo from the library as shown below.

Screen 5: Select Photo Source

In this case I take the following photo of the invoice.

Screen 6: Invoice Image

When I click on Choose, the image is submitted to the AI Builder form processor for interpretation. The result is shown below.

Screen 7: Invoice Interpretation Result

As you can see, the image is brought back into the app and the form processor has interpreted the invoice. The image has labels applied that show the captured fields and their accuracy. In the lower section of the screen, I am using this information to fill out the variables I need to submit the invoice to Dynamics 365 Finance.

Since I am using an English form processing mode, the description has been interpreted as “110mm ror” instead of “110mm rør”. After the invoice has been interpreted, I take the description and process is through the category classification model in built earlier. This identifies the description to belong to the category “ror” and this category is tagged to the main account “2020”.

Since the invoice has been interpreted correctly, I can click on the accept icon to go back.

Screen 8: Submit Invoice to Dynamics 365 Finance

Coming back to the main processing screen, the captured image is now displayed. Also, the save icon becomes available and when I click on it, the invoice is submitted to Dynamics 365 Finance.

Once the Power Automate flow has finished processing, the user is told the invoice has been processed.

Obviously, in the real world I should have trained the model to read multiple lines into a table. However, I hope this simple example has shown the power of using AI Builder in apps.

In the next post, I will take a look at the mechanics of this app. Stay tuned…

Published by Henrik Marx Larsen

This blog is my personal contribution to the Microsoft Dynamics 365 for Operations and Finance (D365F&O) community debate. Having worked with Axapta / Dynamics AX / D365F&O for 20 years , I think I may have some interesting insights and views to share. However, I tend to always look forward so I will mostly be writing about current subjects. I have a vain hope that my musings may inspire some of you and provoke some new thoughts in the interest of our community. My day job is to work as a Senior Technical Specialist at Microsoft and help take D365F&O capabilities to new and existing customers. Please note, the views expressed in this blog are mine alone and do not necessarily reflect the views of my employer, Microsoft.

2 thoughts on “Using a Power Apps app with AI Builder models to Capture a Vendor Invoice

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: