A Closer Look at How I Use AI Builder Models inside a Power Apps App

Previously, we looked at how to create and train AI models using AI Builder in Power Apps in these three posts:

In this post we will take a look at the mechanics behind how the AI models are used in the app.

As you may recall from my previous post, the user was guided to a screen where he or she could take a picture of an invoice for processing. The following screenshot shows this screen in edit mode in the Power Apps editor.

Figure 1: Upload Invoice Screen in Power Apps Editor

As you can see, at the top of the screen, I have inserted a Form processor AI Builder control. This is done simply by going to the Insert menu in the ribbon and selecting Form processor from the AI Builder drop-down. You are then presented with a selection box where you can select from existing AI Builder models (or create a new one).

In the properties pane on the right-hand side, you can see the various options available for the control. This is all we need to do to embed the form processor AI Builder model we created earlier into the app.

Now we need to react to the result once the user has uploaded an invoice image and the image has been processed. We do this by writing some code in the OnChange method.

Before we start looking at the code, here is some important information:

  1. The form processor control is named AIInvoice.
  2. You refer to field on the invoice using AIInvoice.Fields.[Field].
  3. Instead of using the category classification AI Builder model I created earlier, I am actually using the underlying Azure cognitive service directly because when I built the app, it was not possible to access the model programmatically.
  4. The Azure text analysis service has been added to my data sources and is named TextAnalytics.
  5. In CDS I have created a table called TextToAccounts when I match key phrases with ledger accounts.

Now, let’s take a look at the code in the OnChange method:

Set(PostingText,AIInvoice.Fields.Description);

// This statement sets the value of the PostingText variable to the description found on the invoice.

ClearCollect(phrasesCollect,TextAnalytics.KeyPhrases(
{
language: “en”,
text: PostingText
}).keyPhrases);

// This statement creates a new collection and collects key phrases from the invoice description using text analysis.

Set(AccountTag,First(phrasesCollect).Value);

Set(CostAccount,LookUp(TextToAccounts,Text = AccountTag).Account);

// The first statement sets the AccountTag variable to the first tag found in the collection. Based on this tag, the CostAccount variable is set to the ledger account value found by looking up in the TextToAccounts table in Common Data Service (CDS).

ClearCollect(PhotoTaken,AIInvoice.OriginalImage);

// The captured image is stored in a new collection.

Set(VendAccount,AIInvoice.Fields.Vendor)

// The VendAccount variable is set to the captured vendor account from the invoice.

As you can read from this short article, it is now possible (in preview) to use an AI Builder model programmatically within a Power App. This means that instead of using the Azure text analysis service, I would have been able to use a model built in AI Builder instead. A topic for a future blog post…

Now we have combined the form processor results with the text analysis service from Azure to provide us with the information we need to submit the invoice to Dynamics 365 Finance.

We will look at how this is done using Power Automate in the next and final post in this series.

Published by Henrik Marx Larsen

This blog is my personal contribution to the Microsoft Dynamics 365 for Operations and Finance (D365F&O) community debate. Having worked with Axapta / Dynamics AX / D365F&O for 20 years , I think I may have some interesting insights and views to share. However, I tend to always look forward so I will mostly be writing about current subjects. I have a vain hope that my musings may inspire some of you and provoke some new thoughts in the interest of our community. My day job is to work as a Senior Technical Specialist at Microsoft and help take D365F&O capabilities to new and existing customers. Please note, the views expressed in this blog are mine alone and do not necessarily reflect the views of my employer, Microsoft.

One thought on “A Closer Look at How I Use AI Builder Models inside a Power Apps App

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: