Get hands on with the Azure Personalizer API

In my last article, (https://www.garudax.id/pulse/make-better-recommendations-reinforcement-learning-azure-jake-wang/), I described the benefits of using Reinforcement Learning for creating recommendation models. Basically, Reinforcement Learning Models continually update based on new user preferences and make creation/maintenance of the recommendation engine much easier to manage. In addition, I described the Azure Personalizer API service which brings you the power of Reinforcement Learning with very simple interface. I typically find that the best way to get started with something is to run through a live demonstration. Seeing and feeling how something works just makes things click for me, as opposed to reading diagrams and abstract descriptions. So in this article we will go step by step through a live demonstration of Azure Personalizer API, which you can follow along with.

Scenario

Imagine you are running a news website that serves the most relevant articles available online. Your website only has room for displaying a couple featured articles in the main page above the fold. In order to engage your readers and ensure they keep coming back to your site, you decide to employ a recommendation engine to serve the most relevant articles to each user. You can't just display all new articles since there isn't enough room on the page. Likewise, you cannot just pick random articles because they may not align with the interests of the readers. So our best course of action is to use some sort of intelligence to recommend articles to each user for their reading pleasure.

Start by going to the link below:

This is the screen you should be greeted with once you follow the link above. There's alot going on here so let's break it down.

The Request

Context

In our scenario, we begin with the user accessing our website. When the user makes their first request for our site, we only have a few bits of information. For this demo, we don't have any user info, but, we do know:

• The day of the week
• Time of day
• Weather

Also, in most cases we have access to the information provided by the user:

• Browser Type
• Device Type
• Location (IP Address)

You can see this information displayed in the Request - Context section of the page.

Actions

In addition to the information we have about the user and context, we also have a list of available articles to display. The articles themselves come with some additional information, we call features:

• Recency - Published Date
• Breaking News flag
• News Source

You can see this information displayed in the Request - Actions section of the page.

The Recommendation

Now that we have all the context information along with the available actions for the reader, we can send all the information to the Recommendation Engine, the Azure Personalizer API, to retrieve a recommended article for our reader.

Go ahead and click the "Show a Personalized Article" button.

Now you should see an article displayed, which was recommended by the service:

In addition to the suggested article, you will also see a Response section which shows the JSON data of what the service returned.

{
  "result": {
    "ranking": [
      {
        "id": "downer-deploys-microsoft",
        "probability": 0.664000034
      },
      {
        "id": "ai-for-earth",
        "probability": 0.084
      },
      {
        "id": "microsoft-partners-with-openclassrooms",
        "probability": 0.084
      },
      {
        "id": "microsoft-ai-chief",
        "probability": 0.084
      },
      {
        "id": "microsoft-executives-meet",
        "probability": 0.084
      }
    ],
    "eventId": "e7f97e3d-9ea1-4246-8863-7a470f438279",
    "rewardActionId": "downer-deploys-microsoft"
  }
}

This result shows that based off of the user context and available actions, the recommendation gave the "downer-deploys-microsoft" article the highest probability of a match. So to recap, when we clicked the "Show a Personalized Article" button, we submit Request containing the Context along with the Actions and we received a Response with our Result of all the actions along with match probabilities.

Rewarding the Model

In addition to asking for suggestions from the Azure Personalizer Service, in order to train the model for better suggestions, you can reward the model by sending back the response to the suggestion it made. In our scenario, the reward function for the recommendation can be defined as whether or not the user reads the article. In the demo we reward the amount of article read, such that if a user finishes reading the article the reward is higher than if the user only reads the first few paragraphs and moves on.

The rewards are indicated in the reward section:

The demo defines a reward period (15 seconds) where a reward (percentage of article read) will be sent back to the model once the time period elapses.

This reward will inform the model of how the Context, Actions, and Result, all interact and how accurate the suggestion was in order to improve and provide better results for the next interaction. Using this feedback function provides the training for the Reinforcement Learning model and enables the model to stay up-to-date without any additional training operations.

Note: The eventid is used to to tie this reward back to the initial result.

Exploration vs Exploitation

The last portion of this demo relates to the exploration vs. exploitation topic in Reinforcement Learning. Exploration, is when the model sends a suggestion that would not necessarily be the highest probability of match. This is done so that the model can stay up to date with new trends and discover new patterns that will ultimately lead to better recommendations. Exploitation (Using Learned Model), is when the model provides the highest probability of match by using the learned patterns in the model. This is the typical use of the model, however in Reinforcement Learning, exploration is a necessary operation for avoiding model stagnation which would result in stale recommendations.

Summary

Hopefully, this walkthrough of the Personalizer API Demo website, provides a good explanation of how the service works in a working scenario. Feel free to change the context settings and see how the changes affect the recommended news articles. In addition, if you are interested in testing out the Azure Personalizer API with your own use case, you can provision your own instance in the Azure portal (portal.azure.com). In my next post, I will dive a little deeper into the Personalizer API from the portal interface.