Measuring Meaningful Engagement

As alternatives in banking emerge, banks may find it difficult to prove their value to customers. Why should the customer dedicate time to…

As alternatives in banking emerge, banks may find it difficult to prove their value to customers. Why should the customer dedicate time to their bank when they could just as well give it to Snapchat or Apple via their banking apps? We at Meniga believe the key to maintaining an edge over alternative banking services is to build the banking experience on meaningful engagement by building positive habits and catering to long-term relationships with customers. In our experience, many banks are willing to explore this path but, when the time comes, lack the tools to measure their success.

Building Positive Habits is a part of catering for Meaningful Engagement
Histogram of our Engagement Index. User count on the X-axis

As a general rule, we will want the Engagement Index to increase over time. That is why we have designed it in such a way that there is room for improvement. Furthermore, since the Engagement Index is calculated down to the person level, we can compare the index between user segments, A/B groups or use it to cluster people during deeper analysis.

Let’s look at how we at Meniga have used it to gain insights into our user base and then how each of the six qualities is measured.

Challenging Yourself Increases Your Engagement

One of the things we have used the Engagement Index for is to compare engagement between user segments. Take for example a part of our challenges analysis. Meniga’s challenges are a way for users to challenge their spending patterns in order to save here and there and use the savings elsewhere. We segmented people based on their challenge activity and above, we see the Engagement Index between the segments. In the first segment are the people that have at no point accepted a challenge and in the second segment are the people that have accepted at least one challenge.

Using those segments, we see that people that accept challenges are significantly more engaged with the app than people that don’t. They are more invested to engage with Meniga and, thereby, their finances.

This is but an example of how the Engagement Index enables us to compare different groups of people. But how is the Engagement Index calculated?

The Sub-Indices

As mentioned above, each of the qualities of an engaged person is measured by a separate index, each on the scale of 0 to 10. The first three indices are similar in the way that they rely only on when the person uses the app, and not on what they do. However, the later three indices rely only on how people behave when using the app. The Engagement Index is then calculated as the average of those sub-indices.

First of the indices is the Loyalty index which is calculated from the number of sessions per week and is given by the equation

Equation for the Loyalty index

So, a person that only uses the app once a week has Li = 0, a person that uses it twice has Li = 5, and a person that has used the app seven times in a week has Li = 6.7. As stated above, we feel that a person that is active at least twice a week is likely to understand the value of frequently checking the status of their accounts, be actively challenging their financial habits or using CLO on a regular basis. This person is therefore considerably more engaged with the product than people that have only been active once in the last seven days.

Similarly to the Loyalty index, the Recency index measures usage frequency. The Recency index is a function of the time since last time the person used the app and is given by the equation

Equation for the Recency index

By this formula, a person that used the app yesterday and again today has Ri = 10, but a person that used the app five days ago has Ri = 2. The Recency index measures a slightly different type of frequency than the Loyalty index as one can have high Loyalty but low Recency for a given day and vice versa.

Those two indexes work very well to measure the activity of people that use the app at least weekly. Those measurements indicate a type of habit. However, not all Meniga users do that. We have a large group of people that only use Meniga for reviewing the year and budgeting the next, for calculating their spending in certain categories each month or for some other reason only need Meniga every now and then, but periodically. Those people have developed a long-term relationship with Meniga. This is why we introduce the Consistency index. The Consistency index is measured by how many preceding months, quarters or years in a row people have been active. To calculate this, we need historical data as far back as possible. The index is calculated thus

Equation for the Consistency index

where time units can be months, quarters or years. A person that has used the app twice a month for the last three months therefore has COi = 6.7, the same as a person that has used the app only once a year for three consecutive years. This index is starts at 0 for all new users and is likely to become higher as the person ages, if the person uses the app every month, quarter or year.

We have looked at how we measure the times at which people are active. We will now continue to the indices that measure how the people are active. First of those is the Interaction index. The measurement of the Interaction index is based on the assumption that in order to perform some actions, the person needs to be in a relatively engaged mindset and therefore doing those actions is a token of engagement or intention. Following is the equation.

Equation for the Interaction index

where an interactive session is any session where the person performs an action considered interactive. The definition of interactive actions varies greatly between apps and scenarios have to be considered specifically for each. For the Meniga app, we have identified over twenty actions to be interactive, ranging from recategorising transactions to starting a spending challenge or activating an offer. Many of the interactive actions are a part of our efforts to build positive habits, but they are not actions like opening the app, scrolling through or other trivial actions. For Twitter, examples of interactive actions might be tweeting, retweeting and searching for an account. Using those examples we see that person that opened Meniga four times in the last week and in one session they accepted some challenges and in two of the other session they recategorised transactions has Ii = 7.5 while a Twitter user that opened Twitter five times in the last week and only in one of the sessions they did anything else than scroll through their feed to kill time has Ii = 2.

In the next two indexes, the Click Count index and the Duration index, we use a similar approach as the Interaction index. We count the sessions where a certain criteria is met and divide it by the total number of sessions. For the Click Count index the criteria is met when the person does a certain amount of clicks or taps, and for the Duration index the criteria is for a sessions length to be of certain length. Those are based on the idea that a person that often clicks deep into an app or stays long is likely to be there for a reason and thereby be an engaged person. The equations for the Click Count index and the Duration index are

Equation for the Click Count index


Equation for the Duration index

Those three indexes, the Interaction, Click Count and Duration indexes, measure similar types of engagement but it is easy to imagine a scenario where a person only scores high on one or two of them and not on the others. That is why we feel none of the indexes is sufficient alone but each contributes their own flavour to the whole.

Joining the Dots

Now that we have calculated all the indices for each person, combining them into the Engagement index is a simple question of averages. Since each index is on the scale of 0 to 10, the Engagement index will be on the scale of 0 to 10 as well, no matter if we use weighted averages or simple averages. This is up to each organisation to choose, along with the thresholds for the Click Count index and Duration index and the definition of interactive actions.

With all of this intact we have created a new indicator for our analysis arsenal. We should be well equipped to take on the challenge of meaningful engagement and measure the impact.