What is the difference between Any Order conversion funnels and closed funnels

Cameron Khozuri What are conversion funnels?

Let’s start by defining the funnel.

The conversion funnel is a method of analysis that allows you to understand exactly how users interact with the game, which way they are going.

Funnels are usually used when analyzing how many users have overcome certain conversion points (these include in-game events, for example, purchases or passing a level).

Despite the fact that the functionality for counting funnels is presented in most analytical platforms, the logic of counting depends on the technological capabilities of the platform and the results of such calculations may vary greatly.

Any Order type Funnel

As the name implies (from English, any order — any sequence), Any Order funnel is a type of funnel that is calculated:

• or based on the number of users who performed the action;
• or based on events committed in any sequence and without any conditions.

Simply put, the user who performed the actions listed in the request will be included in the calculation regardless of the sequence in which these actions were performed.

This logic, when calculating the results, does not take into account (1) the correspondence to the sequential order of actions performed or (2) the chronological order of completed actions.

As an example of Any Order funnel logic, consider a funnel request of four steps, where: “Step 1 = Event A”, “Step 2 = Event B”, “Step 3 = Event C” and “Step 4 = Event D”.

In the table above, we look at seven users who interacted with these events.

Using Any Order funnel logic, all events executed by users are included in the funnel calculation regardless of the order specified in the request.

The visual output of the funnel for this data will be the following diagram:

Based on the graph above, an Any Order funnel simply counts the number of events that correspond to the listed events in the request, provided that the user interacted with all previous events in the request, and sorts the data by step number.

Any event, regardless of the event preceding it or the sequence of event execution, will be included in the results. For example, User 2 does not match the Step 1 (Event A) query in its user path, but this user is still included in the funnel data. Next, User 5 starts with the last step of the request (Event D) and ends its path with the first step of the request (Event A), and it is also added to the calculation.

Funnels like Any Order are false funnels that mislead the developer when analyzing the performance of projects or the conversion path. The data calculated in this way does not represent a funnel of any kind, but simply displays the number of events performed by users.

The use of such logic for the construction of funnels also limits the data that can be obtained as a result of the analysis. Important KPIs for understanding the funnel of a project’s performance (such as “action completion time”, “average time to action”, or “dumps”) cannot be calculated because the events in the query are not organized in chronological order.

Closed funnel

A closed funnel is a type of funnel calculated using data only from those users who perform actions in a certain order.

This logic requires the fulfillment of two conditions at once:

• performing actions in the order that was specified in the request;
• chronological order of actions with time cuts.

Funnels with this counting logic display correct data about the conversion path of each individual user and allow you to see how long the conversion takes from one step to another.

Using exactly the same set of user data shown in the example with funnels of the Any Order type, let’s consider how closed funnels will calculate a funnel of four steps, where: “Step 1 = Event A”, “Step 2 = Event B”, “Step 3 = Event C” and “Step 4 = Event D”.

It follows from the table above that only users who start with Event A and follow the specified order of steps will be included in the funnel data.

For example, User 4 events will be qualified only partially, since the logic of a closed funnel will exclude events that do not have all the matching preceding events in the prescribed order. Instead, the system recognizes Event A as the beginning of the user’s conversion path and will start looking for the next steps corresponding to the order of the request to include them in the funnel.

In addition, User 3 will have only one event that can be included in the funnel calculation, since the beginning of the user’s conversion path corresponds to “Step 1 = Event A”, but the subsequent step is not “Step 2 = Event B”.

The visual output of the funnel for this closed funnel dataset will be as follows:

The closed funnel graph paints a completely different picture of how the user group goes through the funnel specified in the request. Excluding user events that do not correspond to the order specified by the developer allows you to find out exactly what is happening on the way of the cohort of users to Event D.

Comparison of data obtained from Any Order funnels and closed funnels

The purpose of the analytical platform is to provide developers with both tools and methods for obtaining information for effective product and business decision-making.

In mobile development, funnels are a benchmark used in making key decisions that relate to user experience, retention, monetization, and overall profitability of the project.

By resorting to Any Order funnels, analytical tools not only distort data, but also in fact sabotage the use of funnel analysis. The difference between the results of two methods using the same data set is significant and only highlights the importance of choosing a specific tool for analysis.