Measuring Addictive Digital Entertainment User Engagement Scales
"engagement is a multidimensional construct which is subject to a context-specific expression of relevant cognitive, emotional, and behavioural dimensions."
Problem definition
When looking at ways to benchmark our digital products we are mostly following frameworks that are technology and product agnostic such as the Net Promoter Score and System Usability Scale.
While useful, this approach might have its limitations when testing products that require the user to enter a Flow state. To deliver measurable iterative improvements, the product needs to be observed from different angles.
One suggestion is to explore multiple data dimensions to get more product-specific insights. For this document, I will be limiting the scope to online betting products (classified as addictive digital entertainment).
Available Tools
Net Promoter Score (NPS)
How likely is it that you would recommend the product to a friend or colleague? The NPS is typically interpreted and used as an indicator of customer loyalty.
Questionaire load: 2 Questions
System Usability Scale (SUS)
A technology-agnostic way to measure the usability of products.
Questionaire load: 10 Questions
Suggested Scales:
User Engagement Scale - Short Form (UES-SF)
Measures self-reported user engagement. Purposed to measure six dimensions of engagement: aesthetic appeal, focused attention, novelty, perceived usability, felt involvement, and endurability. This scale had its psychometric quality validated. Developed and tested with Western adult populations.
Questionaire load: 12 Questions
See Appendix B for scoring details: https://www.sciencedirect.com/science/article/pii/S1071581918300041
Flow State Scale (FSS)
Measure flow experiences during physical activity. This scale had its psychometric quality validated. Dimensions:
Challenge-skill balance;
Action-awareness merging;
Clear goals;
Unambiguous feedback;
Concentration on the task at hand;
Sense of control;
Loss of self-consciousness;
Time transformation;
Autotelic experience (Jackson et al., 2008).
Questionaire load: 14 Questions
Scale details: https://www.sciencedirect.com/science/article/pii/S1569186113000442
Game-experience Questionnaire (GEQ)
The questionnaire consists of different modules:
Core module - actual experiences during gameplay (33 + 14 in-game Questions)
Social presence module - gaming with others (17 Questions)
Post-game module - experiences once a player has stopped gaming. (17 Questions)
Captures the game experience based on several items (such as positive affect, competence, immersion, flow, challenge); Captures also the playing experience when playing with others, as well as the post-playing experience;
Some items are difficult to fill in by participants when they only have a short time available to play the game (e.g. in lab settings).
Scoring details: https://pure.tue.nl/ws/files/21666907/Game_Experience_Questionnaire_English.pdf
References & further reading:
https://pure.tue.nl/ws/files/21666907/Game_Experience_Questionnaire_English.pdf
https://www.sciencedirect.com/science/article/pii/S1071581918300041
https://www.sciencedirect.com/science/article/abs/pii/S0747563213004433
https://www.hindawi.com/journals/ijcgt/2017/7363925/