Process Evaluation of the Player Awareness System pilot.
The Player Awareness System (PAS) pilot involving six retail operators was evaluated in 2016 through an independent analysis commissioned by GambleAware. This was a process evaluation, conducted by PriceWaterhouseCoopers (PwC) through large data-set analysis across the participating operators. The findings led to substantial improvements in PAS algorithms, implementation and monitoring, and a commitment to a further evaluation to assess improved effectiveness in harm-minimisation.
What was the evaluation about?
Research for GambleAware in 2014 encouraged the industry to identify markers of harmful gambling in order to intervene with players at risk. Using this and other research, the Association of British Bookmakers (ABB) and members developed Player Awareness Systems (PAS). These use behavioural indicators to distinguish between problem and non-problem gambling, by people using accounts to play gaming machines in licensed betting offices. The aim of PAS is to intervene to prevent customers becoming problem gamblers, by spotting those who are on a trajectory towards harmful play and intervening with messages in order to make the player aware of their own behaviour and to halt and reverse that trend. This innovative initiative directly involved Coral, Ladbrokes, Paddy Power, William Hill and machine manufacturers Scientific Gaming and Inspired Gaming (on behalf of independent bookmakers), with all ABB member signed up to a wider roll-out.
When was it done, who by?
To ensure its independence, the management of the evaluation was undertaken by GambleAware and externally commissioned following an open tendering process. The contract was awarded to PwC who acknowledged its existing auditing relationship with Ladbrokes (one of the pilot operators). This was not regarded by the GambleAware selection panel as a conflict of interest. The evaluation commenced in March 2016.
How was it done?
PAS implementation was at an early stage, with the pilot scheme put in place early in 2016. This involved each operator developing different algorithms and ways of intervening. Process evaluation was needed to review early implementation, compare effectiveness across the different PAS processes, identify improvement needs and provide insight for subsequent development. This was a multi-method evaluation combining an intensive quantitative analysis with operator interviews, specifically:
This early evaluation was limited to customers whose sessions could be tracked through loyalty cards. The evaluation did not review costs or impacts – both of which were seen as premature given the pilot nature of the scheme.
What worked well?
The evaluation was conducted across multiple systems with different characteristics. This comparative aspect of the evaluation provided a rich source of evidence of which aspects of operator arrangements worked better or less well. Undertaking a process evaluation early in the implementation helped identify how systems could be improved. The PwC team reported to a GambleAware steering group which included an external evaluation expert and provided a direct link to operators to ensure access to data and sharing of experience.
What lessons were learnt?
The process evaluation showed considerable variety of algorithms used by the ABB’s member companies, and differences in maturity. The findings demonstrated PAS was work in progress but showed potential as a systems-based approach to responsible gambling. It also showed scope for closer integration of Fixed Odds Betting Terminal (FOBT) PAS data with data from other operator betting activities, more use of insights from behavioural psychology, and tighter controls for PAS messaging and customer interactions. The process evaluation confirmed it remained premature to look at impacts on player behaviours linked to PAS and suggested a further process review when improvements had been made. This would also assess the most viable focus and timing for a subsequent impact evaluation – to see how PAS encourages customers to think about how they are gambling and influences in changed behaviour.
GambleAware has published the full process evaluation here.