Find information about

Section 4 Techniques and Tools

4.08 Mystery shopping

Mystery shopping is the use of individuals trained to observe, experience and measure any customer service process, by acting as service users or customers and reporting back on their experiences in a detailed and objective way. This procedure can be used over the telephone, in face to face situations or by e-mail.

The idea is to test out the actual customer experience of services. It might be used as a free-standing exercise, to follow up an issue identified through other methods such as a satisfaction survey or after analysing recent complaints.

Telephone based mystery shopping may be well suited to covering any large, dispersed population. There may be scope to undertake this kind of approach on an on-going basis to get more regular feedback.

The exercise involves deciding on suitable scenarios – typical situations or issues that service users may present; rather like ‘frequently asked questions’. The whole quality and value of the mystery shopping process depends on the design and execution of the scenarios used to test service delivery. Experience suggests that this approach should not be too ambitious. Planned but simple approaches are likely to be most effective.

This approach raises a number of issues of ethical research practice. It is important that staff and other appropriate parties such as trade unions know that mystery shopping is planned. They should not be told exactly when and where it is to happen as this may undermine the process.

As with the use of complaints as feedback, the critical issue is the culture of the organisation and an attitude that is not about allocating blame for poor performance but to draw out wider lessons. This means that the identity of the parties is not really the point.

There should also be feedback to staff on the findings and the intended follow up actions based on using this technique.

Mystery shopping scenarios

The scenarios to be used in mystery shopping exercise should be:

• Relevant: designed to test the specific service on which data is required?

• Credible: not too ambitious, but mimicking natural service user behaviour and able to be enacted convincingly? The use of jargon or technical language will jeopardise the exercise.

• Practical: simple, brief and appropriate? Complex or unrealistic scenarios will compromise the quality of the data and the exercise and place an undue burden on staff time.

• Safe: not risking the personal safety of the mystery shoppers themselves?

• Objective: focusing on factual information? Recording what happened, rather than how the shopper feels about it in order to be consistent across all assessments made by different shoppers. However, more subjective assessments may be used to assist in interpretation of data. These may include perceived confidence of staff or overall satisfaction with the service, for example.

How to sample in mystery shopping

Mystery shopping assessments should be made on the basis of a sufficient sample of calls to be able to analyse how services are delivered differently across different locations, offices or at different times of the day, without compromising staff anonymity. The number of calls required will depend on the type of analysis required; for example the level of disaggregation across geographical areas, different services or types of inquiry. Samples should be selected to reflect patterns of service usage based on an analysis of any existing management information and may be based on quotas drawn up to reflect this pattern, so that the sample chosen is typical of the usual pattern of inquiries.

Who should conduct the mystery shopping?

The use of professional mystery shoppers employed by market research companies or consultancies may provide a degree of independence for the exercise and may be important in reassuring staff about the purpose and quality of the exercise. Some organisations might be interested in using their own staff or tenants to be mystery shoppers. People with ‘insider knowledge’ of an organisation may find it more difficult not to use jargon or to retain their independence.

There are issues about care in recruitment and training of whoever is to undertake this kind of exercise, whether they are staff, tenants or people employed by a market research company, to ensure that people understand the need for consistency of approach. Shoppers need to be trained to ensure they understand and are able to follow standardised procedures; it is important that the details of the calls made, timing and outcomes are recorded. They also need to be able to follow up the responses they receive with further questions in a fair and objective way.

The use of tenants to undertake mystery shopping is an interesting development, which may offer other advantages in terms of broader tenant participation. It does beg the question about whether it is possible to make better use of the actual experience of tenants, in effect to systematically gather information from real scenarios by actual tenants. Ways to do this might be through the use of stories, analysis of ‘significant’ events and analysis of complaints.

How to analyse and report data

The data collected through mystery shopping exercises should not be reported on an individual or case-by-case basis; the purpose of the exercise is not to identify individual instances of performance, but to examine whether there are consistent patterns in service response. This means that where the number of incidences is small, care should be taken to protect the anonymity of staff in reporting. The overall analysis should identify any requirements for further investigation of trends or issues.

The results should be reported to all key audiences in an appropriate and accessible way. This should be proportionate to the scale of the exercise and use existing media or forums. This need not take the form of a report but could use newsletters or local newspapers, posters, leaflets or presentations to tenants groups or forums. Other service users, staff, committee members and councillors should also be informed. The implications of the research for service delivery and any further research needs should be clearly identified.

Mystery shopping: checklist

√ The scenarios used should be relevant, credible, practical and safe.

√ The sample size and selection should be appropriate for the kind of analysis required and should reflect the usual pattern of inquiries across the service.

√ The objectivity of the exercise should be safeguarded by careful selection of mystery shoppers and thorough training.

√ Data should be recorded consistently and analysed objectively and appropriately.

√ Data should be reported only at an aggregate level and the anonymity of staff protected.

√ Staff and trade unions should be told that mystery shopping is planned and that they may be involved in the decision-making process.

√ The findings should be written up in an appropriate and accessible way and reported to all key audiences, including staff.

√ The implications of the research for service delivery should be identified.

√ Any further research needs should be identified.

Alternatives and related approaches:

• Using stories

• Significant event analysis

• Using comments, compliments and complaints as feedback

Previous page Next page