Find information about

Section 4 Techniques and Tools

4.06 Service specific, recent contact or exit surveys

These are surveys based on people who have recently used a service, such as the repairs service. They are appropriate when it is more important to have informed feedback based on experience, rather than the views of a wider sample of the general population or all service users. They can be used in a more regular and routine way than general satisfaction surveys which only happen every few years.

Such surveys can be used for different purposes and this will affect they way they should be conducted. Data on experience of services may be used to support long term monitoring of services, assess the performance of contractors and to identify and follow up individual dissatisfied service users. These approaches are good for getting short and simple, more ‘everyday’ feedback.

Service specific surveys have many advantages. They allow for targeted and focused feedback, which can frequently be designed and managed in-house. They run a risk of becoming so routine that it is unclear exactly what their purpose is and of over-collecting data that is not analysed or used fully. Where this is the case, there is a risk of cynicism and ‘survey fatigue’ amongst staff and service users. It will be important to keep them under regular review.

Who should conduct service specific surveys?

Service specific surveys are likely to be of wide relevance across an organisation, rather than being seen as the preserve of the research or consultation team, so for example, property maintenance or environmental services teams are likely to be interested in using these kinds of approaches.

Experience suggests that such surveys are less likely to be conducted by an external contractor, however, this might be an appropriate option. Queens Cross Housing Association has used a range of survey approaches which it has managed in-house. An issue to consider is the capacity to analyse all the data collected on a regular basis in order to make full use of the information gathered.

Practice Point

• It is important to define the target population very clearly; it is particularly important to ensure that questions are asked of people with recent experience of the use of the service which is being investigated.

Asking the right questions of the right people

Much of the guidance on general satisfaction surveys is relevant here too. It will often not be possible or even necessary to draw a statistically representative random sample. The most likely way to do this would be draw a random sample of recently completed jobs, say in the last month, from a record of all jobs. If a Panel is being used as a sample, it will be particularly important to confirm that Panel members have recent experience of service use, given the purpose of the specific exercise.

If a sample is selected, it should be based on comprehensive and up to date contact information such as addresses or telephone numbers. The survey should take place as soon as possible after the service has been delivered.

If there is no sample frame of all service users and the purpose of the survey is to collate typical or general experience of service use, samples could be based on quotas reflecting the spread of locations, service delivery points and types of service.

Flow sampling may be appropriate. However, it may be difficult to rely on staff selecting a sample at the point of delivery and may be more reliable to use a census rather than a sample. Samples based on comment cards, pre-paid reply slips, suggestion boxes or local surgeries are self-selecting and may be very useful in highlighting problematic issues, but should not be used as the basis of an overall assessment of the service.

Where the intention of the survey is to follow up dissatisfaction, the survey should be a continuous exercise rather than an occasional one and be based on a 100 per cent census of all service users, rather than a sample, so that everyone has the opportunity to have their problems addressed.

What type of survey should be conducted?

A variety of methods can be used to gather feedback in this way. These include interviews, postal surveys, telephone surveys, self-completion tear-off slips and so on each with advantages and limitations. See Figure 4.1.

The nature of the service being surveyed will affect the choice of method. In most cases, telephone, postal or self-completion surveys and the use of tear-off slips will be suitable, as long as they are short and use simple, “tick boxes” types of questions. There should be an opportunity for service users to add their own comments.

Self-completion surveys may be better at getting a more considered, honest appraisal than a telephone survey and fuller information than a tear-off slip. Use of in-house staff to undertake interviews may compromise data quality. Here use of independent interviewers or self-completion methods that offer greater privacy may be more appropriate.

The use of the data should be designed into the survey from the start as part of a utilisation-focus to the research. Questions should be worded to provide useful information on which responsive action can be based. Simulating how the data will be used will also help in the design of the questions to be asked.

These kinds of surveys should be able to be analysed fully and quickly in order that the data is as recent and as useful as possible.

It will be important to consider the range and source of responses to service specific surveys to consider whether and how those who have responded are likely to differ from those that have not. Surveys based on small samples and low response rates should be treated with care. For example, if 250 questionnaires were posted to recent service users and only 36 replied, giving a response rate of 14 per cent, this would be of little value. A few simple questions asked routinely at the point of delivery, together with use of existing information would be likely to be of more value. Surveys with low response rates may provide informative data for landlords, but it is important not be tempted to make wide ranging claims on the basis of such responses and it will be necessary to undertake more robust exercises from time to time to address these deficiencies.

It may be helpful to encourage a better response rate to ensure that even simple reply slips tell people the purpose of the exercise, what action has been taken on the basis of such exercises and how feedback has been incorporated into service standards.

Data collected in this way should be analysed frequently as it will become out of date quickly and be less useful. Individual instances of dissatisfaction should be followed up promptly. The analysis should identify any requirements for further investigation of trends or issues.

The results should be reported to all key audiences in an appropriate and accessible way. This need not take the form of a report but could use newsletters or local newspapers, posters, leaflets or presentations to tenants groups or forums. Other service users, staff, committee or board members and councillors should also be informed.

Service specific surveys: checklist

√ Be clear about the purpose of the survey and the use to which the data will be put.

√ Check that the survey is of recent service users, rather than a general sample.

√ Keep it short and simple and use local terminology.

√ Be clear about what service changes may realistically be expected as a result of the survey.

√ Design use in: each survey should have a tightly defined focus and ask for views and information on clear specific issues that can be easily analysed and acted upon.

√ Focus groups or other consultative input from residents and staff can assist in both the design of surveys and the further analysis of the findings.

√ Check that the sample was selected in an appropriate way and based on reliable, up to date information.

√ Choose an appropriate type of survey and consider how to maximise response rates. Distribution and collection of questionnaires by local staff or contractors may encourage better response rates.

√ Consider whether anonymity is necessary; it may not be an advantage for survey responses to be anonymous. There should at least be some provision for more in-depth follow up of responses on an individual basis at the discretion of the individual resident.

√ Be aware that small sample sizes are likely; this will mean that it is usually not possible to analyse the data on a sub-group basis, such as by different age groups.

√ Ensure that all the data is analysed promptly and fully.

√ Ensure that appropriate follow up in individual cases is undertaken.

√ Analyse and report the findings in an appropriate and accessible way to all relevant parties including service users themselves, ensuring that the implications for service delivery been identified and invite feedback.

√ Identify any further research and consultation needs.

Alternatives and related approaches:

• General satisfaction surveys

• Continuous monitoring surveys

• Using existing information

• Mystery shopping

• Using comments, compliments and complaints as feedback

• Significant event analysis

Previous page Next page