Find information about

Section 3 Issues and concerns in research and consultation

3.03 Assessing satisfaction and expectations

Understanding and measuring service user satisfaction is a central concern of research and consultation amongst social landlords. Satisfaction is a widely accepted concept despite real difficulties in measuring and interpreting typical approaches to its assessment. The most common approach is the use of general satisfaction surveys undertaken every few years and designed to track changes over time.

There are a number of difficulties with the concept of satisfaction.

• It is not static, but changes over time; new experiences and levels of awareness will alter the potential levels of satisfaction that could be achieved.

• It is likely to be complex and the result of a mix of experiences before, during and after the point at which it is measured.

• It occurs in social contexts which are varied and changing and may be unpredictable or inexpressible to the service user.

• It may be difficult to express the reasons for satisfaction; particularly where less tangible aspects of services are being considered.

• It may be easier to express the reasons for dissatisfaction, particularly if this is the exceptional state.

• Without understanding the causes of satisfaction, there is a danger that a landlord might treat a ‘good result’ as a reason not to change anything, seeing it largely as a PR tool.

A typical way of measuring satisfaction is to use a five point scale which ranges from ‘very satisfied’ to ‘very dissatisfied’. This approach has the value of simplicity and consistency. However, a utilisation focused approach will follow this up with a question that asks for reasons for the answer, whether positive or negative. Without this follow up, there is no opportunity to check that this is a valid measure of satisfaction to the individual. In addition, although it might be possible to track change over time by repeating the approach, the reasons for any change and what action would be required to sustain it would not be understood. There is a need for further questioning or follow up with another approach, such as a focus group.

Understanding expectations

Social landlords tend to treat high levels of satisfaction as a measurement of high service quality, but service quality and satisfaction are not necessarily the same thing. The role of expectations is crucial. Satisfaction occurs when expectations are met or exceeded. The main problem with typical questions about satisfaction is that they make assumptions about expectations of services that may not be part of the individual’s idea of satisfaction, which could be very wide ranging, varied and quite at odds with the assumptions of the landlord organisation. There are many sources of service users’ expectations:

• personal needs

• previous experience

• word of mouth communication

• explicit service communication

• views about government

• values and beliefs

• implicit service communications.

The nature and impact of each of these influences will vary for different people and services. It is not clear how expectations and satisfaction ratings interact. It may be that lower expectations produce higher satisfaction ratings, as in such circumstances it should be easier to pleasantly surprise customers. Alternatively, lower expectations may make it harder to achieve higher satisfaction ratings. Expectations are discussed more fully elsewhere (see references (13)).

In typical satisfaction surveys, priority tends to be given to issues that social landlords deem to be important, rather than checking out assumptions about service users’ expectations. To assess expectations it is important to identify service characteristics so that it is clear how the different dimensions of service provision are valued. An example of these dimensions is included in the section on Servqual which is a technique that measures both expectations and perceptions.

A practical modification of the satisfaction survey approach would be to undertake pre-survey qualitative work to explore expectations amongst customers which could then be built into the design of the questions.

Types of rating scales

It is not possible to provide a review of different approaches to assessing satisfaction and expectations in this guidance. However, it is worth commenting on the use of rating scales. Studies of satisfaction show that the common aggregation of ‘very’ and ‘fairly’ satisfied responses will cover a very wide range of service experiences and attitudes and hence, also a fairly large proportion of the customer base. Whilst very satisfied generally did represent a positive statement, fairly satisfied could well be taken to mean ‘I am not very satisfied’.

There are a number of different types of rating scales that can be used to measure attitudes each of which will produce different responses:

• satisfaction scale (‘How satisfied are you with…?’) with the response choices being ‘very satisfied’, ‘satisfied’, ‘neither satisfied nor dissatisfied’, ‘dissatisfied’, and ‘very dissatisfied’;

• performance scale (‘Overall how would you rate…’) with choices such as ‘very good’, ‘good’, ‘fair’ etc;

• expectations scale (‘Overall, compared with what you expected, how would you rate…’) with choices such as ‘much better than expected’, ‘better than expected’, ‘about as expected’, etc;

• improvement scale (‘Indicate the amount of improvement, if any, that is needed’) with choices such as ‘none’, ‘slight’, ‘some’, ‘much’, and ‘huge’;

• compared to the ideal scale (‘Compared to the ideal…, how would you rate…?’) with the response choices being: ‘very good’, ‘good’, ‘fair’, ‘poor’, and ‘very poor’; and

• recommend scale (How likely are you to recommend… to a friend’) with the response choices being ‘very likely’, ‘likely’, ‘neither likely nor unlikely’, ‘unlikely’, and ‘very unlikely’.

Overall performance and satisfaction scales tend to receive higher average ratings. In the light of the range of responses encompassed in the widely used satisfaction scales, it may be worth using scales that result in less favourable and more testing assessments.

Measuring satisfaction and expectations

There are a number of useful lessons in relation to the measurement of satisfaction and expectations.

• In relation to validity, it is important to ensure that what is measured reflects all the different elements of satisfaction and to make sure that social landlords know what’s important to their service users, in their language, not in the language of the organisation.

• In terms of consistency, it is important to make sure the process of measurement is reliable and consistent across different customer groups and over time.

• Adopting a utilisation focused approach will ensure that the results are specific enough to act on and that ensure sufficient information is collected to define priorities for change.

• In relation to needs and expectations, it is important to focus on identifying customer needs and expectations and whether these are being met, rather than on appraising existing services. The interpretation of findings must be undertaken in the light of knowledge of expectations.

It is also important to take a long term, strategic approach to measuring change over time in expectations and satisfaction.

Previous page Next page