The approach
4.1 In addition to developing and implementing LHS to tackle the housing problems identified in their area, local authorities will need to measure how successful their strategy has been at different points in time. Adjustments to the strategy can then be made.
4.2 As intimated earlier in this guidance, the Housing (Scotland) Act 2001 requires local authorities to keep their LHS under review, and to provide Scottish Ministers with such information as they require about the implementation of their strategy.
4.3 Local authorities will therefore be expected to produce annual updates to their LHS. These will provide an opportunity to take account of new developments and changes affecting the strategy.
4.4 Measuring performance is more than a retrospective act. Considering how progress will be measured needs to be thought through at the time objectives are constructed and actions are planned.
4.5 This is a fundamental part of the planning process. As noted earlier in the guidance, strategic objectives must be explicit in terms of what is to be done within a specified time, which provides the basis for monitoring progress.
4.6 Local authorities should develop a framework for monitoring and evaluating their LHS. However, this can be a complex process because:
Indicators of performance and indicators of change have to be devised which are specific to strategic objectives and these will vary according to the local housing context;
Indicators will require a base line to be set, against which progress can be measured. Data then needs to be collected for the base line and also at regular intervals to measure change;
Specifying indicators of change can be difficult, especially when changes can be attributed to other events not related to housing activity or the LHS;
A consensual view on the indicators and performance measures to be used needs to be reached if possible and data is likely to be required from others to allow measurement of progress.
4.7 Detailed guidance is available to assist local authorities devise monitoring and evaluation processes. In particular, the ODPM good practice guidance on "Running and Sustaining Renewal Areas" is helpful.
4.8 Examples of other indicators used for monitoring poverty and social exclusion are available.
4.9 The distinction between monitoring and evaluation needs to be clear:
Monitoring refers to an on-going process, checking and overseeing that the actions agreed in the strategy are taking place and that short-term targets are being met;
Evaluation refers to a process that takes place periodically and involves a wider assessment of the effects or impact of the actions and the extent to which strategic objectives are being achieved.
4.10 Guidance and good practice indicate that common key elements of a monitoring and evaluation framework are:
Setting objectives or long term targets - as noted earlier in the guidance, these should be measurable and designed to address the housing problems identified;
Agreeing inputs or actions - these include the resources, investment, mechanisms and activities for achieving the objective as agreed through the option appraisal process detailed elsewhere in the guidance. They should be specified in advance and form the basis for management reporting;
Specifying outputs - these tend to be easily measurable or quantifiable and include e.g. houses built and projects implemented. Outputs can be less product focused and include the number of people or households housed. Outputs are what the inputs and actions produce and form the basis for the monitoring of the strategy. They should meet SMART criteria. Outputs are the means to the end and not the end itself.
Specifying outcomes or impact - these need to demonstrate the extent to which strategic objectives are being achieved. Measuring the outcomes or impact requires indicators of change to be developed and relevant data collected and compared over time. This measurement of change through time constitutes evaluation. In LHS, outcomes should be based on objectives, meet SMART criteria and be clearly based on housing activity.
4.11 Examples of how this framework can be used are available.
4.12 A description of the types of measures and indicators available is also provided along with information on the issues and dilemmas in measurement.
4.13 A helpful checklist is provided in the CIH/LGA publication.
