The appropriateness of a realist review for evaluating the South African Housing Subsidy Programme

Abstract

Conducting meta-reviews of government programmes has become common practice. In South Africa, the national Department of Human Settlements and the national Department of Performance Monitoring and Evaluation recently commissioned a team to review the extent to which the Housing Subsidy Programme had provided assets to municipalities and the poor and whether these assets had helped poor households escape from poverty. A realist approach was employed to conduct the review. We argue that, given the
complex nature of housing programmes, the realist review methodology was an appropriate approach to follow in answering the review questions. We explored how the realist review method allowed us to work with the uneven and contested nature of the housing literature and how the review nonetheless enabled elucidation of the factors that had contributed to the expected outcomes. Because this case was the first time that this method was used in a government-commissioned evaluation of housing, there were some practical challenges involved in its use. Some of the challenges were related to the nature of the questions that were asked. At the time of the review, the Department of Human Settlements was in the process of reviewing the 1996 White Paper and, to inform this process, the Housing Subsidy Programme review included a copious number of questions set by the Department of Human Settlements and Department of Performance Monitoring and Evaluation, which made the review rather large and, in some cases, complicated the analysis. In some cases, because the Departments wanted clear-cut answers, the commissioners perceived the theoretical strength of the method, such as offering explanatory instead of conclusive judgement, as a weakness. The paper reveals some limitations of the realist review method for evaluating the multifaceted outcomes of a complex programme, particularly the practical difficulty of dealing with large quantities of data. We do however consider this method to have potential for further reviews.

Significance:

• Housing research in South Africa is uneven which makes any review process difficult.
• The review was unable to offer judgement on the effect that the Housing Subsidy Programme has had on the asset base of the poor.
• The review was useful for making clear which factors will help the Programme to achieve the intended outcomes and also for pointing out on what government should focus to build assets for the urban poor.