Scholars have called for further and more thorough research on NGOs, cautioning that the trends in NGO performance are based on ideological grounds in the absence of empirical verification.⁸ Scholars further argue that inadequate research on NGO performance would cause this area of study to remain immature, make it difficult to get accurate and comprehensive data and thus adversely influence funding decisions.^4,9,10 Intensification of research on development NGO performance can be attributed to Edwards and Hulme¹⁰ who, in the mid-1990s, assembled a collection of studies on NGO performance. However, little is known regarding the distinctive characteristics of the existing body of NGO performance literature.

The diversity of NGO typology is a complex, contested and controversial matter. For the purposes of this paper, development NGOs refers to a diverse set of institutions (from grass roots or community level to national, regional or international level) that operate on a non-profit basis, generally serve the public sector, and are engaged in long-term development work within the framework of international development cooperation.⁹ As the primary mission of development NGOs is focused on development goals desired by their targeted beneficiaries and their communities, the performance of NGOs should be assessed by their effectiveness as organisations to assist beneficiaries achieve mutually identified social goals. Traditionally, the evaluation results of NGO interventions have been utilised as a proxy for the overall performance of the organisation. There is now a call for NGO performance measurement, beyond project assessment,⁶ to focus on the overall functioning of the organisation.

In a recent systematic review of journal articles published in online English-language academic journals between 1996 and 2008, factors influencing development NGO performance were investigated.¹¹ This paper takes the literature review further and focuses on the research characteristics of the 31 articles that met the inclusion criteria. Special attention was paid to, (1) the number of publications, (2) publication outlets (journals and journal cluster), (3) author collaboration (sole or joint authors), (4) author affiliation, (5) study location, (6) study period, (7) study topics and (8) method and sources of information. Findings, which will inform future research and policy directions, confirm: the continued need for increased research in NGO performance; the comparatively low reporting on beneficiaries’ voices in NGO performance research; the low number of published researchers in Africa; and minimal collaborative efforts between the ‘North’ and ’South’ in this field.

A great deal of time was spent becoming familiar with search parameters specific to the subject literature, and periodically consulting experienced librarians at the University of Cape Town. The initial attempts did not produce the final search strategy – the design progressed as knowledge was gained of the platforms, databases, keywords, indexing and structuring of texts.

Internet searches were conducted using Google Scholar. Further searches were undertaken of those journals from which articles were accessed. Once relevant publications were sourced, their reference lists (and references of references) were examined and followed up for further relevant articles. Contact was made with experts in the field who suggested additional sources such as articles accepted by journals but not yet published, and names and email contacts of authors who had published work that may contribute to the review.

• Articles published between 1996 and 2008 in scholarly peer-reviewed journals.
• Articles written in English on NGOs engaged in long-term development work anywhere in the world.
• Studies that examined different categories of development NGOs or not-for-profit/non-profit organisations such as national, regional or international NGOs, civil society organisations, community-based organisations, grass roots organisations and faith-based organisations.
• Studies that focused on the performance of development NGOs, with empirical evidence or analysis of underlying facilitators and constraints.

The exclusion criteria were:

• Journal editorials, book reviews, books, reports, newspaper articles and other types of ‘grey’ literature.
• Articles on other types of organisations including public and private sector (corporate) organisations.
• Articles on NGOs engaged in humanitarian and emergency relief; on organisations that represent and advance the views of business interests and educational institutions; on NGO individuals or interventions; and on NGO theories, conceptual frameworks and models.

A grid was developed in MS Excel which was populated with the extracted data. This grid was exported to Statistical Package for the Social Sciences (SPSS),¹⁵ a statistical software package for data analysis, where variables were defined, coded and screened for detailed and complete information. Data were ‘cleaned’ by checking each variable for consistent and accurate information. Frequency analysis was mainly utilised to generate descriptive statistics.^16,17 In this process, new variables, such as number of authors, journal cluster, and author affiliation and study location (developing or developed country) were created from the initial administrative data set.

Over the years, there has been a steady increase in the number of academic articles analysing factors influencing NGO performance (Figure 1). The cumulative analysis shows that only 3 articles were published between 1996 and 2000, which increased to a total of 11 articles by 2004 and to 31 articles by 2008. Most studies (20) were published between 2005 and 2008.

FIGURE 1: The number of articles published between 1996 and 2008 on factors influencing the performance of NGOs.

The 20 journals, in which the 31 articles were published, were clustered into six categories, based on core academic disciplines. The most preferred category was development studies journals with over half of the articles (18). There was medium preference for evaluation and accountability, and health journals (with 4 articles each). The least preferred were political journals (3 articles), and information technology and law journals (1 article each).

The journal cluster analysis by year shows that development studies journals have been utilised consistently over the years, with some growth since 2001. Evaluation and accountability journals have only been utilised since 2005, and the same trend is evident in the health journals.

In each article, authors reported their institutional affiliation at the time of publication. These institutions were categorised by country and are hereafter referred to as author location. The resulting frequency of author locations is summarised in Table 1. The UK and USA were the most frequently reported author location, with nearly two-thirds of the total frequency. Overall, the likelihood that an author location was a developed country was four times greater than that of the location being a developing country.

Of the 31 studies, 23 were written by authors in developed countries, 4 were written by authors in developing countries and 4 by authors in both developed and developing countries. Of the 31 studies, 6 study locations were in developed countries, 19 were in developing countries and 6 were in both developed and developing countries.

Of the 19 studies located in developing countries, 12 were reported by authors in developed countries, and the remaining 7 by authors in both developed and developing countries. Additionally, of the 13 authors who conducted studies in Africa, only 3 were affiliated to an institution in Africa. In contrast, the articles on the six studies located in developed countries were all authored by researchers in developed countries.

Of the 19 articles with available information, the data collection period ranged from 3 to 36 months. Most of the studies collected data for 1 year or more. Further analysis was conducted on the time between the year of completion of data collection and the year the article was published. The time lag ranged from 1 to 6 years. The majority (15) of the studies experienced a time lag of 3 years or more, with only four studies having been published within 2 years of the completion of data collection.

This finding should, however, be interpreted with caution, as the categories of topics identified, and the frequency with which they were reported, are products of the way in which the studies defined their topics. For instance, some studies used the term ‘governance’ while discussing ‘accountability’ issues. Further, there seems to be an overlap between various practices, as some topics were examined under the banner of others. Most commonly, accountability emerged as an umbrella term that incorporates aspects of governance, financial reporting, monitoring and evaluation. Additionally, organisational learning involves the processes of monitoring, evaluation and reflection whereby lessons are learnt from experiences. This finding reflects an overlap and interconnection between the different organisational processes and organisational performance assessment processes. This finding further shows that the field of organisational performance is multidimensional, and relevant to a broad range of management themes.

FIGURE 1: Frequency of sources of data in articles published between 1996 and 2008 on factors influencing the performance of NGOs.

Despite the large growth of interest in NGOs and the large body of NGO publications, there are a relatively small number of academic peer-reviewed journal articles that examine factors influencing NGO performance. These findings substantiate previous arguments that the organisation and management of NGOs has received relatively little attention from researchers,⁴⁹ and that the subject of NGOs has not yet entered the academic mainstream and hence the overall state of knowledge remains somewhat underdeveloped.⁵ The low academic research volume may be attributed to several factors. Firstly, the fields of NGO management and research are relatively new,⁴⁹ and measuring NGO performance continues to be relatively underdeveloped.⁶ Secondly, many evaluations and studies are not published – numerous organisational reviews are kept confidential because there is a bias towards publishing only positive results, with failures concealed because of various sensitivities and fears (particularly the fear of revocation of funding). There is also a bias towards undertaking reviews and publishing results from well-resourced organisations.^4,50 Thirdly, this study indicates that the lengthy, intensive and costly nature of the research process may mean that such studies are difficult to undertake in low-resourced NGO settings. One proxy indicator is the long period of data collection – most studies took more than 1 year, with some taking up to 3 years. However, these figures need to be treated with caution, as the duration is a product of the way studies reported the period of data collection and of the methodology of the study. Another proxy is the time lag between completion of data collection and publication of an article, as a majority of the articles had a time lag of 3 years or more. This length suggests that data capture, analysis, verification, writing, peer-review and acceptance of an article for publication are extremely intensive, time-consuming processes, which may reduce the likelihood of studies being published in this field. Nonetheless, this time lag is not atypical of research work in other areas which culminates in the publication of a journal article.

Interest in factors influencing NGO performance has found a home in a wide array of journals. This observation is consistent with sentiments by Edwards and Fowler³ that NGO literature is widely dispersed and locating it requires long and difficult searches – as was the experience of the researchers in this study. The same observation may further confirm that NGO performance is a multidisciplinary field continually drawing ideas and techniques from other fields,⁴⁹ particularly from international development, management, health, politics and law.

Location analysis by author and study reveals that there is an international scope in researching NGO performance. Authors were located and the studies conducted in both developing and developed countries covering the continents of Africa, Asia, Australia and Oceania, Europe, North America, and South America. That there were comparatively few studies in Latin America is attributed to the fact that only English-language publications were included.

Further analysis revealed that most authors were affiliated to institutions in developed countries, although the majority of studies examined NGOs in developing countries. Author and study location analysis revealed that most studies in developing countries were conducted by researchers in developed countries, whilst all studies in developed countries were conducted by researchers in developed countries. Whilst there is rhetoric of ‘North-South’ and even ‘South-South’ collaboration and participation in international development,^51,52,53 this collaboration is not evident in the research on understanding NGO performance.

This striking finding raises further questions. The first is whether this trend could be attributed to low capacity in developing countries. The predominance of author locations being in developed countries may suggest that these institutions have more established resources, reputations and capacities to examine NGO performance than those of institutions in developing countries, and hence scholars interested in this field are affiliated with these institutions. According to Mishra et al.⁵⁴, India has many institutions to prepare management professionals in the private corporate sector, but none in the NGO sector, hence scholarship in development management of NGOs has lagged far behind practice. There may also be additional challenges for developing country authors to break into discourses, such as engaging in the academic style of writing, prejudice by journal editors and publishing networks. Another question is whether there is a trend of ‘Northern’ institutions examining ‘Southern’ institutions, particularly as large amounts of development funding originate from ‘Northern’ countries. Wallace and Chapman⁵² argue that while UK NGOs and donors state commitment to downward accountability and promotion of local ownership and control of development, the policies and procedures of funding disbursement and accounting ensures that upward accountability dominates, which is part of a wider problem of donor domination of recipients. According to Muchungunzi and Milne⁵⁵, the ‘South’ always has to account to the ‘North’ and not vice versa; donor money and expertise from the ‘North’ are more valued than are labour and expertise from the ‘South’.

Based on the findings of this study, several areas of future research can be identified. Firstly, future research should explore the wide range of factors that influence NGO performance, including the institutional context and influences of the external environment,⁵⁶ along with the strategic location of NGOs. These factors may provide evidence for policy and funding allocation effectiveness to improve beneficiaries’ access to essential resources.²

Secondly, although providing details of the period of data collection is a usual requirement in research, less than half (13 out of 31) of the studies explicitly met this requirement. Some studies did not discuss the methods used. Future research in the area of understanding NGO performance would benefit from an explicitly reported methodology section.

Thirdly, future research should engage more rigorously with the opinions and needs of beneficiaries. Analysis of the sources of data distinguished eight categories, reflecting the trend that researchers gather data from various sources, confirming that a wide range of stakeholders are involved in varying magnitudes in assessing NGO performance. However, further analysis showed a marked preference towards gathering data from NGO staff, particularly from senior-level and middle-level managers including directors and executives, programme coordinators and programme staff. Few studies endeavoured to source information from NGO field workers. Comparatively few studies gathered data from beneficiaries, which suggests that the rhetoric of beneficiary participation and engagement seems not to have been effectively implemented in research projects, and may be a proxy reflection of the reality in intervention practices. Of the two forms of participation – in implementation and in decision-making – beneficiaries are often involved in implementing activities based on decisions taken in other fora. Beneficiaries are important stakeholders who should be involved from the design to evaluation, and back again to reformulation of development interventions.^57,58 Furthermore, incorporating their voices into studies generates data that would otherwise be overlooked and can make the realities and experiences of beneficiaries more prominent.^59,23

Finally, we turn to focus on the African continent. Strikingly, only 3 of 13 authors who conducted studies in Africa were affiliated to an institution in Africa, namely in Cameroon, South Africa and Tanzania. These 3 authors represent less than a tenth of all the 38 authors of the 31 articles. This result reveals the need to address the low number of published researchers in Africa in this field, and the crucial need to nurture African researchers. Along with this need, the scarcity of partnerships with African-based researchers in the field of NGO performance research needs to be further addressed.

However, there are limitations that should be taken into consideration. The search was restricted to online academic articles, written in English, specifically focused on factors influencing NGO performance. The search criteria thus excluded journal articles in other languages and in non-electronic journals, and other published but ‘grey’ literature. These publications could have possibly contained relevant data that may have made significant contributions to our understanding of development NGO performance.

Firstly, from 1996 to 2008 there was steady growth in the number of journal articles reporting factors influencing NGO performance. However, given the small number of studies, there is a need to increase published research, particularly research examining the facilitating and constraining factors influencing NGO performance. These studies would benefit from the inclusion of an explicit and more robust methodology section to enhance further meta-analysis and comparison studies. Secondly, there is a need to increase the practice of hearing beneficiaries’ voices, not only in assessing beneficiaries’ needs, but also in gathering their perspective regarding organisational performance. Finally, this study found plausible evidence of a relatively small number of published researchers based in Africa in the field of NGO performance assessment. Additionally, little academic research on NGO performance is undertaken in partnership with researchers in Africa. These findings reveal the need to advocate for policymakers in academic, donor, government and development institutions to intensively identify and nurture researchers in Africa in the field of development and performance management.

2. Kareithi RNM, Flisher AJ. Location of development NGOs providing HIV and AIDS services to young people in Cape Town, South Africa. J Soc Dev Afr. 2009;24(2):9–36.

3. Edwards M, Fowler A, eds. The Earthscan Reader on NGO management. London: Earthscan; 2002.

4. Unerman J, O'Dwyer B. Guest editorial: On James Bond and the importance of NGO accountability. Acc Auditing Accountability J. 2006;19(3):305–318. http://dx.doi.org/10.1108/09513570610670316

5. Lewis D. Issues and priorities in non-governmental organisation research. J Hlth Manag. 2006;8(2):181–193. http://dx.doi.org/10.1177/097206340600800202

6. Fowler A. Demonstrating NGO performance: Problems and possibilities. Dev Practice. 1996;6(1):58–65.

7. Paton R. Managing and measuring social enterprises. London: Sage; 2003.

8. Edwards M, Hulme D. Too close for comfort? The impact of official aid on non-governmental organizations. World Dev. 1996;24(6):961–973. http://dx.doi.org/10.1016/0305-750X(96)00019-8

9. Fowler A. Striking a balance: A guide to enhancing the effectiveness of non-governmental organisations in international development. London: Earthscan; 1997.

10. Edwards M, Hulme D, eds. Non-governmental organisations performance and accountability: Beyond the magic bullet. London: Earthscan; 1995.

11. Kareithi RNM. Performance of development NGOs in HIV prevention for young people. PhD thesis, Cape Town, University of Cape Town, 2012.

12. Khan KS, Riet G, Glanville J, et al. Undertaking systematic reviews of research on effectiveness: CRD's guidance for those carrying out or commissioning reviews. CRD Report No.: 4. York: NHS Centre for Reviews and Dissemination; 2001.

13. Petticrew M, Roberts H. Systematic reviews in the social sciences: A practical guide. Oxford: Blackwell; 2006. http://dx.doi.org/10.1002/9780470754887

14. Mouton J. How to succeed in your masters and doctoral studies: A South African guide and resource book. Pretoria: Van Schaik; 2001.

15. Statistical Package for the Social Sciences (SPSS). Version 12.00 for Windows. Armonk, New York: IBM; 2003.

16. Weathersby R, Freyberg R. Study guide and SPSS manual. New York: Worth; 2008.

17. Pallant JF. SPSS survival manual: A step by step guide to data analysis using SPSS. 3rd ed. Crows Nest, Australia: Allen & Unwin; 2007.

18. Barr A, Fafchamps M, Owens T. The governance of non-governmental organizations in Uganda. World Dev. 2005;33(4):657–679. http://dx.doi.org/10.1016/j.worlddev.2004.09.010

19. Bebbington A. Donor–NGO relations and representations of livelihood in non-governmental aid chains. World Dev. 2005;33(6):937–950. http://dx.doi.org/10.1016/j.worlddev.2004.09.017

20. Ebrahim A. Accountability in practice: Mechanisms for NGOs. World Dev. 2003;31(5):813–829. http://dx.doi.org/10.1016/S0305-750X(03)00014-7

21. Edwards M. NGO performance – what breeds success? New evidence from South Asia. World Dev. 1999;27(2):361–374. http://dx.doi.org/10.1016/S0305-750X(98)00125-9

22. Kilby P. Accountability for empowerment: Dilemmas facing non-governmental organizations. World Dev. 2006;34(6):951–963. http://dx.doi.org/10.1016/j.worlddev.2005.11.009

23. Kelly L, Kilby P, Kasynathan N. Impact measurement for NGOs: Experiences from India and Sri Lanka. Dev Practice. 2004;14(5):696–702. http://dx.doi.org/10.1080/0961452042000239841

24. Mebrahtu E. Perceptions and practices of monitoring and evaluation: International NGO experiences in Ethiopia. Dev Practice. 2002;12(3):501–517. http://dx.doi.org/10.1080/0961450220149645a

25. Roche C. Impact assessment: Seeing the wood and the trees. Dev Practice. 2000;10(3):543–555. http://dx.doi.org/10.1080/09614520050116703

26. Dar S. Negotiating autonomy: Organising identities in NGOs. J Hlth Manag. 2007;9(2):161–188. http://dx.doi.org/10.1177/097206340700900202

27. Nanavati A. Performance evaluation of non-government development organisations: A study in Vadodara, Gujarat. J Hlth Manag. 2007;9(2):275–299. http://dx.doi.org/10.1177/097206340700900208

28. Srinivasan V. Understanding the human resource challenges in not-for-profit organisations from a lifecycle perspective. J Hlth Manag. 2007;9(2):189–199. http://dx.doi.org/10.1177/097206340700900203

29. Benjamin LM. Account space: How accountability requirements shape nonprofit practice. Nonprofit Volunt Sector Q. 2008;37(2):201–223. http://dx.doi.org/10.1177/0899764007301288

30. Ebrahim A. Information struggles: The role of information in the reproduction of NGO-funder relationships. Nonprofit Volunt Sector Q. 2002;31(1):84–114. http://dx.doi.org/10.1177/0899764002311004

31. Ebrahim A. Accountability myopia: Losing sight of organizational learning. Nonprofit Volunt Sector Q. 2005;34(1):56–87. http://dx.doi.org/10.1177/0899764004269430

32. Barr A, Fafchamps M. A client-community assessment of the NGO sector in Uganda. J Dev Stud. 2006;42(4):611–639. http://dx.doi.org/10.1080/00220380600682009

33. Fruttero A, Gauri V. The strategic choices of NGOs: Location decisions in rural Bangladesh. J Dev Stud. 2005;41(5):759–787. http://dx.doi.org/10.1080/00220380500145289

34. Goddard A, Assad MJ. Accounting and navigating legitimacy in Tanzanian NGOs. Acc Auditing Accountability J. 2006;19(3):377–404. http://dx.doi.org/10.1108/09513570610670343

35. O’Dwyer B, Unerman J. The paradox of greater NGO accountability: A case study of Amnesty Ireland. Acc Organ Soc. 2008;33(7–8):801–824. http://dx.doi.org/10.1016/j.aos.2008.02.002

36. Kegeles SM, Rebchook GM, Tebbetts S. Challenges and facilitators to building program evaluation capacity among community-based organizations. AIDS Educ Prev. 2005;17(4):284–299. http://dx.doi.org/10.1521/aeap.2005.17.4.284

1. Carman JG. Evaluation practice among community-based organizations: Research into the reality. Am J Eval. 2007;28(1):60–75. http://dx.doi.org/10.1177/1098214006296245

38. Bornstein L. Systems of accountability, webs of deceit? Monitoring and evaluation in South African NGOs. Development. 2006;49(2):52–61. http://dx.doi.org/10.1057/palgrave.development.1100261

39. Chaplowe SG, Engo-Tjega RB. Civil society organizations and evaluation: Lessons from Africa. Evaluation. 2007;13(2):257–274. http://dx.doi.org/10.1177/1356389007075227

40. Gneiting U. Assessing the effects of market-based performance measurement on NGOs. J Dev Soc Transformation. 2008;5:33–42.

41. Höhn S. The failures of success–NGOs and the decreasing support for civil society in Namibia. Polit Perspectives. 2008;2(1):1–21.

42. Jung K, Moon MJ. The double-edged sword of public-resource dependence: The impact of public resources on autonomy and legitimacy in Korean cultural nonprofit organizations. Policy Stud J. 2007;35(2):205–226. http://dx.doi.org/10.1111/j.1541-0072.2007.00216.x

43. Porter G. NGOs and poverty reduction in a globalizing world: Perspectives from Ghana. Prog Dev Stud. 2003;3(2):131–145. http://dx.doi.org/10.1191/1464993403ps057ra

44.Bornstein L. Management standards and development practice in the South African aid chain. Public Adm Dev. 2003;23(5):393–404. http://dx.doi.org/10.1002/pad.291

45. Khan MM. Accountability of NGOs in Bangladesh: A critical overview. Public Manag Rev. 2003;5(2):267–278. http://dx.doi.org/10.1080/1461667032000066435

46. Wallace T. New development agendas: Changes in UK NGO policies & procedures. Rev Afr Polit Econ. 1997;24(71):35–55. http://dx.doi.org/10.1080/03056249708704237

47. Kalb J. The institutional ecology of NGOs: Applying Hansmann to international development. Tex Int Law J. 2006;41(2):297–320.

48. Lewis D, Madon S. Information systems and nongovernmental development organizations: Advocacy, organizational learning, and accountability. Inform Soc. 2004;20(2):117–126. http://dx.doi.org/10.1080/01972240490423049

49. Lewis D. The management of non-governmental development organizations. 2nd ed. London: Routledge; 2007.

50. Cassen R. Does aid work?: Report to an Intergovernmental Task Force. Oxford: Clarendon Press; 1986.

51. Davids I, Theron F, Maphunye KJ. Participatory development in South Africa: A development management perspective. Pretoria: Van Schaik; 2005.

52. Wallace T, Chapman J. Some realities behind the rhetoric of downward accountability. Working paper presented at: INTRAC 5th Evaluation Conference; 2003 Mar 31 – Apr 04; Soesterberg, the Netherlands.

53. Theron F, ed. The development change agent: A micro-level approach to development. Pretoria: Van Schaik; 2008.

54. Mishra D, Biswas SN, Roy S. Governance of NGOs: Contextualizing in the Indian experience. Int J Rural Manag. 2005;1(2):185–201. http://dx.doi.org/10.1177/097306800500100203

55. Muchungunzi D, Milne S. Perspectives from the south: A study on partnership. Ottawa: CIDA, NGO Division; 1995.

56. Kareithi RNM, Lund C. Obstacles to creating an environment conducive to HIV prevention for young people. Afr J AIDS Res. 2011;10(4):465–477. http://dx.doi.org/10.2989/16085906.2011.646661

57. Szporluk M. A framework for understanding accountability of international NGOs and global good governance. Indiana J Global Legal Stud. 2009;16(1):339–361. http://dx.doi.org/10.2979/GLS.2009.16.1.339

58. Kareithi RNM. Understanding influences of the social and economic environment on HIV prevention: Drawings and voices of young people in Cape Town, South Africa. Paper presented at: 6th S.A.H.A.R.A Conference: Are we turning the tide?; 2011 Nov 28 – Dec 02; Port Elizabeth, South Africa.

59. Chambers R. So that the poor count more: Using participatory methods for impact evaluation. J Dev Effectiveness. 2009;1(3):243–246. http://dx.doi.org/10.1080/19439340903137199