The conversion of scientific discoveries to new products and processes and their launch onto the market can be a lengthy process. Similarly, it takes many years before the impact of scientific research on society and the economy is realised and a further length of time before its performance can be measured. Higher education and research institutions, and their governments, often make significant investments into intellectual property management and technology transfer activities through legislative and policy development, human resource development, financial allocation and infrastructure improvement. Since returns on such investments are not immediately apparent, it is important to establish a means by which the impact of their efforts can be determined. In this paper, I examined the measures and indicators that could be developed by institutions and their stakeholders in order to monitor, evaluate and determine the impact of research output and outcomes on the market.

South African higher education and research institutions are growing to appreciate the value of intellectual property (IP), which is evident in the establishment of in-house IP management and technology transfer offices (TTOs) to manage the various aspects related to IP and its transfer from the research environment to the market. The capacity to manage IP varies considerably across institutions; the Council for Scientific and Industrial Research, the Medical Research Council and the Universities of Cape Town, Pretoria, the North-West, Stellenbosch and the Witwatersrand all having relatively well-established and resourced TTOs,^2,3,4 while, at the other end of the spectrum, some institutions may have a single person who, in addition to other responsibilities, is expected to oversee the institution’s entire IP management portfolio and its transfer to the market. As the level of awareness, experience and knowledge of IP and all its facets increases, TTOs are using more sophisticated methods to manage and transfer IP to the market, as is evident from the structures and policies being implemented for these activities. Furthermore, institutions are developing the capacity and capability to effectively exploit existing and newly generated IP.³

Figure 1 is an illustration of IP value as a function of the strategic use of IP.⁵ As indicated in the figure, the most basic function is to create and maintain IP, which is now standard practice at the more research-intensive institutions in South Africa. On observation, the majority of local institutions operate between level 0 and level 2 of the illustration. As institutions graduate from the start-up phase, gaining more transactional experience and know-how, and as increasing resources are deployed to IP management and technology transfer activities, it becomes necessary to monitor and evaluate the performance of TTOs.

Several stakeholders have an interest in TTO operations. Scientists require reasonable turnaround times, good coordination, effective IP management and successful technology transfer of their ideas to the market. Institutional leaders are under increasing pressure to justify their spending on IP protection, human resources, training and technology transfer activities and would therefore want to evaluate the effectiveness and efficiency of TTOs. Government, on the other hand, is concerned with the contribution of science and technology to the application of various socio-economic solutions that result in the betterment of people’s lives.

I have thus structured this paper in two parts: Part 1 discusses the importance of a monitoring and evaluation (M&E) system and the value that institutions could derive from implementing such a system, while Part 2 deals with common indicators to assess the performance of TTOs.

Figure 1:Intellectual property (IP) value as a function of strategic use

Definitions relating to monitoring, evaluation and impact assessment

‘Monitoring’⁶ involves data recording, analysis, reporting and information storage. It is the continuous assessment of TTO activities against (predetermined?) expected deliverables. The outcome of monitoring should be a set of corrective actions at the operational level.

‘Evaluation’,⁶ on the other hand, is a periodic assessment that involves the analysis of data generated during monitoring, as well as information gathered from other sources. The recommendations generated from the evaluation process are used to affirm or modify objectives, resources and processes.

‘Impact’⁶ is the intended and unintended effects of a given intervention on target beneficiaries. The resulting impact can include economic, sociocultural, environmental and institutional effects and can affect production. Depending on the result of the impact, behaviour, practice and/or circumstances could change.

Why is M&E and impact assessment important?

Besides reporting on institutional performance, a good M&E system assists with decision-making, planning, strategy and policy development. Through self-monitoring and benchmarking, monitoring can help to increase the effectiveness of staff performance, identify staff development needs and facilitate the establishment of management standards, with the aim of improving working practices and efficiencies by identifying system, process and practice problems and opportunities. Furthermore, effective evaluations enable institutions to remedy problems timeously, thus managing risk. M&E systems offer accountability for resources invested.

Selecting an evaluation system
An effective M&E system needs to have a structured set of indicators, as well as a clear and simple approach to assessing the impact on beneficiaries. There needs to be provision for collecting data and managing records that enables the information to remain accessible. Systems for gathering, analysing and reporting information need to be sustainable and, for this, resources and capacity are required. When choosing an M&E system, institutions need to determine⁶:

• What should be measured?

• For whom it should be measured?

• For what purpose it should be measured, that is, what is the intended use of the results?

• How it should be measured?

• How the data will be collected and stored?

• When, and in what form, is the information needed?

• Who will collect, analyse and present the information?

The above issues will influence the type of evaluation chosen and, therefore, the time and resources allocated to this exercise.

Characteristics of a good evaluation system
South Africa needs to build a community of practice from which to learn, against which to benchmark and with which to monitor TTOs. Performance evaluation systems need to be balanced by ensuring that they are not driven solely by quantitative data, but also by an evaluation of qualitative information. Data should be used to ask questions and stimulate debate, but should not be the end in itself. For instance, poor results do not necessarily mean poor execution or negative impact, but could be a result of challenges that need to be addressed. Good performance evaluation systems should be transparent to ensure credibility; they must be simple for easy adoption and implementation, as well as cost-effective. These systems provide a valuable tool for management and continuous improvement; however, their use is valuable only for as long as the measures are a good approximation of reality and only to the extent that individuals have not tried to adjust the performance results.

Requirements of the Intellectual Property Rights Act of 2008

clear approach to intellectual property that arises from publicly financed research, enhance national capacity to manage intellectual property (especially intellectual property derived from publicly financed research) and strengthen the initiatives for the commercialisation of intellectual property.⁷

According to the R&D strategy, there is an urgent need for the creation of a proper framework and enabling legislation for the management of intellectual property arising from publicly financed research. This [legislation] will define the ‘playing field’ for publicly financed research and research that is undertaken in parastatal institutions.⁷

The Intellectual Property Rights from Publicly Financed Research and Development Act (IPR Act) was enacted in December 2008.⁸ The objective of this Act is to make provision that intellectual property emanating from publicly financed research and development is identified, protected, utilised and commercialised for the benefit (whether social, economic, military, or any other) of the people of South Africa. The purpose of this Act is, therefore, to inter alia:

• Provide for a more effective use of IP emanating from publicly financed research and development.

• Establish the National Intellectual Property Management Office and the Intellectual Property Fund.

• Provide for the establishment of institutional TTOs and other related matters.

In order to fulfil the above requirements, higher education and research institutions will be expected to establish M&E systems to track their IP management and technology transfer activities, as they will have to report to the National IP Management Office. Data collection and reporting is only half the work, because analysis of the reported data is necessary to identify trends and assess actual performance against predetermined targets or external benchmarks.

Functions of a TTO

Relationship between the TTO and other organisational units

Developing evaluation indicators

• an in-house TTO

• an independent company, wholly owned by the institution

• a function within a larger portfolio, such as the grants and contract research management office

• an external function, which is outsourced to a consulting or professional company

• a combination of the above.

Measures used to evaluate the performance of a TTO may be influenced by the operational model of the office and can be both quantitative and qualitative. Measures of performance comprise an evaluation of input measures, such as research expenditure, which is linked to the output of the TTO in the form of patents, licences, start-ups and so on. Measures of internal process efficiencies and performance would then be considered, followed by measures of the productivity of the research community in generating new IP and inventions. Measuring process efficiencies, whereby the speed with which cases received are cleared and the volume decreased, is a less developed area of IP management evaluation.

Inputs are resources required to initiate a programme of activity (e.g. people, funds and equipment), while activities are the actions required to achieve a given objective.¹ Certain IP management activities may take several years, for example, an application for an international patent may take a number of years to be granted. In such cases, it is sensible to develop in-process measures linked to the stages of the application process. Thus, as each examination stage is reached, the office is acknowledged for its accomplishment.

At the other end, one finds the immediate outputs, which are the observable products of the programme, such as the number of patents granted and licence agreements signed, as well as the outcomes – the long-term results of what a programme or process sets out to achieve. While an output could be the actual granting of a patent, an outcome would be the patenting success, based on the increased number of international patents granted or the extent to which a patent is successfully commercialised.

It is essential that performance measures promote appropriate behaviour among TTO staff and within the research community. Moreover, the objective of the implemented measures should support the institution’s IP policy and strategy. For instance, if the objective is to increase the number of patents granted at the institution annually, it would be counter-productive if the measure used to reward researchers is limited solely to scholarly publications, to the exclusion of patents. Therefore, an institution needs to consider the drivers of research output, in relation to the use of incentive models, in order to assess how organisational behaviour can be managed.

Mary Sue Coleman, President of the University of Michigan, summarises the ultimate role of most institutions as follows:

Many people are often confused about why we are interested in technology commercialisation, in nurturing start-up companies, and in facilitating more patents and licence agreements. It’s not about the promise of future revenues that might be generated from this activity. It is not about the money. Technology transfer must serve our core mission: sharing ideas and innovations in the service of society’s well being.¹⁰

It is therefore important for decision-makers not to focus solely on the potential monetary gains, but to look at the broader role of institutions in respect of knowledge transfer. Knowledge-sharing enables further research, leading to the development of new and improved processes, products and services. By facilitating this transfer, TTOs are instrumental in promoting innovation that ultimately benefits the public. TTOs provide a service to the academic community by creating, supporting and maintaining research partnerships with industry, as well as by promoting economic development and ensuring a fair compensation for institutional IP.¹¹ In essence, the role of the TTO is to expand the traditional knowledge that is generated and the publication-driven mission of the institution, to ensure that R&D results achieve at least one of the following outcomes:

• income generation for the institution

• service provision for the benefit of the research community

• economic development in relation to industry partnerships

• an increase in contract research and collaboration opportunities with industry and other institutions owing to perceived institutional strength

• jobs created through new start-up companies.

Challenges of applying the M&E system to IP management and technology transfer activities

The IP management and technology transfer environment presents its own unique set of challenges in terms of monitoring, evaluation and impact assessment. Firstly, there are long time lags associated with the different IP management and technology transfer processes. These raise the question of whether or not researchers should be rewarded for milestone achievements and what these milestone achievements should be.

Secondly, TTO managers often have limited direct control over certain activities, because the completion of these activities may be dependent on other collaborative or research partners. In addition, certain activities do not take place by design and are not always subject to pre-determined schedules. Under such circumstances, it is not easy to determine the contribution of the TTO.

Thirdly, institutions need to develop the appropriate capacity to develop M&E systems and processes, the expertise to carry out M&E functions correctly, the financial resources to support M&E activities, and representation throughout the institution in order to implement a robust M&E and impact assessment system.

Fourthly, it is not always possible to link cause and effect in the context of IP management, due to diverse external factors that influence management and technology transfer processes, for example, institutional and national policies and legislation that facilitate the transfer of knowledge and technologies to the market, thus contributing to the transfer outcomes and impact achieved.

Fifthly, it is a continual challenge to implement an M&E and impact assessment system that satisfies the needs of, and is relevant to, the wider group of IP stakeholders. The government or investors (both local and international) often want to be informed of the outcome and impact of the interventions they have funded and thus require, as a part of the funding conditions, that institutions implement an M&E assessment and reporting framework that reveals such outcomes and impacts when they occur. The results of a technology programme and its implementation thus impact:

• the organisation conducting the research

• the client or the organisation funding the work

• industry and society

• the country and region to which the technology has been transferred.

Finally, M&E frameworks need to adequately address data quality and evaluation methodology shortfalls that exist with all assessment methods.¹² The assessment methods used need to be appropriate and aligned with the objectives of the TTO being assessed. Qualitative assessment methods are generally more complex because they are subjective and focus on descriptions to describe symbols or experiences, which means that they are difficult to validate and, therefore, they can compromise credibility.

Understanding performance measures

Figure 2: Performance measurement cycle

Elements of M&E and impact assessment

Leadership support and commitment
M&E and impact assessments will not be properly done unless institutional leaders demonstrate a sustained level of support by committing resources to an assessment system, as well as by calling for assessments to be carried out and advocating their importance to the organisation.

A desire and culture for accountability
Institutions must be held accountable for the use of public funds. To this end, M&E and impact assessments should form an integral part of the research process and should not be viewed as an onerous responsibility in the daily research activities, but rather as one that fosters a culture of accountability.

A conceptual framework
It is important to realise that M&E and impact assessment is a process and not an event. Therefore, such a process must be structured in a way that all parties understand what is required of them and how they can use the outputs to benefit the institution.

Strategic alignment
An assessment framework must be linked to the goals and objectives of the TTO, which, in turn, are linked to the objectives of the overall research programme and, ultimately, the institution. Therefore, assessment indicators should generate a report that will enable evaluators to determine how well a given intervention is achieving strategic objectives.

Commitment to excellence
It is important that assessment processes and procedures are of the highest quality so as not to compromise the credibility of results; this requires a sustained commitment to excellence by all parties involved in these processes.

Knowledgeable and trained staff
Staff involved in M&E must be knowledgeable in M&E processes. They need to be trained to ensure that procedures are carried out correctly and that information transfer is effective.

Effective internal and external communication
In order for stakeholders to buy into a new M&E system, the benefits of, and the need for, such a system must be communicated to all relevant stakeholders through a well-prepared awareness campaign.

A reward-oriented culture
Using a punitive approach to ensure adherence to policies and procedures is less effective than encouraging and promoting innovation and performance through an incentive-based reward system.

Integrated data-processing systems
Integration of the data-processing systems used in M&E and impact assessment helps to ensure that those involved in M&E are able to carry out, record and report on any progress made in an efficient and effective manner.

Reasonable timeframes and adequate resources
It needs to be realised that assessments of any nature take time and require resources to perform; quality output can only come from quality input, particularly in terms of time and resources, and so these elements must remain integral to M&E and impact assessments.

Commitment to acting on recommendations
The purpose of any assessment is to determine whether a given intervention has the potential to succeed. If the recommendations of an assessment are not going to be considered at all, then the exercise and resources will have been for nought.

The practical use of M&E and impact assessments

The use of quantitative indicators is common when determining the performance of a TTO. Other approaches that have been used in this area include:

• peer review by an expert panel of assessors

• economic methods such as ratio and cost benefit analyses

• customer and beneficiary surveys

• case studies that provide qualitative information.

Table 1 provides a summary of common M&E methods and their strengths and weaknesses.^15,16

IP management and technology transfer measures

IP management indicators
The number of invention disclosures registered: An invention disclosure is the point where a researcher begins to share information on a potential discovery with a TTO, which, effectively, is the start of the technology transfer process. Typically, an invention disclosure will entail an evaluation of the technology, which will possibly lead to its protection and commercialisation. Measuring the number of disclosures is particularly important in situations where IP protection is not possible, or a decision is made not to protect the invention. Furthermore, this could indicate the extent of awareness of IP within an institution. In addition to counting the number of patents, the number of disclosures will reveal serial inventors and creators of IP. A well-structured invention disclosure process will not only capture patentable inventions, but non-patentable discoveries as well and can be used to track the participation of researchers in the technology transfer process. It is also useful to monitor performance by tracking the number of disclosures that are not only converted to patent applications or patents, but also those that are ‘converted’ into successful technology transfer opportunities and ultimately applied in industry and society.

The number of provisional patent applications: This measure could be a proxy for future patenting potential. A provisional patent application allows for patent filing without any formal patent claim, oath or declaration, or any information disclosure (prior art) statement. This measure provides the means to establish an early, effective filing date in a non-provisional patent application and allows the term ‘patent pending’ to be applied.

The number of South African patent applications granted: The South African patent registration office does not subject patent applications to substantive examination in respect of novelty, inventiveness or utility and, therefore, the weighting placed on this measure needs to be contextualised. Institutions interested in promoting a culture of registering and protecting IP may want to use this measure to encourage and educate the research community. Once researchers are familiar with the processes and practices of IP protection, institutions may decide to apply measures that hold greater value, such as patents granted by examining countries.

The number of international patents, trademarks, registered designs and plant-breeders’ rights granted: These are universal measures used as an indication of the productivity of the private and public sector research environment.

The number of Patent Cooperation Treaty applications submitted: Given that it may take 18 months or longer from the earliest priority date to publication of a patent application, it is often useful to include in-process measures that will enable evaluators to track progress and identify problem areas well in advance. Measuring the number of Patent Cooperation Treaty (PCT) applications thus enables evaluators to track the number of patent applications and assess how many of these are eventually granted.

The number of international preliminary reports on patentability issued by the International Bureau: The objective of the international preliminary reports on patentability is to formulate a preliminary and non-binding opinion on the patentability of an invention. The report is issued after a search and analysis of the claims covering the invention and provides a preliminary opinion on the novelty, non-obviousness and industrial applicability of the invention. A positive response could be considered an early indicator for patentability; however, a considerable number of negative reports must be investigated, as this could suggest a problem with the quality of the specifications or claim drafting of the PCT applications submitted. A negative response could also provide a basis for the amendment of claims to remove the objections by the examiners.

The number of publications, conference papers and the like that are accepted: Traditionally, TTOs are not responsible for tracking the number of publications published, even though they fall within the broader sphere of knowledge output. Publications are an important output measure, especially for academic institutions, and their inclusion in a measuring framework is an important element in assessing transfer of knowledge to society and increases the scope of IP measures.^17,18,19,20 This calculation, though, is quite complex, with weightings assigned to different types of publication. For example, an entire book will receive a different weighting in relation to a book chapter or a journal paper. Generally, only peer-reviewed publications or a paper published in the approved list of journals are considered.

Patent citations: These are references to prior technology, either in the form of patent applications, patents or other scientific literature, which the current patent builds upon or uses.²¹ Patent citations are increasingly used as a measure of patent quality and innovation performance. When filing a patent in the United States of America, it is a legal requirement for applicants to supply a complete list of prior articles and, so rather than run the risk of having their patent revoked, applicants quote every reference, even if it is only remotely related to what is being patented.²² For this reason, the US Patent and Trademark Office has significantly more patent citations than any other patenting office, including the European Patent Office. While, patent citations are used to indicate the relative strength of a patent, this measure does not take into account the fact that citations originate from patents of different qualities.

Technology transfer and commercialisation indicators
Financial indicators: The amount of money generated by a TTO is a common indicator of commercialisation performance. This is income obtained from royalty payments or other licence fees (e.g. upfront and milestone payments), sale of intellectual property, research contracts secured as a result of a licensing agreement or linked to existing IP within the institution. This is an output measure used to determine investment into research, development and commercialisation. It also provides an indication of the value of the technology in the market and the level of success with respect to knowledge flow. When assessing financial indicators, the following factors need to be taken into account:

• Revenues received as a result of IP-induced research contracts; there is a correlation between the quality and quantity of an institution’s knowledge output and the funding it receives in the form of sponsored research.²³ A strong IP portfolio in a given domain will attract the attention and interest of industry and other researchers. At the same time, it is also important to note that the conditions placed on sponsored research can influence the institution’s ability to patent and publish knowledge generated through such contracts. Such conditions may limit patenting opportunities and therefore influence performance results.

• Revenues received from the sale, licensing and/or assignment of IP rights.

• The source/s of revenue, because, in certain cases, revenues from highly regarded sources raise the profile of the institution and can be a sign that the quality of research and innovation is of a high calibre.

Non-financial indicators: The actual outputs and outcomes of technology transfer activities are equally important and should be reviewed regularly. The non-financial indicators selected to measure productivity in terms of knowledge and technology transfer should be in line with the general strategies and policies set by the institution. If licensing is of strategic importance to an institution, one would ensure that a count of the number of licenses signed in a given period is included in the performance review of that TTO. Therefore, a pragmatic approach is needed when selecting outputs and outcomes to be included in an M&E system, taking into account the following:

• The number of new licences, assignments, donations and options signed; each indicator in this group is a measure on its own and could be divided according to the number of signed and non-income yielding agreements.

• The number of new start-up companies (new companies) and spin-out companies (those formed from an existing parent company).

• The number of new products, processes or services introduced into the market for both industrial and social good applications.

• The number of technology transfer opportunities under negotiation.

• The deal flow or stream of technology transfer projects, which provide an indication of the size, sector, complexity and quantity of projects managed by the TTO. An analysis of this information should reveal which resource and capacity issues need attention.

• The number of new collaboration programmes and contract research opportunities as a result of IP generated by the institution.

Internal processes: It is not often that institutions use measures of internal process performance in M&E systems. The main reason for this is that internal performance measurement is the least developed area in IP management. Thus said, institutions are beginning to take notice of aspects of internal performance measurement and, in an informal way, estimate how well or badly the TTO is doing in some of the following areas:

• The case elimination⁹ process requires that a well-resourced TTO should demonstrate efficiency in terms of eliminating the number of projects or negotiations to which it is attending. A stage-gate process can be applied where unpromising cases can be abandoned early in the technology transfer process in favour of more promising opportunities, before the office wastes critical resources or incurs excessive costs. This approach enables evaluators to assess the number of cases that pass and fail the technology transfer process.

• Measuring the number of iterations with respect to drafting and reviewing of contracts and key documentation considers the number of revisions required for documents and the time spent on each revision. However, the value of this measure, in terms of efficiency, is questionable due to the complexities of each case being assessed.

• Assessing case load queue times refers to an evaluation of the number of projects a TTO is handling at a given point in time. The turnaround time of documents submitted to the TTO is important in order to identify blockages in the process, capacity problems and inefficiencies.

Indicators that measure process efficiency focus mainly on managing disclosed technologies and IP and thus can be challenging to implement, unless the TTO has an effective administration system for processing large volumes of documents. Where multiple people are involved in reviewing and processing information, a system is required that allows each participant to register their input and be held accountable for their component of the work.

Institutions need to determine the relevance of the various indicators for their own particular use. They can then substitute, add or discard indicators accordingly.²⁴ In this regard, benchmarking can be most useful in cases where institutions use common indicators. However, benchmarking needs to be approached with caution because one of its common shortcomings is when institutions assume that there are pre-existing benchmarks for everything they wish to measure. It is therefore necessary to investigate and verify the measures being benchmarked. The greatest advantage of a benchmarking exercise is that it gives an institution a reference on the possibilities available in respect to IP management and technology transfer performance. In addition, benchmarking assists leaders in assessing exactly how much improvement is required to achieve superior performance.

Monitoring and evaluation tools

Assessing the impact of technology transfer

• The use of a new technology as a product, process, service, et cetera.

• Widespread diffusion of a new technology, such that it is accessible and available to a large number of beneficiaries

• Application of the technology, whether at a test site or in actual practice.

In developing an impact assessment survey, it is important to define the nature of the technology and changes that are to be analysed. It is also essential to identify the audience for the assessment. The resources, capacity, geographical spread and available time will determine the design, size and intensity of the survey, thus, during the planning phase, evaluators should determine how and what type of impact will be measured. The greatest challenge when assessing impact is to demonstrate a link between changes in society and industry and the technology and knowledge transfer intervention. In the long term, evaluators should work towards building quality into the performance system and process. As such, key steps in developing an impact assessment plan are to:

• Identify the need or problem.

• Define the objectives for the assessment.

• Develop the impact measures.

• Analyse and interpret the results.

• Report on findings and develop policy or strategic options.²⁷

Case studies, as an impact assessment method, provide an opportunity to ‘tell the story’ and, in so doing, communicate qualitative information, in addition to the quantitative facts and figures. Qualitative and in-depth case studies can thus be used to share economic achievements and entrepreneurial successes on technical and scientific innovations.²⁸ Well-crafted case studies add the human element to technology transfer activities and demonstrate commitment to making an impact. An example of the use of case studies is the AUTM Better World Project, which shares inspiring stories behind the innovations that have changed the way we live. The initiative draws from years of case studies and news from AUTM members.²⁹

Social impact assessments

Social impact assessment should not be understood only as the task of predicting social impacts in an assessment process. It includes the processes of analysing, monitoring and managing the intended and unintended social consequences, both positive and negative of planned interventions (policies, plans, projects) and any social change processes invoked by those interventions. Its primary purpose is to bring about a more sustainable and equitable biophysical and human environment.³⁰

Useful guides on social impact assessment have been developed by the IAIA and other institutions, such as the World Bank, the Asian Development Bank, the African Development Bank and the European Commission.^31,32,33,34

M&E and impact assessment are complex activities that need to be planned and implemented well to benefit an institution, its members and stakeholders. When M&E and impact assessment processes, procedures and systems have been well established, they can be highly effective in assisting institutions to perform more efficiently and make better decisions. No single best practice exists in this regard; however, as institutions develop their IP management and technology transfer capacities to the next level, they need to engage in M&E and impact assessment practices to ensure that they are achieving the desired objectives for investment in an area that, in itself, is complex, multidimensional and characterised by long-term activity.

1. US Department of Energy, Office of Policy and Office of Human Resource and Administration. Guidelines for performance measurement [document on the Internet]. c1996 [2009 Sep 16]. Available from: http://www.directives.doe.gov/pdfs/doe/doetext/neword/120/g1201-5.pdf

2. Tektique [homepage on the Internet]. c2010 [2010 Feb 8]. Available from: http://www.tektique.co.za/aboutus.aspx

3. Sibanda M. Framing a national policy and legislation for managing IPR resulting from publicly financed research and development [document on the Internet]. c2009 [2010 Feb 8]. Available from: http://www.wipo.int/edocs/mdocs/tk/en/wipo_ip_lss3_08/wipo_ip_lss3_08_www_114635-main1.pdf

4. Sibanda M. The economics of intellectual property in South Africa. Publication No. 1013(E). Geneva: World Intellectual Property Organization, 2009; p. 113–145.

5. Otsuyama H. Patent valuation and intellectual assets management. In: Samejima M, editor. Patent strategy handbook. Tokyo: Chuokeizai-sha, 2003; p. 175.

6. Jayasekeram PA. Monitoring, evaluation and impact assessment. Paper presented at: Planning, monitoring, evaluation and impact assessment of R&D investment in agriculture workshop; 2006 Oct 16–27; Alemaya University, Ethiopia.

7. Department of Science and Technology. South Africa’s national research and development strategy [document on the Internet]. c2002 [2009 Sep 11]. Available from: http://www.dst.gov.za/publications-policies/strategies-reports/strategies-reports

8. Intellectual Property Rights from Publicly Financed Research and Development Act No. 51 of 2008, South Africa [statute on the Internet]. c2008 [2010 Feb 07]. Available from: http://www.naci.org.za/pdfs/workshops/julNBAC/Act%20No.%2051%20of%202008%20-%20Intellectual%20Property%20Rights%20from%20Publicly%20Financed%20Research.pdf

9. Auril/Universities UK. Managing intellectual property, the guide: A guide to strategic decision-making in universities [document on the Internet]. c2002 [2009 Sep 17]. Available from: http://www.ipo.gov.uk/managingipguide.pdf

10. Association of University Technology Managers (AUTM), USA. AUTM US licensing survey, FY 2004 survey summary [document on the Internet]. c2005 [2009 Sep 17]. Available from: http://www.autm.net/AM/Template.cfm?Section=Licensing_Surveys_AUTM&Template=/TaggedPage/TaggedPageDisplay.cfm&TPLID=6&ContentID=2409

11. Louis C. Measuring the impact of University of California technology transfer: Aligning metrics with goals [document on the Internet]. c2009 [2009 Sep 10]. Available from: http://www.aplu.org/NetCommunity/Document.Doc?id=1739

12. Jordan G, Malone E. Performance assessment [document on the Internet]. c2006 [2009 Sep 18]. Available from: http://www.au.af.mil/au/awc/awcgate/doe/benchmark/ch06.pdf

13. Hoffman C, Cullen K, Croft K, Stevens A. Benchmarking technology transfer metrics from different countries [document on the Internet]. c2008 [2009 Sep 20]. Available from: http://www.autm.net/AM/Template.cfm?Section=Licensing_Surveys_Affilated_Organizations&Template=/CM/ContentDisplay.cfm&ContentID=2152

14. Gardner F, Fong A, Huang R. Measuring the impact of knowledge transfer from public research organizations: A comparison of metrics used around the world [document on the Internet]. C2007 [2009 Sep 20]. Available from: http://www.triumf.info/public/repository/ttb/ttr20070603.pdf

15. International Service for National Agricultural Research (ISNAR). Monitoring, evaluation, and impact assessment of R&D investments in agriculture. The Hague: ISNAR; 2003.

16. Capron H, Pottelsberghe de la Potterie B. Public support to R&D programmes: An integrated assessment scheme [document on the Internet]. c1997 [2009 Sep 18]. Available from: http://www.oecd.org/dataoecd/34/37/1907902.pdf

17. Department of Education. Policy procedure for measurement of research output of public higher education institutions [document on the Internet]. c2003 [2010 Jan 18]. Available from: http://docs.google.com/viewer?a=v&q=cache:dm5kN_j1ZqwJ:www.education.gov.za/Documents/policies/PolicyMeasurementResearch.pdf+Publications+as+a+measure+for+the+quality+of+research+output&hl=en&gl=za&sig=AHIEtbQCp43OSNOFmOw2tj3aUz706uaSyA

18. Charlton BG, Andras P. Oxford University’s research performance – Publication and citation metrics 1975–2004 [homepage on the Internet]. c2006 [2010 Jan 18]. Available from: http://www.hedweb.com/bgcharlton/oxmagarts.html

19. Heale JP, Shapiro D, Egri CP. The determinants of research output in academic biomedical laboratories. Int J Biotechnol. 2004;6:134–154.

20. Hall B. Patents as indicators [document on the Internet]. c2004 [2009 Sep 11]. Available from: http://www.wipo.int/meetings/en/2004/statistics_workshop/presentations/statistics_workshop_hall.pdf

21. Boaden RJ, Cilliers JJ. Quality and the research assessment exercise: Just one aspect of performance. Qual Assur Educ. 2001;9:5–13.

22. Michel J, Bettels B. Patent citation analysis: A closer look at the basic input data from patents search reports. Scientometrics. 2001;51:185–201.

23. National Science Foundation, Division of Science Resources Statistics. Survey of federal science and engineering support to universities, colleges and non-profit institutions [document on the Internet]. c2009 [2009 Sep 17]. Available from: http://www.nsf.gov/statistics/nsf09315/pdf/nsf09315.pdf

24. Muir A. The technology transfer system. New York: Latham Book Publishing, 1997; p. 154–168.

25. Federal Laboratory Consortium Mid-Atlantic Region. Technology transfer electronic tools [document on the Internet]. c2007 [2009 Sep 20]. Available from: http://www.flcmidatlantic.org/pdf/2007/tt_electronic_tools.pdf

26. Intellectual Property Software Resources [homepage on the Internet]. c2010 [2010 Jan 18]. Available from: http://www.ipmenu.com/ipsoftware.htm

27. European Union. Common approach to impact [document on the Internet]. c2005 [2009 Dec 8]. Available from: http://ec.europa.eu/governance/impact/docs/key_docs/sec_2005_0791_en.pdf

28. Jarret JE. Impact assessment of the advanced technology program [document on the Internet]. c2005 [2009 Sep 20]. Available from: http://www.ic2.org/publications/bbr_atpi_exsum.pdf

29. AUTM. Better world report part one – Building a stronger economy: Profiles of 25 companies rooted in academic research [homepage on the Internet]. c2009 [2009 Sep 17]. Available from: http://www.betterworldproject.org

30. International Association for Impact Assessment. Social impact assessment: International principles [document on the Internet]. c2003 [2009 Sep 16]. Available from: http://www.iaia.org/modx/assets/files/SP2.pdf

31. The World Bank. A user’s guide to poverty and social impact [document on the Internet]. c2003 [2010 April 16]. Available from: http://www.iaia.org/modx/assets/files/SP2.pdf

32. The Asian Development Bank. Handbook on poverty and social analysis [document on the Internet]. c2001 [2010 April 16]. Available from: http://www.adb.org/Documents/Handbooks/Poverty_Social/Section3.pdf

33. African Development Bank. Integrated environmental and social impact assessment guidelines [document on the Internet]. c2001 [2010 April 16]. Available from: http://www.afdb.org/fileadmin/uploads/afdb/Documents/Policy-documents/Integrated%20Environmental%20and%20Social%20Impact%20Assesment%20Guidelines.pdf

34. The European Commission. Guidance for assessing social impacts within the commission impact assessment system [document on the Internet]. c2009 [2009 Dec 8]. Available from: http://www.siteresources.worldbank.org/INTPSIA/Resources/490023-1121114603600/12685_PSIAUsersGuide_Complete.pdf