Given the persistent pattern of low achievement scores for students from low-income households, the research and policy challenge is how to improve the schooling system to break this cycle of poor achievement in mathematics, as well as in other problem areas, namely, languages and sciences. We need to move beyond the legacy questions to understand why, despite many efforts of government and other key role players, we have not been successful in improving these educational outcomes. This analytic–descriptive study reports on an analysis of students’ pathways and performances in the high school phase.

To create the sample population, we used a unique panel-like data set which identified a group of students who participated in 2002 in the Grade 8 Trends in International Mathematics and Science Study (TIMSS 2002) and were also in the Grade 12 (matriculation) examinations data set. This group provided a good example of a longitudinal data set with achievement scores at both the Grade 8 and Grade 12 levels. Using these two data sources, it was possible to examine the associations between Grade 8 mathematics performance in TIMSS and the selection of mathematics as a matric subject, Grade 12 mathematics performances, and patterns of passing matric.

The South African school system can be seen as being made up of two historically and persistently differently functioning subsystems,^4,5,6 and it is appropriate to disaggregate the data and outputs for these two subsystems. The study categorises the schools which largely serve students from poorer homes as Subsystem P and those which largely serve middle-class students as Subsystem M. Subsystem P schools, which are the majority (80%), refer to schools which historically served Black African students in South Africa during the apartheid era. These schools were provided with the fewest resources and still bear the scars of that legacy; they are located in areas occupied by low-income households. These schools cater for a majority of students for whom the language of instruction (English) is their second or third language. By contrast, schools which were categorised as Subsystem M schools were historically attended by White and Indian students (we use the race terms in this article to reflect the historical resourcing patterns to different groups). These schools were better resourced under apartheid, are generally located in higher-income areas, and the majority of their students study in English as their first language. These schools constitute about 14% of the schools in the country. Because of their heterogeneity, and because they did not fit well into the two categories, for this analysis we ignored the schools that were attended historically by Coloured students.

Since the 1970s, the availability of large-scale representative data sets, with information on student academic achievement, has provided an opportunity to undertake statistical or econometric analyses that can identify variables subject to policy control which can influence student cognitive achievement. Analyses to identify these key determinants using education production–function models^7,8,9,10 or school effectiveness or school improvement models^11,12,13,14 for a quality education, have been undertaken in both high-income and low-income country contexts.

Although these studies have advanced our understanding of the education process, there are limits to how they can advance our understanding of the determinants to improve education outcomes. These limitations apply even in high-income countries with good quality data and less extreme educational inequality, and where the basic inputs are in place.¹⁵ In developing country contexts, where achievement scores are low and where the basic educational components are not in place, results from these statistical analyses are sometimes less informative regarding what could make a difference in achieving quality education.¹⁵ There are limitations to statistical analyses and modelling from large achievement data sets: they can only show a ‘snapshot’ of a school at any particular time¹⁶; there is a lack of agreement on the determinants of educational quality¹⁵; there is a danger of reductionism and invalid specification of causality inherent in school effectiveness studies^17,18; the determinants of improved education quality identified are more effective in already well-performing schools¹⁹; and there is a concern that these studies do not deal adequately with social class in the analysis.¹² Extending the analysis, especially in low-income countries, from a snapshot to a longitudinal framework, provides possibilities for a different insight on how to improve achievement outcomes.

There have been only a few investigations of the pathways and performances of school students over time. However, with an increasing number of large and representative panel data sets, a body of literature is developing. Panel studies have been undertaken on student pathways in relation to progression through an educational system^20,21,22; to educational and career aspirations^{23,24,25,26,27}; to aspirations, performance and transition from school to post-school institutions; and to labour markets and employment.^{10,28,29,30,31,32} The few panel studies that have included cognitive or academic performance data have been used to analyse academic performance patterns over time, thus predicting patterns of future performance.^{33,34,35,36,37}

The main findings that emerged from the review of panel data sets which included cognitive and achievement data were, firstly, that performance in earlier years predicts later performance; and, secondly, that gaps in cognitive ability emerge during early childhood as a consequence of differences in family background and, over time, these gaps widen.^34,36,38 Thirdly, children with educated and wealthy parents who score poorly in the early tests tend to catch up, whereas children with lower educated and lower-income parents who score poorly are unlikely to catch up.^33,35

The early years of one’s life are an important phase for promoting cognitive development and the acquisition of foundational knowledge and skills. Many poor children fail to reach their potential cognitive development because of deficiencies in their early development. Heckman³⁸ reports that, in the USA:

going across income groups, gaps in cognitive ability emerge early in the life cycle and widen slightly in the early years of schooling. They stay constant after the age of eight, and school environments play only a small role in accounting for, narrowing or widening the gaps.

In Britain, Feinstein³³ found that pre-school development tests provided a strong indication of a child’s later educational success and that this success was largely attributable to family background. Children with educated and wealthy parents who scored poorly in the early tests tended to catch up, whereas children with lower educated and lower-income parents who scored poorly were unlikely to catch up, and were an at-risk group. A subsequent study by Blanden and Machin³⁵ corroborated these findings. Children born in 2000 to the lowest income households who had scored some of the best results in tests at age three had, by the age of five, lost much of their early advantage. By age seven, these youngsters were overtaken by children from the wealthiest homes who were bottom in the tests at age three. The gap in the average percentile ranking in the tests between high achieving children from poor backgrounds and low achieving children from affluent backgrounds had shrunk from more than 70 percentiles at age three to 20 percentiles by age five.

In the unequal South African school system, the rate of grade progression is considerably higher amongst students within historically White schools (Subsystem M) than amongst those in historically Black schools (Subsystem P). For example, Lam et al.³⁷ found that 84% of White students who were in Grades 8 and 9 in 2002 successfully advanced three grades by 2005 compared with only 32% of Black African students. Furthermore, Lam et al.³⁷ demonstrated that grade progression in the schools typically attended by Black students was poorly linked to actual ability (as measured by assessment items) and learning. They found that baseline literacy and numeracy scores strongly predicted grade progression between Grades 8 and 11 for White and Coloured students, but weakly predicted progression for Black students. In contrast, no racial differences were found in the relationship between baseline scores and passing the matric examination, which is nationally standardised. They therefore propose that grade progression within schools attended by Black children is characterised by a considerable degree of randomness, with the consequence of high enrolment despite high rates of failure.

We know little about the patterns of cognitive development and mathematical performance over time. Given the limited literature in South Africa that uses panel data to track student academic performance, this study adds to the literature by using mathematics achievement data from two different time periods in two grades, as well as aggregate performance data from Grade 12, and analyses the pathways (subject choices) and performance patterns of students in school.

South Africa participated in the TIMSS in 2002. The study collected mathematics and science achievement data from 8952 Grade 8 students in the country⁵ and created a record of the name, date of birth and school attended in 2002 for each student who participated in the study. Grade 12 is the last year of schooling, during which students sit for a public, common examination (called matric). For the purposes of this study, the Department of Education allowed access to the matriculation 2006 and 2007 databases; these were searched for TIMSS 2002 participants. A total of 2734 (30.1%) unique student records from the TIMSS 2002 Grade 8 data set were found in the matric 2006 and 2007 data sets (repeaters were found in both the 2006 and 2007 matric databases). The General Household Surveys (GHS) of 2005 and 2006 calculated the progression rate from Grade 8 to Grade 12 as approximately 57%. This rate provides an indication of how many students were mistakenly not tracked to matric as a result of the imperfect matching process. Similarly, expected progression rates for each race group in our data were obtained from the GHS data. These progression rates were then used to weight up those students identified in matric and to weight down those students not identified in matric. This weighting was done separately for each race group. The weighting procedure ensured that the proportions within our matric sample were broadly representative of the entire population of matriculants in South Africa. In our subsequent analysis, where appropriate, we present the weighted information.

We used the TIMSS 2002 and matric mathematics scores as the proxy measure of analytical skills. These analytical skills are highly valued and are important for an individual’s personal, social and economic development. Students who were in Grade 8 in 2002 made a range of subject choice selections and could have traversed one of four pathways: not continued with schooling after Grade 8; continued to Grade 12 without mathematics; continued to Grade 12 with mathematics at standard grade; and continued to Grade 12 with mathematics at higher grade.

FIGURE 1: Kernel density of the Trends in International Mathematics and Science Study (TIMSS) mathematics scores by identification and matric year.

FIGURE 2: Kernel density of the Trends in International Mathematics and Science Study (TIMSS) mathematics scores by school subsystem and identification.

The analysis was extended to examine the patterns of TIMSS score for those who ‘passed matric’ and those who ‘did not pass’, in Subsystem P and Subsystem M schools (Figure 3). There was a high degree of overlap of the TIMSS scores between those who ‘passed matric’ and those who ‘did not pass matric’ in Subsystem P schools. The mean TIMSS scores were extremely low (226) for those who did not pass matric and 261 for those who passed matric. There was thus a small difference of 35 points between the two groups. In the Subsystem M schools, there was a higher degree of differentiation. The mean TIMSS score was 324 for those who did not pass matric and 444 for those who did pass (Table 3). There was thus a sizeable difference of 120 points between the two groups.

FIGURE 3: Kernel density of the Trends in International Mathematics and Science Study (TIMSS) mathematics scores by school subsystem and matric passing.

FIGURE 4: Conversion of the Trends in International Mathematics and Science Study (TIMSS) mathematics achievement to matric pass rates by school subsystem and decile of TIMSS scores.

FIGURE 5: Kernel density of the Trends in International Mathematics and Science Study (TIMSS) mathematics scores by school subsystem and mathematics selection in Grade 12.

As noted in Table 1, matric mathematics performance and TIMSS mathematics performance were low. The relationship between the average matric mathematics mark and TIMSS mathematics scores is illustrated by a plot of these two sets of scores by the TIMSS deciles into which the student scores fall (Figure 6). Although low, the average matric mathematics mark increased in higher TIMSS deciles, and there was a strong correlation between Grade 8 TIMSS and Grade 12 matriculation mathematics performance. Thus the TIMSS Grade 8 mathematics mark strongly correlated with the mathematics performance in Grade 12.

FIGURE 6: Average matric mathematics mark by performance in the Trends in International Mathematics and Science Study (TIMSS), showing 95% confidence intervals.

Grade 8 mathematics scores are a good indicator of analytical capabilities, and one would expect that those with higher mathematics scores would have progressed to Grade 12 and achieved success in the Grade 12 examinations. The expectation would also be that their subject choices in the senior secondary level would have included mathematics at the higher-grade levels, and that those with better TIMSS mathematics performance would have achieved higher matric mathematics scores.

The findings of the study indicate, firstly, that educational achievement in South Africa, measured by TIMSS mathematics scores, is extremely low. The participation, performance and progression rates in Subsystem M and Subsystem P are significantly different, with Subsystem M students performing at a higher level than those in Subsystem P.

Secondly, we found that students starting with similar TIMSS Grade 8 mathematical scores may have quite different educational outcomes 4 years later. Grade 8 mathematics scores appear not to predict who will or will not reach matric, although this result may at least partly be attributable to our not being able to successfully identify all those who actually reached matric. However, Grade 8 mathematics scores are a good indicator of who can pass matric in Subsystem M schools. For Subsystem P schools, although the higher TIMSS scores can predict who has a higher probability of passing matric examinations, this relationship is not as strong. Students who come to secondary school with high Grade 8 mathematics scores, whether from Subsystems M or P, are able to convert to passing matric. For those in the middle bands of performance, the rate of conversion is different in the two subsystems, with Subsystem M achieving higher rates of conversion than Subsystem P schools. A surprising finding was that, in Subsystem P schools, one in five students (20%) whose TIMSS score was in the lowest four deciles was nevertheless able to convert that low demonstrated capability into passing matric.

Thirdly, for Subsystem M schools, TIMSS Grade 8 scores are a good sorter for the choice of matric mathematics as a subject, but, for the majority in Subsystem P schools, the subject choice of mathematics has little to do with earlier mathematics performance in TIMSS. Many students with weak TIMSS scores have high aspirations for participation and performance in mathematics, and, even with low scores, register for higher-grade rather than for standard-grade mathematics.

Lastly, there is a high correlation between the mean Grade 8 mathematics score and the matric mathematics scores, with this correlation being higher in Subsystem M schools than in Subsystem P schools. Students with higher Grade 8 mathematics performance scores tend to achieve success in matric mathematics. However, it would seem that for students who have low mathematics scores in Grade 8, schooling cannot provide the necessary inputs to overcome their low mathematics scores achieved in earlier grades and cannot improve their mathematical competencies.

The strong relationship between Grade 8 and Grade 12 mathematics scores corroborates findings in the literature that earlier mathematics performance and strong foundational knowledge form the base for subsequent learning. Analytical skills in mathematics need to be built up from the early years. Mathematical knowledge is hierarchical in nature, and strong prior knowledge is therefore critical for conceptual development. The acquisition of these capabilities is shaped in the early years by the nature and quality of interactions in the home and community, and by the quality of inputs from the school.

In Subsystem P, the progression from Grade 8 to Grade 12 does not fit the expected pattern, that is, that those with high Grade 8 mathematics scores will reach and pass matric and those with lower mathematics scores may not do so. Students starting with similar mathematics scores at the Grade 8 level may have different educational outcomes 4 years later. The reason why students with low TIMSS mathematics scores from poorer schools pass at matric level may be that TIMSS mathematics scores are not an adequate indicator of requirements for passing matric, or that students with weaker mathematics background are nonetheless successful in passing matric because of better performance in other subjects. Educational investments made post Grade 8 may enable students to improve their performance in subjects besides mathematics, and to pass matric despite failing mathematics.

The pathways of students post Grade 8 in Subsystem P schools, that is, whether or not they select mathematics as a subject, shows that there is little relationship between demonstrated ability and choice of subjects. Students do not seem to be using information about their prowess in mathematics to make appropriate subject choices, perhaps because they do not receive enough accurate feedback at school about their mathematics performance.

The policy implication from these findings is that raising the mathematics scores at Grade 12 requires raising the scores at Grade 8. Extrapolating from this, and linking to the literature on cognitive development, we need to raise the mathematics and numeracy scores in the earlier years of schooling. High levels of attention paid to the early years of learning (reception year and foundation phase) for children from environments of lower household and parental resources would contribute to breaking the cycle of poor academic performance. Without this, both the background and school will continue to let the children down and the reproduction of inequality will continue. Students must know and understand earlier concepts; only when they do understand these early concepts, will they progress. We have shown that by the time students reach the secondary level, it is too late to significantly improve matric mathematics performance.

Ideally, the study would have used data of cognitive scores from the early years and tracked the cohort to later years, but as the only available cohort cognitive performance data is for Grades 8 and 12, only the relationship between Grades 8 and 12 could be examined. How cognitive development is shaped, in mathematics and in other subjects, can be assisted by panel studies research, by collecting data from the earlier years of schooling, and by paying greater attention to obtaining cognitive data. These issues should therefore be on the education research agenda for future studies.

2. Organisation for Economic Co-operation and Development (OECD). Review of national policies for education: South Africa. Paris: OECD; 2008.

3. Reddy V. State of mathematics and science education: Schools are not equal. Perspect Educ. 2005;23(3):125–138.

4. Reddy V. Mathematics and science achievement at South African schools in TIMSS 2003. Pretoria: HSRC Press; 2006.

5. Van der Berg S. Apartheid’s enduring legacy: Inequalities in education. J Afr Econ. 2007;16(5):849–880. http://dx.doi.org/10.1093/jae/ejm017

6. Fleisch B. Primary education in crisis: Why South African schoolchildren underachieve in reading and mathematics. Cape Town: Juta & Co; 2008.

7. Simmons J, Alexander L. The determinants of school achievement in developing countries: A review of the research. Econ Dev Cult Change. 1978;26(2):341–357. http://dx.doi.org/10.1086/451019

8. Fuller B. What school factors raise achievements in the third world? Rev Educ Res. 1987;57(3):255–292. http://dx.doi.org/10.3102/00346543057003255

9. Van der Berg S, Louw M. South African student performance in regional context. In: Bloch G, Chisholm L, Fleisch B, Mabizela M, editors. Investment choices for South African education. Johannesburg: Wits University Press, 2008; p.49-69.

10. Bhorat H, Oosthuizen M. Determinants of Grade 12 pass rates in the post-apartheid South African schooling system. J Afr Econ. 2008;18(4):634–666. http://dx.doi.org/10.1093/jae/ejn027

11. Heyneman SP, Loxley WA. The effect of primary school quality on academic achievement across twenty-nine high and low-income countries. Am J Sociol. 1983;88(6):1162–1194. http://dx.doi.org/10.1086/227799

12. Glewwe P, Grosh M, Jacoby H, Lockheed M. An eclectic approach to estimating the determinants of achievements in Jamaican primary education. World Bank Econ Rev. 1995;9(2):231–258. http://dx.doi.org/10.1093/wber/9.2.231

13. Thrupp M. Recent school effectiveness counter-critiques: Problems and possibilities. Brit Educ Res J. 2001;27(4):443–457. http://dx.doi.org/10.1080/01411920120071452

14. Lee VE, Zuze TL, Ross KN. School effectiveness in 14 sub-Saharan African countries: Links with 6th graders’ reading achievement. Stud Educ Eval. 2005;31:207–246. http://dx.doi.org/10.1016/j.stueduc.2005.05.011

15. United Nations Educational, Scientific and Cultural Organization (UNESCO). Education for all, the quality imperative, EFA global monitoring report 2005. Paris: UNESCO; 2004.

16. Luyten H, Visscher A, Witziers B. School effectiveness research: From a review of criticisms to recommendations for further development. Sch Eff Sch Improv. 2005;16(3):249–279. http://dx.doi.org/10.1080/09243450500114884

17. Goldstein H, Woodhouse G. School effectiveness research and educational policy. Oxford Rev Educ. 2000;26(3):353–363. http://dx.doi.org/10.1080/713688547

18. Wrigley T. ’School effectiveness’: The problem of reductionism. Brit Educ Res J. 2004;30(2):227–244.

19. Yu G. Research evidence of school effectiveness in sub-Saharan Africa: EdQual Working Paper No. 7. Bristol: EdQual RPC; 2007.

20. Robinson R. Pathways to completion: Progression through a university degree. High Educ. 2004;47:1–20. http://dx.doi.org/10.1023/B:HIGH.0000009803.70418.9c

21. Dalton B, Glennie E, Ingels SJ, Wirt J. Late high school dropouts: Characteristics, experiences, and changes across cohorts, descriptive analysis report, 2009. Washington DC: NCES; 2009.

22. Ginsburg C, Richter LM, Fleisch B, Norris SA. An analysis of associations between residential and school mobility and educational outcomes in South African urban children: The Birth to Twenty Cohort. Int J Educ Dev. 2011;31(3):213–222. http://dx.doi.org/10.1016/j.ijedudev.2010.03.006

23. Braddock JH II, Dawkins MP. Ability grouping, aspirations, and attainments: Evidence from the National Educational Longitudinal Study of 1988. J Negro Educ. 1993;62(3):324–336. http://dx.doi.org/10.2307/2295468

24. Khoo ST, Ainley J. Attitudes, intentions and participation, LSAY research report No. 41. Melbourne: ACER; 2005.

25. Beutel AM, Anderson KG. Race and the educational expectations of parents and children: The case of South Africa. Sociol Quart. 2008;49(2):335–361.  http://dx.doi.org/10.1111/j.1533-8525.2008.00118.x

26. Cosser M. Studying ambitions: Pathways from Grade 12 and the factors that shape them. Pretoria: HSRC Press; 2009.

27. Reddy V, Bantwini B, Visser M. Youth into science strategy tracking studies’ report. Report commissioned by the Department of Science & Technology. Pretoria: DST; 2009.

28. Thomson S. Pathways from school to further education or work: Examining the consequences of Year 12 course choices. LSAY Research Report No 42. Melbourne: ACER; 2005.

29. Cosser M, Sehlola S. Ambitions revised: Grade 12 learner destinations one year on. Pretoria: HSRC Press; 2009.

30. Letseka M, Cosser M, Breier M, Visser M. Student retention and graduate destination: Higher education and labour market access and success. Pretoria: HSRC Press; 2009.

31. Marks GN. The occupations and earnings of young Australians: The role of education and training. LSAY Research Report No. 55. Melbourne: ACER; 2009.

32. Visser M, Kruss G. Learnerships and skills development in South Africa: A shift to prioritise the young unemployed. J Voc Educ Train. 2009;61(3):357–374. http://dx.doi.org/10.1080/13636820903180384, PMid:2169505

33. Feinstein L. Inequality in the early cognitive development of British children in the 1970 cohort. Economica. 2003;70:73–97. http://dx.doi.org/10.1111/1468-0335.t01-1-00272

34. Cunha F, Heckman JJ. Formulating, identifying and estimating the technology of cognitive and noncognitive skill formation. Working paper for the University of Chicago. Chicago, IL: University of Chicago; 2006.

35. Blanden J, Machin S. Recent changes in intergenerational mobility in Britain. Report for the Sutton Trust. London: Sutton Trust; 2007.

36. Cunha F, Heckman JJ. The technology of skill formation. Am Econ Rev. 2007;97(2):31–47. http://dx.doi.org/10.1257/aer.97.2.31

37. Lam D, Ardington C, Leibbrandt M. Schooling as a lottery: Racial differences in school advancement in urban South Africa. J Dev Econ. 2011;95(2):121–136. http://dx.doi.org/10.1016/j.jdeveco.2010.05.005, PMid:21499515

38. Heckman JJ. Skill formation and the economics of investing in disadvantaged children. Science. 2006;312(5782):1900–1902. http://dx.doi.org/10.1126/science.1128898, PMid:16809525

39. Brooks-Gunn J, Phelps E, Elder GH Jr. Studying lives through time: Secondary data analyses in developmental psychology. Dev Psychol. 1991;27(6):899–910. http://dx.doi.org/10.1037/0012-1649.27.6.899