8189_xml 20200926100000 assa nadiag@assaf.org.za assa South African Journal of Science S. Afr. J. Sci 1996-7489 09292020 116 9/10 Sisa Pazi Department of Statistics, Nelson Mandela University, Port Elizabeth, South Africa https://orcid.org/0000-0002-8880-3881 Chantelle M. Clohessy Department of Statistics, Nelson Mandela University, Port Elizabeth, South Africa https://orcid.org/0000-0002-4612-2228 Gary D. Sharp Department of Statistics, Nelson Mandela University, Port Elizabeth, South Africa https://orcid.org/0000-0003-0321-8067 In the electrical domain, a non-technical loss often refers to energy used but not paid for by a consumer. The identification and detection of this loss is important as the financial loss by the electricity supplier has a negative impact on revenue. Several statistical and machine learning classification algorithms have been developed to identify customers who use energy without paying. These algorithms are generally assessed and compared using results from a confusion matrix. We propose that the data for the performance metrics from the confusion matrix be resampled to improve the comparison methods of the algorithms. We use the results from three classification algorithms, namely a support vector machine, k-nearest neighbour and naïve Bayes procedure, to demonstrate how the methodology identifies the best classifier. The case study is of electrical consumption data for a large municipality in South Africa. 09292020 1 10.17159/sajs.2016/crossmark sajs.co.za false 2020-04-20 2020-06-13 2020-09-25 Nelson Mandela University http://dx.doi.org/10.13039/501100013969 National Research Foundation of South Africa https://creativecommons.org/licenses/by/4.0/ 10.17159/sajs.2020/8189 20200926100000 https://www.sajs.co.za/article/view/8189 https://www.sajs.co.za/article/view/8189/10468 https://www.sajs.co.za/article/view/8189/10468 https://www.sajs.co.za/article/view/8189/10468 https://www.sajs.co.za/article/view/8189/10468 https://www.sajs.co.za/article/view/8189/10468 https://www.sajs.co.za/article/view/8189/10468 https://www.sajs.co.za/article/view/8189/10468