Comparing the performances of prediction models: A study on growth of lambs Tahmin modellerinin performanslarının değerlendirilmesi: Kuzularda büyüme üzerine bir çalışma


Gullu Ö., Gurcan I. S.

Ankara Universitesi Veteriner Fakultesi Dergisi, cilt.64, sa.2, ss.131-136, 2017 (SCI-Expanded) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 64 Sayı: 2
  • Basım Tarihi: 2017
  • Doi Numarası: 10.1501/vetfak_0000002787
  • Dergi Adı: Ankara Universitesi Veteriner Fakultesi Dergisi
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, TR DİZİN (ULAKBİM)
  • Sayfa Sayıları: ss.131-136
  • Anahtar Kelimeler: AUC, IDI, model performance measures, NRI, reclassification, RISK STRATIFICATION, CURVE
  • Ankara Üniversitesi Adresli: Evet

Özet

© 2017, Chartered Inst. of Building Services Engineers. All rights reserved.The aim of this study is to assess the impact of the birth weight variable on the performance of the model through the use of the classical methods employed to evaluate the performances of prediction models, namely, coefficient of determination, Brier score, area under the ROC curve (AUC), and two new alternative methods, namely, Net Reclassification Improvement (NRI) and Integrated Discrimination Improvement (IDI). The material of the study consists of the data on the growth of 433 lambs in Sivas-Ulaş Agricultural Enterprise between 1996 and 1997. The study examines the impact of birth weight on the model's performance in the classification of lambs as those having and not having the desired weaning weight (WW). The results indicate that the contribution of birth weight to the discrimination of the model is 2.1% according to AUC. NRI was found to be 11.6% (p<0.001). Thus, when the birth weight variable is added, the probability of lambs with the desired WW to be included in the low risk category is 11.6% higher than the probability of those lambs to be included in the high risk category. Categorical independent IDI was calculated to be 3.3% (p<0.001). In conclusion, NRI indicates the impact of birth weight more sensitively than AUC by measuring the change on the basis of the risk categories. These performance indexes (NRI and IDI) newly developed in the literature produce more sensitive results compared to the classical approach (AUC).