Also available as PDF (figures and tables appear in the PDF).
Diabetes mellitus is a major chronic disease, and statistical risk modelling of its onset has been studied for several decades. The Pima Indians Diabetes Database, comprising 768 women aged 21 years and above, has become one of the most frequently reused benchmark datasets in the biomedical statistics and machine learning (ML) literature, yet the statistical properties of the data and the validity of results derived from it are rarely examined together. This review combines a narrative synthesis of the literature with a full statistical re-analysis of the supplied dataset (268 diabetic, 500 non-diabetic; prevalence 34.9%). We examine data quality, descriptive and distributional properties, univariate group comparisons with effect sizes, categorical association tests, correlation structure, multivariable logistic regression and a ten-fold cross-validated comparison of seven classifiers. Our synthesis finds that (i) five physiological variables encode missing measurements as zero, so that only 392 of 768 records (51.0%) are complete; (ii) all eight predictors differ significantly between groups (Benjamini-Hochberg adjusted p < 0.001), with plasma glucose showing by far the largest standardised effect (Cohen’s d = 1.19); (iii) after mutual adjustment, only glucose, BMI, number of pregnancies and the diabetes pedigree function remain independent predictors (McFadden R2 = 0.282); and (iv) seven classifiers achieve cross-validated AUCs of 0.813-0.837 and accuracies of 74.6%-77.2%, with no model significantly better than plain logistic regression. We propose a short reporting checklist for studies using this dataset and identify six priority research directions.
Keywords: Diabetes mellitus; Pima Indians Diabetes Database; logistic regression; hypothesis testing; effect size; missing data; machine learning; narrative review; Diabetesmellitus; PimaIndiansDiabetesDatabase; logisticregression; hypothesistesting; missingdata; machinelearning; andhowlargeare
size,missingdata,machinelearning,narrativereview 1.Introduction RQ2:Whichmeasuredcharacteristicsdifferbetween diabeticandnon-diabeticwomen,andhowlargeare 1.1 Motivation thedifferences?
Type2diabetesmellitusresultsfromacombination RQ3:Whichcharacteristicsareindependentlyassoci- ofinsulinresistanceandprogressivebeta-celldysfunc- atedwithdiabetesaftermutualadjustment? tion,anditsriskisshapedbyadiposity,age,reproduc- tivehistory,glycaemicstatusandheredity.Earlyiden- RQ4:DoflexibleMLclassifiersimproveonlogisticre- tificationofhigh-riskindividualsisastandingpublic- gressionwhenvalidationisdonecorrectly? healthobjective,andstatisticalmodelsthattranslate RQ5:Whatlimitationsremainandwhatarethepriority routineclinicalmeasurementsintoariskestimatehave directionsforfutureresearch?
a long history in this field (Hosmer and Lemeshow, 2000; Kavakiotisetal., 2017). ThePimaIndianpop- 2.BackgroundandTheoreticalFoundations ulationofArizonawasstudiedintensivelybecauseof itsexceptionallyhighdiabetesincidence(Knowleret 2.1 OriginandContentoftheDataset al., 1978), andasubsetofthosesurveydatawasre- ThedatasetanalysedherehasthestructureofthePima leasedtotheresearchcommunityandlaterdeposited IndiansDiabetesDatabaseoriginallycompiledbythe intheUCIMachineLearningRepository(Smithetal., NationalInstituteofDiabetesandDigestiveandKidney 1988;DuaandGraff,2019).
Diseases(Smithetal.,1988): 768recordsofwomen ofPimaIndianheritageagedatleast21years,eight Theresultingtableof768recordsandeightpredictors numericpredictorsandabinaryoutcome.Predictors is small, clean in appearance and freely available, arethenumberofpregnancies,plasmaglucosecon- whichexplainsitspopularityasateachingandbench- centrationtwohoursintoanoralglucosetolerancetest markingdataset. Thesesamepropertiesarealsoits (mg/dL),diastolicbloodpressure(mmHg),tricepsskin- danger. The data hide a substantial missing-value foldthickness(mm),two-hourseruminsulin(µU/mL), problem,thesampleisrestrictedtoonedemographic bodymassindex(BMI,kg/m ),thediabetespedigree group,andaccuracyfiguresreportedintheliterature function (a score summarising family history of dia- dependheavilyonhowdatawerepre-processedand betes)andage(years). Descriptionsoftheexactout- validated(Temurtasetal.,2009;SisodiaandSisodia, comedefinitiondifferbetweensecondarysources,so 2018). A statistical appraisal that examines these usersshouldconfirmitagainsttheprimarydocumen- issuesbefore,andalongside,predictivemodellingis tationbeforemakingclinicalclaims.
2.2 StatisticalToolsforGroupComparison 1.2 ScopeandContribution Distributional shape was assessed with skewness, Thisarticleisanarrativereviewsupportedbyacom- excess kurtosis and the Shapiro-Wilk test (Shapiro pletestatisticalre-analysisofthesupplieddataset.Our andWilk,1965).Becausevariancesdifferedbetween contributionsarefourfold: groups for several variables, mean differences were Astructureddescriptionofthedataset,includingprove- tested with Welch’s unequal-variance t-test (Welch, nance,variabledefinitionsandaquantitativeauditof 1947),andresultswerecross-checkedwiththerank- implausibleandmissingvalues.
basedMann-Whitneytest(MannandWhitney,1947), Abatteryofclassicalinferentialtools(normalitytests, whichdoesnotassumenormality. Statisticalsignifi- Welch’st-test,theMann-Whitneytest,effectsizes,chi- cancealonesayslittleaboutpracticalimportance,so squaretestsandcorrelationanalysis)withmultiplicity standardisedmeandifferences(Cohen’sd,interpreted control.
withtheconventional0.2/0.5/0.8benchmarks;Cohen, 1988) and rank-biserial correlations are reported Multivariablelogisticregressionwithdiagnosticsanda alongsidep-values.Becauseeightvariablesaretested sensitivityanalysisoftheconclusionstothehandling simultaneously,thefalsediscoveryratewascontrolled of missing values, followed by a leakage-free cross- withtheBenjamini-Hochbergprocedure(Benjamini validatedcomparisonofsevenclassifiers. andHochberg,1995).
A reporting checklist and six priority research direc- tionsforfutureworkthatusesthisdataset. 2.3 AssociationandRegressionMethods Associationsbetweencategorisedriskfactorsanddi- 1.3 ReviewQuestions abetesstatusweretestedwithPearson’schi-square RQ1:Whatisthestructureandqualityofthedataset, testandsummarisedwithCramér’sV.Monotonerela- andhowshouldimplausiblevaluesbehandled? tionshipsweremeasuredwithSpearman’srankcorre- lation.Multivariablerelationshipsweremodelledwith (a) the raw data, with zeros left as recorded; (b) a binarylogisticregression,whoseexponentiatedcoef- median-imputeddataset,usedforinference;and(c) ficientsareoddsratios(OR)(HosmerandLemeshow, a complete-case data set (n = 392). For prediction, 2000). Multicollinearitywasexaminedwithvariance median imputation and standardisation were fitted inflationfactors(VIF),goodnessoffitwiththeHosmer- inside each cross-validation training fold so that LemeshowtestandMcFadden’spseudo-R ,anddis- no information from the test fold could leak into criminationwiththeareaunderthereceiveroperating pre-processing.
characteristiccurve(AUC)(HanleyandMcNeil,1982). 3.3 StatisticalAnalysisPlan 2.4 PredictiveModelFamilies Theanalysisproceededinsixstages: (i)data-quality Predictive studies on this dataset use three broad audit;(ii)descriptivestatisticsanddistributionaltests; families: (1)linearandprobabilisticmodels(logistic (iii)univariategroupcomparisonswitheffectsizesand regression, linear discriminant analysis, Gaussian multiplicitycontrol;(iv)chi-squaretestsonclinically naiveBayes);(2)distance-andmargin-basedlearners categorised variables (age bands, WHO BMI classes, (k-nearest neighbours, support vector machines; paritybandsandatwo-hourglucosethresholdof140 Cortes and Vapnik, 1995); and (3) tree ensembles mg/dL);(v)correlation,VIFandmultivariablelogisticre- (random forests, Breiman, 2001; gradient boosting, gression,withHosmer-LemeshowandCook’sdistance Friedman, 2001). Each family offers a different diagnostics;and(vi)predictivecomparison.With268 trade-offbetweenflexibility,interpretabilityanddata eventsandeightcandidatepredictorsthereareabout requirements.
33eventspervariable,comfortablyabovethecustom- aryminimumof10(VittinghoffandMcCulloch,2007). 2.5 HandlingMissingandImplausibleValues LittleandRubin(2019)distinguishdatathataremiss- 3.4 ValidationofPredictiveModels ingcompletelyatrandom,atrandom,ornotatrandom, Seven classifiers (logistic regression, LDA, Gaussian andshowthatsimplestrategiessuchascomplete-case naiveBayes,k-NNwithk=15,RBF-kernelSVM,random analysisorsingleimputationarevalidonlyunderre- forestwith500treesandgradientboosting)wereeval- strictiveassumptions.InthePimadata,physiologically uatedwithstratifiedten-foldcross-validation(seed42) impossiblezeros(forexampleadiastolicbloodpres- usingdefaulthyperparameters. Accuracy, AUC,sen- sureof0mmHg)actasmissing-valuecodes.Treating sitivity, specificity, precision and F1 are reported at themasgenuinemeasurementsdistortsmeans,vari- a0.5probabilitythreshold. Tocomparemodelsfor- ancesandcorrelations,sothechoiceofstrategyisit- mally,ten-foldcross-validationwasrepeatedfivetimes selfamodellingdecisionthatshouldbereportedand (50 folds), and AUC differences from logistic regres- stress-tested.
sion were tested with the Nadeau-Bengio corrected resampledt-test,whichcorrectstheover-optimismof 3.MethodologyoftheReview naivepairedtestsonoverlappingtrainingsets(Nadeau 3.1 ReviewDesign andBengio,2003).AnalysesusedPythonwithSciPy, Thestudyhastwocomponents. Theliteraturecom- statsmodels and scikit-learn (Seabold and Perktold, ponentisaselectivenarrativereviewoffoundational 2010; Pedregosaetal.,2011). Reportingfollowsthe andwidelycitedworkonthedatasetandonthesta- spiritoftheTRIPODstatement(Collinsetal.,2015). tisticalandMLmethodsappliedtoit;itdoesnotclaim theexhaustivesearchandscreeningcountsofaformal 4.StructureandQualityoftheDataset PRISMAsystematicreview. Thequantitativecompo- nentisafullre-analysisofthesuppliedfile,inwhich 4.1 Variables every figure and table in Sections 4 and 5 was com- 4.2 ClassDistribution puteddirectlyfromthedata.
Of 768 women, 268 (34.9%) are labelled diabetic and500(65.1%)non-diabetic, aclassratioofabout 3.2 DataandPre-processing 1.9:1.Aclassifierthatalwayspredicts“non-diabetic” The file contains 768 rows and nine columns (eight wouldthereforebecorrect65.1%ofthetime,andthis predictorsandthebinaryOutcome),withnoduplicate majority-class rate is the correct reference point for rows and no empty cells. Zero values in glucose, judginganyreportedaccuracy.Agesrangefrom21to bloodpressure,skinthickness,insulinandBMIwere 81years(median29), andthemaximumnumberof treated as missing, whereas zero pregnancies (111 pregnanciesis17.
women)isavalidvalue.Threedatasetswerederived: Variable Description(unit) Type Pregnancies Numberoftimespregnant Count Glucose Plasmaglucose,2horalglucosetol- Continuous erancetest(mg/dL) BloodPressure Diastolicbloodpressure(mmHg) Continuous SkinThickness Tricepsskinfoldthickness(mm) Continuous Insulin 2-hourseruminsulin(µU/mL) Continuous BMI Bodymassindex(kg/m ) Continuous DiabetesPedigreeFunction Scoresummarisingfamilyhistoryof Continuous diabetes Age Age(years) Integer Outcome Diabetesstatus(1=diabetic,0=non- Binary diabetic)
BMI 11 1.4% 757 4.3 AuditofImplausibleZeroValues 142.3vs110.6mg/dL,d=1.19),followedbymoderate effects for insulin (d = 0.68), BMI (d = 0.69) and skin Insulin(48.7%)andskinthickness(29.6%)arethemost thickness(d=0.57).Age,pregnancies,bloodpressure affected.Only392records(51.0%)arecompleteacross and the pedigree function show small-to-medium allfivevariables,socomplete-caseanalysiswoulddis- effects (d between 0.36 and 0.48). The pedigree card376women(49.0%).Becausethemechanismthat functionandbloodpressurearetheweakestdiscrimi- generatedthezerosisundocumented,assumingthat nators,withdifferencesof0.121unitsand4.4mmHg theyaremissingcompletelyatrandomisunsafe,and respectively.
Section5.5testshowmuchtheconclusionsdependon thisassumption.
5.2 AssociationswithCategorisedRiskFactors 4.4 DescriptiveStatisticsandDistributionalShape ZerosinglucoseandBMIexcluded.Theunderweight TheShapiro-Wilktestrejectednormalityforeveryvari- BMIclasscontainsonly4women,sotheBMIchi-square able(allp<0.001),thoughwithnabove700thetest approximationisimperfect.The140mg/dLcut-offis detects even trivial departures. The departures are theconventionallowerboundofimpairedglucosetol- practicallymeaningfulforinsulin(skewness2.16,ex- eranceforatwo-hourpost-loadvalue(AmericanDia- cesskurtosis6.27)andthepedigreefunction(skewness betesAssociation,2024).
1.92,excesskurtosis5.55),bothofwhicharestrongly Allfourfactorswerestronglyassociatedwithdiabetes right-skewed(Figure1). Ageisalsoright-skewed,re- (allp<0.001).Thegradientsaresteep:prevalencerises flectingasampleconcentratedintheearlyreproduc- from21.2%amongwomenaged21-29to55.1%atages tiveandmiddleyears(median29years).Theseshapes 40-49,andfrom6.9%inthenormalBMIclassto46.4% motivatetheuseofrank-basedtestsalongsidet-tests. intheobeseclass,where472of757women(62.4%)are found.Atwo-hourglucoseofatleast140mg/dLisas- 5.EmpiricalEvidencefromtheDataset sociatedwithanunadjustedoddsratioof7.23(95%CI 5.05-10.35)forthediabeticlabel.Parityshowedanon- 5.1 UnivariateGroupComparisons monotonicpattern(20.2%for1-2pregnanciesagainst Zerosinphysiologicalvariablesexcluded. Welchde- 34.2%fornone),whichprobablyreflectsconfounding grees of freedom vary by variable. BH = Benjamini- withageratherthanaprotectiveeffectofafirstpreg- Hochbergadjustedacrosseighttests.
nancy; this cannot be resolved without adjustment Every predictor differed significantly between the (Section5.4).Notealsothattheassociationofglucose groups,andtheWelchandMann-Whitneytestsagreed withtheoutcomeispartlydefinitional,sinceglucose inallcases.Effectsizes,however,varywidely.Plasma toleranceisitselfadiagnosticcriterionfordiabetes. glucoseshowsaverylargedifference(diabeticmean Variable n Mean SD Median Q1-Q3 Range Skew Kurt. Pregnancies 768 3.85 3.37 3.00 1.00-6.00 0.00-17.00 0.90 0.15 Glucose 763 121.7 30.5 117.0 99.0-141.0 44.0-199.0 0.53 -0.28 Blood pres- 733 72.4 12.4 72.0 64.0-80.0 24.0-122.0 0.13 0.90 sure Skin thick- 541 29.2 10.5 29.0 22.0-36.0 7.0-99.0 0.69 2.90 ness Insulin 394 155.5 118.8 125.0 76.3-190.0 14.0-846.0 2.16 6.27 BMI 757 32.5 6.9 32.3 27.5-36.6 18.2-67.1 0.59 0.85 Pedigree 768 0.472 0.331 0.372 0.244-0.626 0.078-2.420 1.92 5.55 function Age 768 33.2 11.8 29.0 24.0-41.0 21.0-81.0 1.13 0.63
5.3 CorrelationStructureandMulticollinearity insulinphysiologyandthemechanicallinkbetween age and parity. With the outcome, glucose has the strongest correlation (Pearson r = 0.49, Spearman ρ =0.48),followedbyBMI(r=0.31)andinsulin(r=0.30). Despitethesecorrelations, multicollinearityisnota seriousproblemforregression:themaximumVIFwas 1.62(age),farbelowtheusualwarningthresholdsof5 or10.
5.4 MultivariableLogisticRegression − 2 Intercept= 9.109.Likelihood-ratio χ =280.6(8df),p <0.001;McFaddenR =0.282;AIC=730.8;BIC=772.6; Hosmer-Lemeshow χ =12.17(8df),p=0.144. Fourpredictorsareindependentlyassociatedwithdi- abetes. Eachadditionalmg/dLofglucoseraisesthe oddsby3.9%(OR1.039),andonestandarddeviationof
SD1.52)andthepedigreefunction(ORperSD1.34;OR tweenBMIandskinthickness(Spearman ρ =0.68),glu- 2.40perunit)follow.Incontrast,bloodpressure,skin coseandinsulin(ρ =0.66)andageandpregnancies(ρ thickness,insulinandagearenotsignificantoncethe =0.61),asexpectedfrombodycomposition,glucose- othervariablesareinthemodel,althoughseveralof Variable Diabetic mean Non-diabetic Difference[95% Welcht Cohen’sd Mann-Whitney (SD) mean(SD) CI] p(BH) Pregnancies 4.87(3.74) 3.30(3.02) 1.57[1.05,2.09] 5.91 0.48 <0.001 Glucose 142.3(29.6) 110.6(24.8) 31.7[27.5,35.9] 14.88 1.19 <0.001 Bloodpressure 75.3(12.3) 70.9(12.2) 4.4[2.6,6.3] 4.66 0.36 <0.001 Skinthickness 33.0(10.3) 27.2(10.0) 5.8[3.9,7.6] 6.18 0.57 <0.001 Insulin 206.8(132.7) 130.3(102.5) 76.6 [50.5, 5.78 0.68 <0.001 102.7] BMI 35.4(6.6) 30.9(6.6) 4.5[3.6,5.5] 9.06 0.69 <0.001 Pedigree func- 0.550(0.372) 0.430(0.299) 0.121 [0.069, 4.58 0.37 <0.001 tion 0.173] Age 37.1(11.0) 31.2(11.7) 5.9[4.2,7.5] 6.92 0.51 <0.001
Age(years) 40-49 118 55.1% 69.9(3) <0.001 0.30 Age(years) 50+ 89 48.3% 69.9(3) <0.001 0.30 BMIclass(WHO) Underweight 4 0.0% 77.1(3) <0.001 0.32 (<18.5) BMIclass(WHO) Normal (18.5- 102 6.9% 77.1(3) <0.001 0.32 24.9) BMIclass(WHO) Overweight(25- 179 22.3% 77.1(3) <0.001 0.32 29.9) BMIclass(WHO) Obese(≥30) 472 46.4% 77.1(3) <0.001 0.32 Pregnancies 0 111 34.2% 46.8(3) <0.001 0.25 Pregnancies 1-2 238 20.2% 46.8(3) <0.001 0.25 Pregnancies 3-5 200 35.5% 46.8(3) <0.001 0.25 Pregnancies 6+ 219 50.7% 46.8(3) <0.001 0.25 2-h glucose <140 566 23.1% 132.5(1) <0.001 0.42 (mg/dL) 2-h glucose ≥140 197 68.5% 132.5(1) <0.001 0.42 (mg/dL) themwerehighlysignificantintheunivariateanalysis. Thisisthepatternexpectedunderconfoundingand sharedvariance: ageiscorrelatedwithpregnancies, skinthicknesswithBMI,andinsulinwithglucose. The model fits adequately (Hosmer-Lemeshow p = 0.144, which gives no evidence of poor calibration, althoughthistesthaslimitedpower)anditsin-sample AUC is 0.845. A total of 49 observations had Cook’s distanceabove4/n(maximum0.072);noneisextreme enoughtowarrantremoval,andnoobservationswere deleted.
5.5 RobustnesstoMissing-DataHandling deviation)with95%confidenceintervals. Glucose,BMIandthepedigreefunctionremainsignifi- cantinbothanalyses,sothoseconclusionsarerobust. Thepregnancyeffect,however,issignificantafterim- lowestpseudo-R (0.272).Theseshiftsshowthatcon- putation(p<0.001)butnotinthecomplete-caseanal- clusionsaboutsecondarypredictorsdependonhow ysis(OR1.086,p=0.138),andagemovesfromclearly thezerosarehandledandshouldnotbereportedwith- non-significant (p = 0.171) to borderline (p = 0.065). outthatcaveat. Medianimputationisadeliberately Thecomplete-casemodelalsohasahigherpseudo-R simplechoicethatshrinksvarianceandattenuatesas- (0.309vs0.282),butitisbasedonhalfthesampleand sociations;multipleimputationbychainedequations apossiblyunrepresentativesubset.Amodelfittedto wouldbethemoredefensibleapproach(LittleandRu- therawdatawithzerostreatedastruevalueshasthe bin,2019).
Predictor SE OR (95% CI), per p OR per 1 SD (95% unit CI) Pregnancies 0.125 0.032 1.133(1.063-1.207) <0.001 1.52(1.23-1.89) Glucose 0.038 0.004 1.039(1.031-1.047) <0.001 3.17(2.51-3.99) Bloodpressure -0.009 0.009 0.991(0.974-1.007) 0.275 0.89(0.73-1.09) Skinthickness 0.003 0.013 1.003(0.978-1.030) 0.793 1.03(0.82-1.29) Insulin -0.001 0.001 0.999(0.997-1.001) 0.301 0.90(0.75-1.09) BMI 0.094 0.018 1.099(1.061-1.138) <0.001 1.91(1.50-2.43) Pedigreefunction 0.876 0.297 2.401(1.342-4.295) 0.003 1.34(1.10-1.62) Age 0.013 0.010 1.013(0.994-1.032) 0.171 1.17(0.94-1.45)
1.086(0.974-1.210) 0.138 Glucose 1.039(1.031-1.047) <0.001 1.039(1.027-1.051) <0.001 Bloodpressure 0.991(0.974-1.007) 0.275 0.999(0.976-1.022) 0.904 Skinthickness 1.003(0.978-1.030) 0.793 1.011(0.978-1.046) 0.511 Insulin 0.999(0.997-1.001) 0.301 0.999(0.997-1.002) 0.528 BMI 1.099(1.061-1.138) <0.001 1.073(1.017-1.132) 0.010 Pedigreefunction 2.401(1.342-4.295) 0.003 3.130(1.354-7.233) 0.008 Age 1.013(0.994-1.032) 0.171 1.035(0.998-1.072) 0.065 5.6 PredictivePerformanceofSevenClassifiers it. Only Gaussian naive Bayes is significantly worse Threshold0.5.Majority-classaccuracy=0.651.Default (meanAUCdifference-0.025,p=0.003),mostplausibly hyperparameterswereusedforallmodels.
becauseitsconditional-independenceassumptionis violatedbythecorrelatedpredictors.Theseresultsare fordefaulthyperparameters; tuninginsideanested cross-validationloopmightnarroworwidenthegaps, butitisunlikelytooverturnthemainconclusion. 5.7 Single-VariableDiscrimination,VariableImpor- tanceandDimensionality Glucose alone achieves an AUC of 0.792, which is within about 0.05 of the full multivariable models, whereasBMIandagealonegiveonly0.686and0.687. Therandomforestimpurityimportancesrankglucose first(0.258),followedbyBMI(0.163),age(0.130)and thepedigreefunction(0.123),whichagreeswiththe regression ranking for glucose and BMI. A principal component analysis of the standardised predictors shows no low-dimensional structure: the first three
of similar magnitude on most variables (28.5% of racyrangesfrom74.6%to77.2%,onlyabout12per- variance).
centagepointsabovethemajority-classrateof65.1%, and AUCs lie within a band of 0.024. At the default 0.5threshold,sensitivityismodest(0.54-0.63)while 6.ComparativeAnalysisandDiscussion specificityishigher(0.82-0.88),asexpectedwitha35% 6.1 RQ1:Structureandqualityofthedataset positiveclass;ascreeningapplicationwouldneedto Thedatasetissmall,hasnoduplicatesandnoblank movethethresholdtofavoursensitivity.
cells,butitisnotclean. Fivevariablesusezeroasa With the Nadeau-Bengio correction, none of the missing-valuecode,andtheaffectedfractionislarge more flexible models (kNN, SVM, random forest, forinsulin(48.7%)andskinthickness(29.6%). Treat- gradient boosting) is significantly different from ingthezerosasrealvalues(asmanyappliedstudies logisticregression,andLDAisindistinguishablefrom implicitlydo)isstatisticallyunjustified,anddiscard- Model Accuracy(SD) AUC(SD) Sensitivity Specificity Precision F1 Logistic regres- 0.767(0.025) 0.837(0.049) 0.556 0.880 0.726 0.623 sion LDA 0.764(0.025) 0.837(0.049) 0.556 0.876 0.723 0.622 GaussianNB 0.746(0.061) 0.813(0.057) 0.601 0.824 0.661 0.622 k-NN(k=15) 0.762(0.057) 0.821(0.059) 0.586 0.856 0.687 0.631 SVM(RBF) 0.747(0.039) 0.826(0.039) 0.537 0.866 0.680 0.597 Randomforest 0.772(0.040) 0.828(0.049) 0.634 0.844 0.689 0.659 Gradientboost- 0.772(0.053) 0.834(0.054) 0.627 0.850 0.697 0.655 ing
k-NN(k=15) -0.013 0.231 SVM(RBF) -0.011 0.185 Randomforest -0.009 0.294 Gradientboosting -0.007 0.471 ingincompleterowsremoves49.0%ofthesample.A reportedperformanceindiabetesstudiesisstrongly defensibleworkflowrecodeszerosasmissing,docu- influencedbydatapreparationandvalidationrather mentstheamountandpatternofmissingness,imputes than by the algorithm alone. On a dataset of this withinthemodellingpipeline,andreportsasensitivity size,withroughlyeightweaklynon-linearpredictors, analysis.
thereislittlesignalleftforaflexiblelearnertocapture beyondwhatalinearlogitalreadyprovides. Studies 6.2 RQ2: Which characteristics differ between thatreportmarkedlyhigheraccuracyonthisdataset groups should therefore be scrutinised for pre-processing Alleightpredictorsdifferbetweengroups,butthesize leakage, forinstanceimputingorscalingbeforethe ofthedifferencematters. Glucosestandsapart(d= train-testsplit,orforselectivechoiceoftestsets. In 1.19);insulin,BMIandskinthicknesshavemoderate thepresentanalysis,pre-processingwaskeptinside effects;age,pregnancies,bloodpressureandfamily eachtrainingfoldspecificallytopreventthis. historyhavesmall-to-mediumeffects.Inlinewiththe definitionalroleofglucosetoleranceindiagnosingdi- 6.5 RQ5:Trade-offsandpracticalimplications abetes,theglucosefindingisexpected,anditshould Interpretabilityvscomplexity: logisticregressionof- beinterpretedasamarkeroftheconditionratherthan ferstransparentoddsratioswithconfidenceintervals asanindependentcausalriskfactor.
at no loss of accuracy. Sensitivity vs specificity: de- faultthresholdsgivemodestsensitivity,soclinicaluse 6.3 RQ3:Independentpredictors wouldrequirethresholdtuningagainstthecostofa After mutual adjustment, glucose, BMI, number of missed case. Representativeness vs size: complete- pregnanciesandthepedigreefunctionareindepen- caseanalysiskeepsmeasuredvaluesbuthalvesthe dentlyassociatedwiththeoutcome;bloodpressure, sample,whereasimputationretainsthesamplebut skinthickness,insulinandagearenot.Thefirstthree imposesassumptions.Generalisability:allconclusions conclusionsarestableundercomplete-caseanalysis areconditionalonasinglepopulationofadultwomen (apartfrompregnancies),whereastherolesofageand andshouldbeexternallyvalidatedbeforeanywider pregnancies are sensitive to missing-data handling. use.
Because the data are cross-sectional and from one population,associationsshouldnotbereadascausal. 7.ChallengesandLimitations 7.1 MethodologicalChallenges 6.4 RQ4: Statisticalversusmachine-learningpre- dictors Missing-datahandling: medianimputationissimple andattenuatesassociations;multipleimputationwas Logistic regression, the simplest and most inter- notusedandwouldbepreferable.
pretable model, matches the best-performing alternatives (AUC 0.837 against 0.834 for gradient Untunedmodels:defaulthyperparameterswereused, boosting and 0.828 for random forest). This agrees sothecomparisonisbetweendefaultconfigurations withtheobservationbyKavakiotisetal. (2017)that andnotbetweenoptimisedones.
Predictor AUC Youdencut-off Sensitivity Specificity Glucose(mg/dL) 0.792 124 0.701 0.732 BMI(kg/m2) 0.686 29.9 0.832 0.470 Age(years) 0.687 29 0.735 0.592
cosed=1.19 RQ3Independentpredictors Glucose, BMI, pregnancies, pedigree Table5.3,5.4 function RQ4MLvsstatistical AUC0.813-0.837;nomodelbeatslogis- Table5.5,5.6 ticregression RQ5Limits Singlepopulation,simpleimputation, Section7 defaulthyperparameters Thresholddependence:sensitivity,specificityandF1 Calibration and decision curves. Report calibration refer to a fixed 0.5 probability threshold and would plotsandnetbenefitinadditiontoAUC,sothatmodels changewiththethreshold.
Multiple testing and diagnostics: the Hosmer- Uncertaintyquantification.Provideconfidenceinter- Lemeshowtesthaslimitedpower,and49observations valsforperformancemetricsthroughbootstrappingor had Cook’s distance above 4/n though none was nestedcross-validation.
Externalvalidation.Testmodelstrainedonthisdataset inother,largerandmorediversecohorts.
7.2 DataChallenges Singlepopulation:onlywomenofPimaheritageaged Interactionandnon-linearity.Examinesplinesandin- 21andoverareincluded,whichlimitsgeneralisability teractions(forexampleage × BMI)withinaregression tomen,otherethnicgroupsandyoungerpeople. frameworkbeforeturningtoblack-boxmodels. Smallsample: 768records(and392completeones) Reportingchecklist.Adoptaminimalchecklistforstud- restricttheprecisionofsubgroupestimatesandthe iesonthisdataset:(i)statethetreatmentofzerovalues; powertodetectinteractions.
(ii)fitpre-processinginsidecross-validationfolds;(iii) Cross-sectionaldesign:temporalorderingofriskfac- reportthemajority-classbaseline;(iv)reportAUCwith torsandoutcomecannotbeestablished,andtheexact aninterval;(v)compareagainstlogisticregression;(vi) outcomedefinitionvariesacrosssecondarydescrip- statethethresholdandthemetricsatthatthreshold. tions.
9.Conclusion 7.3 ReportingChallenges ThePimaIndiansDiabetesdatasetremainsaconve- Inconsistenttreatmentofzerovaluesacrosspublished nientresource,butthisre-analysisshowsthatitsappar- studies.
entsimplicitymasksseveralstatisticalpitfalls.Three Accuracyreportedwithoutthemajority-classbaseline, conclusions are robust. First, 49.0% of records con- calibrationorconfidenceintervals.
tainzero-codedmissingvalues,andthehandlingof Pre-processingstepsandcross-validationseedsoften thesezerosmateriallyaffectstheconclusionsabout undocumented.
secondarypredictors. Second,plasmaglucoseisby farthedominantcorrelateofdiabetesstatus(d=1.19; Theliteraturecomponentofthisreviewisselectiveand ORperSD3.17),followedbyBMI,thepedigreefunc- narrative,andnotanexhaustivesystematicsearch. tionandthenumberofpregnancies,whilebloodpres- sure, skin thickness, insulin and age add little after 8.PriorityResearchDirections adjustment. Third,flexibleMLclassifiersprovideno Multiple imputation and missingness mechanisms. significantimprovementoverlogisticregressionun- Use chained-equation imputation, with pooled derleakage-freevalidation(cross-validatedAUCabout estimates, and test explicitly whether the zeros are 0.84). Futurestudiesshouldthereforejustifyadded missingatrandom.
modelcomplexitywithproperlyvalidatedgains,report uncertaintyandbaselines,andevaluatetransportabil- largerthantheother.AnnalsofMathematicalStatistics, itybeyondthissinglepopulationbeforedrawingclini- 18(1),50-60.
Nadeau,C.andBengio,Y.(2003).Inferenceforthegen- eralizationerror.MachineLearning,52(3),239-281.