ML Reproducability

What does reproducability mean in AI?

• Repeatability (same team, same experimental setup) @lynnerupSurveyReproducibilityEvaluating
• Reproducibility (different team, same experimental setup) @gundersenStateArtReproducibility2018, @lynnerupSurveyReproducibilityEvaluating
  • Reproducability in AI research defined as same data and the same code yielding the same analysis @chenTrainingReproducibleDeep2022
• Replicability (different team, different experimental setup) @lynnerupSurveyReproducibilityEvaluating

What are reproducability challenges in AI?

• Reproducibility Crisis in ML research
• Blog How to Solve Reproducibility in ML
• Dynamic aspect of ML models over time Data drift
• Reproducability in ML Model Training
  • Deep Learning Models not reproducabile due to Software/Hardware non determinism @chenTrainingReproducibleDeep2022
  • Data leakage @semmelrockReproducibilityMachineLearningDriven2023
• Reproducability in ML Model Evaluation
  • Inconsistent Reports / Implementation differences @hendersonDeepReinforcementLearning2018
  • experimental research and reported results @hendersonDeepReinforcementLearning2018
  • Deep Reinforcement Learning requires multiple evaluations and averaging Hyperparameter if Algo does not include random seed selection @hendersonDeepReinforcementLearning2018
  • Need to decoupled evaluation from algorithm @khetarpalREEVALUATEReproducibilityEvaluating2018
• Current ML Platforms are not out-of-the-box reproducable @gundersenMachineLearningPlatforms2022

What happens if you apply the concept of SBOMs to AI?

• Lists BOMs as future work in shared AI models @FrameworkFosteringTransparency2021
  • Talks about SBOM State of the Art @xiaEmpiricalStudySoftware2023
• Impl of Blockchain enabled AI-SBOM for secure sharing with Verifiable Credentials (VCs) @TrustSoftwareSupply2023
  • builds Webtool to verify Models and their Data with SBOMs and Verifiable Credentials @barclayProvidingAssuranceScrutability2021

How to know on which data a model was trained?

• An SBOM could be used to link Model and Data set using VCs
• But what about Online Learning? Say I want to know if a Model has been trained on particular Data?
  • The Model response might be unchanged if the newly trained Data does not have anough weight to influence the model