MLCommons

All

103 repositories

algorithmic-efficiency
Public
MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.
Python
•
Apache License 2.0
•68•351•30•12•Updated Jan 26, 2025Jan 26, 2025
mlperf-automations
Public
automation mlperf mlperf-inference mlcflow mlperf-automations
Python
•
Apache License 2.0
•7•1•49•0•Updated Jan 26, 2025Jan 26, 2025
mlperf_inference_test_submissions_v5.0
Public
HTML
•5•0•0•1•Updated Jan 26, 2025Jan 26, 2025
inference_results_v4.0
Public
This repository contains the results and code for the MLPerf™ Inference v4.0 benchmark.
Apache License 2.0
•17•10•5•5•Updated Jan 26, 2025Jan 26, 2025
mlcflow
Public
MLCFlow: Simplifying MLPerf Automations
automation-framework mlperf mlcflow mlperf-automations
Python
•
Apache License 2.0
•6•2•6•1•Updated Jan 25, 2025Jan 25, 2025
modelbench
Public
Run safety benchmarks against AI models and view detailed reports showing how well they performed.
Python
•
Apache License 2.0
•12•75•197•5•Updated Jan 25, 2025Jan 25, 2025
power-dev
Public
Dev repo for power measurement for the MLPerf™ benchmarks
Python
•
Apache License 2.0
•25•20•10•4•Updated Jan 25, 2025Jan 25, 2025
inference
Public
Reference implementations of MLPerf™ inference benchmarks
benchmark machine-learning
Python
•
Apache License 2.0
•540•1.3k•170•12•Updated Jan 24, 2025Jan 24, 2025
croissant
Public
Croissant is a high-level format for machine learning datasets that brings together four rich layers.
machine-learning schema-org json-ld datasets
Python
•
Apache License 2.0
•46•495•129•18•Updated Jan 24, 2025Jan 24, 2025
inference_results_v4.1
Public
This repository contains the results and code for the MLPerf™ Inference v4.1 benchmark.
benchmark machine-learning
Python
•
Apache License 2.0
•9•3•8•1•Updated Jan 24, 2025Jan 24, 2025
dynabench
Public
Python
•
MIT License
•16•21•7•4•Updated Jan 24, 2025Jan 24, 2025
mlperf_inference_unofficial_submissions_v5.0
Public
These are automated test submissions for validating the MLPerf inference workflows
mlperf mlperf-inference mlperf-inference-results
Mermaid
•2•0•0•0•Updated Jan 22, 2025Jan 22, 2025
inference_policies
Public
Issues related to MLPerf™ Inference policies, including rules and suggested changes
Apache License 2.0
•52•58•18•2•Updated Jan 21, 2025Jan 21, 2025
mobile_app_open
Public
Mobile App Open
C++
•
Apache License 2.0
•26•51•31•3•Updated Jan 21, 2025Jan 21, 2025
tiny
Public
MLPerf™ Tiny is an ML benchmark suite for extremely low-power systems such as microcontrollers
C
•
Apache License 2.0
•93•370•13•6•Updated Jan 20, 2025Jan 20, 2025
training
Public
Reference implementations of MLPerf™ training benchmarks
benchmark machine-learning
Python
•
Apache License 2.0
•563•1.6k•16•42•Updated Jan 15, 2025Jan 15, 2025
ck
Public
Collective Knowledge (CK) and Collective Mind (CM): community-driven projects to learn how to run AI, ML and other emerging workloads in a more efficient and cost-effective way across diverse models, datasets, software and hardware using CK, CM/CMX and MLPerf automations
education automation optimization best-practices collaboration cm modularity workflows portability reusability
Python
•
Apache License 2.0
•120•612•14•0•Updated Jan 15, 2025Jan 15, 2025
cm4mlops
Public
A collection of portable, reusable and cross-platform CM automations for MLOps and MLPerf to simplify the process of building, benchmarking and optimizing AI systems across diverse models, data sets, software and hardware
workflow devops machine-learning automation cross-platform reusable workflow-automation mlops mlperf
Python
•
Apache License 2.0
•24•16•7•0•Updated Jan 15, 2025Jan 15, 2025
training_results_v3.1
Public
This repository contains the results and code for the MLPerf™ Training v3.1 benchmark.
Python
•
Apache License 2.0
•11•17•7•4•Updated Jan 14, 2025Jan 14, 2025
inference_results_visualization_template
Public template
JavaScript
•
Apache License 2.0
•6•0•1•1•Updated Jan 8, 2025Jan 8, 2025
logging
Public
MLPerf™ logging library
Python
•
Apache License 2.0
•46•32•41•2•Updated Jan 8, 2025Jan 8, 2025
mlperf_inference_submissions
Public template
Shell
•1•0•0•2•Updated Jan 8, 2025Jan 8, 2025
mobile_open
Public
MLPerf Mobile benchmarks
Python
•
Apache License 2.0
•2•10•6•2•Updated Jan 7, 2025Jan 7, 2025
ailuminate
Public
Apache License 2.0
•1•1•1•0•Updated Jan 4, 2025Jan 4, 2025
GaNDLF-Synth
Public
Extension for GaNDLF [gandlf.org] to enable synthesis.
medical-imaging synthesis
Python
•
Apache License 2.0
•2•4•6•0•Updated Dec 30, 2024Dec 30, 2024
medperf
Public
An open benchmarking platform for medical artificial intelligence using Federated Evaluation.
Python
•
Apache License 2.0
•33•150•52•23•Updated Dec 21, 2024Dec 21, 2024
GaNDLF
Public
A generalizable application framework for segmentation, regression, and classification using PyTorch
machine-learning framework deep-learning regression pytorch medical-imaging classification segmentation data-augmentation biomedical-image-processing
Python
•
Apache License 2.0
•80•171•16•3•Updated Dec 20, 2024Dec 20, 2024
policies
Public
General policies for MLPerf™ including submission rules, coding standards, etc.
Python
•
Apache License 2.0
•55•28•35•39•Updated Dec 18, 2024Dec 18, 2024
training_results_v4.1
Public
This repository contains the results and code for the MLPerf™ Training v4.1 benchmark.
Python
•6•4•1•0•Updated Dec 11, 2024Dec 11, 2024
mlperf-automations_archived
Public archive
This repository contains the automations and scripts used to automate MLPerf benchmarks (mainly MLPerf inference for now)
automation mlperf
Python
•
Apache License 2.0
•24•0•46•0•Updated Dec 11, 2024Dec 11, 2024