Welcome to the MLPerf Automations and Scripts repository! This repository is your go-to resource for tools, automations, and scripts designed to streamline the execution of MLPerf benchmarks—with a strong emphasis on MLPerf Inference benchmarks.
Starting January 2025, MLPerf automation scripts are built on the powerful MLCFlow automation interface. This modern interface replaces the earlier Collective Mind (CM), offering a more robust and efficient framework for benchmarking workflows.
- Automated Benchmarking – Simplifies running MLPerf Inference benchmarks with minimal manual intervention.
- Modular and Extensible – Easily extend the scripts to support additional benchmarks and configurations.
- Seamless Integration – Compatible with Docker, cloud environments, and local machines.
- MLCFlow (MLC) Integration – Utilizes the MLC framework to enhance reproducibility and automation.
Building on the foundation of its predecessor, the Collective Mind (CM) framework, MLCFlow takes ML workflows to the next level by streamlining complex tasks like Docker container management and caching. The mlcflow package, written in Python, provides seamless support through both a command-line interface (CLI) and an API, making it easy to access and manage automation scripts.
- Script Automation – Automates script execution across different environments.
- Cache Management – Manages reusable cached results to accelerate workflow processes.
We welcome contributions from the community! To contribute:
- Submit pull requests (PRs) to the
devbranch. - Review our CONTRIBUTORS.md for guidelines and best practices.
- Explore more about MLPerf Inference automation in the official MLPerf Inference Documentation.
Your contributions help drive the project forward!
Stay tuned for upcoming updates and announcements.
This project is licensed under the Apache 2.0 License.
This project is made possible through the generous support of:
We appreciate their contributions and sponsorship!
Thank you for your interest and support in MLPerf Automations and Scripts!
