Adding Red Queen CLI

.gitignore

@@ -112,6 +112,7 @@ celerybeat.pid
 # Environments
 .env
 .venv
+.virtualenv
 env/
 venv/
 ENV/

README.md

@@ -1,63 +1,143 @@
-> "Well, in our country," said Alice, still panting a little, "you'd generally 
-> get to somewhere else—if you run very fast for a long time, as we've been 
+> "Well, in our country," said Alice, still panting a little, "you'd generally
+> get to somewhere else—if you run very fast for a long time, as we've been
 > doing."
 >
-> "A slow sort of country!" said the Queen. "Now, here, you see, it takes all 
-> the running you can do, to keep in the same place. If you want to get 
+> "A slow sort of country!" said the Queen. "Now, here, you see, it takes all
+> the running you can do, to keep in the same place. If you want to get
 > somewhere else, you must run at least twice as fast as that!"
 >
 > [Carroll, Lewis: Through the Looking-Glass, Chapter 2](
     https://www.gutenberg.org/files/12/12-h/12-h.htm)
 
-## About
+# About
+
 The Red Queen benchmark framework was created to facilitate the benchmarking
 algorithms used in quantum compilation.
 
-The framework is tightly integrated into `pytest`.  Therefore, to use it 
-effectively, you should know the basics of `pytest` first. Take a look at the 
+The framework is tightly integrated into `pytest`.  Therefore, to use it
+effectively, you should know the basics of `pytest` first. Take a look at the
 [introductory material](https://docs.pytest.org/en/latest/getting-started.html).
 
-## Usage
+# Usage
+
 Red Queen is a framework for benchmarking quantum compilation algorithms. Since
-it was not designed as a package, there is no notion of installation. Hence, you
-must clone this repository to use it:
+is is still in early development, you must clone this repository to use it:
+
 ```bash
 git clone git@github.com:Qiskit/red-queen.git
 ```
 
-To run benchmarks, you must first go to the `red-queen` directory and install 
-the required packages:
+Since Red Queen is a Python package is version control is very important, therefore using a virtual enviornment to run benchmarks is highly recomended.
+
+To get into Red Queen repository
+
 ```bash
 cd red-queen
-pip install -r requirements.txt
 ```
 
+To create a virtual environment
+
+```bash
+python -m venv .virtualenv
+```
+
+After you create the virtual enviornment you need to activate it
+
+To activate the virtual enviornment
+
+[For MacOs and Linux]
+
+```bash
+source .virtualenv/bin/activate
+```
+
+[For Windows]
+
+```bash
+.virtualenv\Scripts\activate.bat
+```
+
+You need to install all of the neccessary dependencies for Red Queen
+
+```bash
+pip install -e .
+```
+
+Lastly `cd` into `red_queen` so that you'll be able to use the `cli` (command line interface)  
+
+```bash
+cd red_queen
+```
+
+With all that you are ready to start using Red Queen CLI.
+
+Red Queen's `CLI` simplifies the execution of benchmarks by constructing the pytest scripts for you all you gotta do is specify the benchmarks that you'd like to run.  
+
+You can find all available benchmark just by running this command
+
+```bash
+red-queen --help
+```
+
+The general templete for the `CLI` is as follows:
+
+```bash
+red-queen -c <complier> -b <benchmark_name>
+```
+
+> Side bar: The complier flag is optional. If you don't specify a complier the `cli` will just run `qiskit`
+
 Now, suppose you want to run the mapping benchmarks using only `tweedledum`.
-You must do it using `pytest`
+You can do this via the `CLI` or with `pytest`
+
+[For MacOs and Linux]
+
+With `CLI`
+
 ```bash
-pytest red_queen/games/mapping/map_queko.py -m tweedledum --store
+red-queen -c tweedledum -b map_queko.py
 ```
 
-To run pytest on Windows, you will have to use `python -m` in order to run the 
-`pytest` command. You will also need to add `-s` to your pytest call to disable 
-stdin handling.
+With `pytest`
+
 ```bash
-python -m pytest -s red_queen/games/mapping/map_queko.py -m tweedledum --store
+python -m pytest games/mapping/map_queko.py -m tweedledum --store
 ```
 
-The benchmark suite will consider all functions named `bench_*` in 
-`red_queen/games/mapping/map_queko.py`. Because we set the `-m` option, only the the ones
-marked with `tweedledum` will be run. (We could easy do the same for `qiskit`).
-If you don't define a `-m` option, all `bench_*` functions will be run.
+[For Windows]
+
+With `CLI`
 
+```bash
+red-queen -c tweedledum -b map_queko.py
+```
+
+With `pytest`
+
+```bash
+python -m pytest -s games/mapping/map_queko.py -m tweedledum --store
+```
+
+To run pytest or any python script on Windows, you will have to use `python -m` in order to run the
+`pytest` command. You will also need to add `-s` to your pytest call to disable
+stdin handling.
+
+```bash
+python -m pytest -s games/mapping/map_queko.py -m tweedledum --store
+```
+
+The benchmark suite will consider all functions named `bench_*` in
+`games/mapping/map_queko.py`. Because
 The `--store` option tells the framework to store the results in json file in
 the `results` directory. To see the results as a table, you can use the you can
 use:
+
 ```bash
 python -m report.console_tables --storage results/0001_bench.json
 ```
 
 ## Warning
+
 This code is still under development. There are many razer sharp edges.
 
 For information of how execution works and other details about the framwork
@@ -73,7 +153,7 @@ on the knowledge of the internals of the following established `pytest` plugins:
 
 ## License
 
-This software is licensed under the Apache 2.0 licence (see 
+This software is licensed under the Apache 2.0 licence (see
 [LICENSE](https://github.com/Qiskit/red-queen/blob/main/LICENSE))
 
 ## Contributing

red_queen/cli.py

@@ -0,0 +1,163 @@
+#!/usr/bin/env python3
+# ------------------------------------------------------------------------------
+# Part of Red Queen Project.  This file is distributed under the Apache 2.0.
+# See accompanying file /LICENSE for details.
+# ------------------------------------------------------------------------------
+
+"""The purpose of this code below is to make the user experience of
+the Red Queen benchmark suite more steamlined
+
+The code achieves that by collecting all avaiable benchmark alongside their paths,
+and uses this information to create exacutable pytest code that
+will exacute the benchmarks for users with worrying about the
+nuisances ofthe pytest framework. The scope of this cli will
+grow with time.
+"""
+import os
+import platform
+import subprocess
+import configparser
+import click
+
+# This function retrives the paths for each benchmark and returns
+# a hashmap with the benchmark's name and it's path
+# It also retrives all available compliers for Red Queen
+def benchmark_complier_retrieval():
+    benchmark_dict = {}
+    list_of_benchmarks = []
+    list_of_compliers = set()
+    benchmark_category_set = set()
+    benchmark_path = "games/"
+    for benchmark_category in os.scandir(benchmark_path):
+        if benchmark_category.is_dir():
+            benchmark_category_set.add(benchmark_category.name)
+            name_path_pair = {}
+            for benchmark in os.scandir(f"{benchmark_path}{benchmark_category.name}"):
+                if (
+                    not benchmark.name.startswith("_")
+                    and benchmark.name.endswith(".py")
+                    and benchmark.is_file()
+                ):
+                    name_path_pair[benchmark.name] = benchmark.path
+            benchmark_dict.update(name_path_pair)
+
+    for benchmark_name, _ in benchmark_dict.items():
+        list_of_benchmarks.append(benchmark_name)
+
+    # Config helps us process the .ini file
+    config = configparser.ConfigParser()
+    config.read("../pytest.ini")
+    for complier in config["pytest"]["markers"].split("\n"):
+        if complier != "":
+            list_of_compliers.add(complier)
+    return benchmark_dict, list_of_benchmarks, list_of_compliers
+
+
+# The function retrives the latest benchmark ran's results
+def result_retrieval():
+    results = {}
+    dir_path = "results"
+    for result in os.scandir(dir_path):
+        if result.is_file():
+            filename = result.name
+            result_count = int(filename.split("_")[0])
+            results[result_count] = {result.name: result.path}
+    return results
+
+
+# This functions creates the pytest command and runs it
+def run_benchmarks(pytest_paths: str, m_tag: str, compiler: str):
+    command_list = ["pytest"]
+    compiler_command = [m_tag, compiler, "--store"]
+    # Are you using a Windows Machine?
+    if platform.system() == "Windows":
+        command_list.insert(0, "-m")
+        command_list.insert(0, "python")
+        command_list.append("-s")
+        for _, string in enumerate(pytest_paths):
+            command_list.append(string)
+        for _, string in enumerate(compiler_command):
+            command_list.append(string)
+        subprocess.run(
+            command_list,
+            check=True,
+        )
+    else:
+        command_list.insert(0, "-m")
+        command_list.insert(0, "python")
+        for _, string in enumerate(pytest_paths):
+            command_list.append(string)
+        for _, string in enumerate(compiler_command):
+            command_list.append(string)
+        subprocess.run(
+            command_list,
+            check=True,
+        )
+
+
+# This function displays the results
+def show_result():
+    results_dict = result_retrieval()
+    result_num = max(results_dict.keys())
+    result_path = tuple(results_dict[result_num].values())[0]
+    command_list = ["python", "-m", "report.console_tables", "--storage"]
+    command_list.append(str(result_path))
+    subprocess.run(
+        command_list,
+        check=True,
+    )
+    click.echo("To view the table again:")
+    click.echo(" ".join(command_list) + "\n")
+
+
+benchmark_hash, benchmarks_list, complier_list = benchmark_complier_retrieval()
+
+
+# The code below deals with each flag of the cli
+@click.command()
+@click.option(
+    "-c",
+    "--compiler",
+    "compiler",
+    multiple=True,
+    type=click.Choice(complier_list),
+    help="enter a compliler here",
+)
+@click.option(
+    "-b",
+    "--benchmark",
+    "benchmark",
+    multiple=True,
+    type=click.Choice(benchmarks_list),
+    help="enter the specfic benchmark(s) here",
+)
+# The main function calls on the above helper functions to run the desired benchmarks
+def main(compiler=None, benchmark=None):
+    benchmark_paths = []
+    pytest_paths = ""
+    m_tag = ""
+    # Is there a specfic compiler listed?
+    if len(compiler) > 0:
+        m_tag = "-m"
+        compiler = compiler[0]
+    else:
+        compiler = ""
+    j = 0
+    i = 0
+    ### Has the user input benchmarks?
+    if len(benchmark) > 0:
+        ### Are the inputted benchmark(s) valid?
+        if set(benchmark).issubset(benchmarks_list):
+            for j, _ in enumerate(benchmark):
+                benchmark_paths.append(benchmark_hash[benchmark[j]])
+                pytest_paths = benchmark_paths
+                run_benchmarks(pytest_paths, m_tag, compiler)
+                show_result()
+                i += 1
+    else:
+        print("Please input benchmark")
+        print("Example:\nred-queen -b grovers.py")
+
+
+if __name__ == "__main__":
+    main()

requirements.txt

@@ -6,4 +6,5 @@ qiskit-terra
 qiskit-aer
 pytket>1.0,<2.0
 setproctitle
-rich
+rich
+click

setup.py

@@ -59,5 +59,10 @@
         "Documentation": "https://qiskit.org/documentation/",
         "Source Code": "https://github.com/Qiskit/red-queen",
     },
+    entry_points={
+        'console_scripts': [
+            "red-queen = red_queen.cli:main",
+        ],
+    },
     zip_safe=False,
 )