Skip to content

Commit

Permalink
Add images
Browse files Browse the repository at this point in the history
  • Loading branch information
Marti2203 authored and rshariffdeen committed Apr 16, 2023
1 parent 84b77d2 commit 3c24223
Show file tree
Hide file tree
Showing 6 changed files with 26 additions and 21 deletions.
16 changes: 8 additions & 8 deletions doc/Examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,49 +3,49 @@
In this document we provide examples of how to use Cerberus with prepared test cases to elaborate the capabilities of Cerberus. Following details explain the test-cases we provide in our repository. Please take a look at the page Getting Started to understand how to use Cerberus.

```bash
cerberus --tool=extractfix --benchmark=vulnloc --bug-index=1
cerberus repair --tool=extractfix --benchmark=vulnloc --bug-index=1
```

This is one of the most basic commands -running ExtractFix on the first vulnerability in Vulnloc.

```bash
cerberus --tool=extractfix --benchmark=vulnloc --bug-index=1 --debug
cerberus repair --tool=extractfix --benchmark=vulnloc --bug-index=1 --debug
```

Simarly we run the tool with debug mode on.

```bash
cerberus --tool=extractfix --benchmark=vulnloc --bug-index-list=1-2,5,10-20
cerberus repair --tool=extractfix --benchmark=vulnloc --bug-index-list=1-2,5,10-20
```

Now let us batch more bugs - the first two bugs, the fifth bug and then from the 10th to the 20th bug.

```bash
cerberus --tool=prophet --benchmark=vulnloc --bug-index-list=1-2,5,10-20 --local
cerberus repair --tool=prophet --benchmark=vulnloc --bug-index-list=1-2,5,10-20 --local
```

Let's assume that we want to run the experiments locally, only thing that needs to be added is the --local and one can ensure that prophet is locally accessible.

```bash
cerberus --tool=prophet --benchmark=vulnloc --bug-index=4 --conf=C4
cerberus repair --tool=prophet --benchmark=vulnloc --bug-index=4 --conf=C4
```

The experiment can also be ran with a different configuration profile.

```bash
cerberus --tool=prophet --benchmark=vulnloc --bug-index=4 --setup-only
cerberus repair --tool=prophet --benchmark=vulnloc --bug-index=4 --setup-only
```

Or until the setup stage only.

```bash
cerberus --tool=prophet --benchmark=vulnloc --bug-index=4 --instrument-only
cerberus repair --tool=prophet --benchmark=vulnloc --bug-index=4 --instrument-only
```

Or until the instrumentation stage only.

```bash
cerberus --tool=prophet --benchmark=vulnloc --bug-index=4 --rebuild-exp
cerberus repair --tool=prophet --benchmark=vulnloc --bug-index=4 --rebuild-exp
```

If needed one can also rebuild the experiment image.
31 changes: 18 additions & 13 deletions doc/GetStart.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

Let's walk through a simple example and a more complicated one to understand how to use Cerberus. One will run a single experiment sequentially, while the latter will run multiple experiments on multiple tools.

Cerberus will do the following actions in order when given a tool list and experiment list:

* Pull the images for each tool in the list, if they are not found locally or are outdated;
* Prepare a image for the benchmark, if one was not already created;
* Prepare an image for each subject, if one was not already created.
* If sequentially running (by default), each experiment is ran on each tool in an isolated container (or locally if selected).

## Sequential

We consider the [libtiff project](https://github.com/vadz/libtiff/) from the VulnLoc benchmark as a medium-sized example.
Expand All @@ -10,7 +17,9 @@ We consider the [libtiff project](https://github.com/vadz/libtiff/) from the Vul
cerberus repair --tool vulnfix --benchmark=vulnloc --bug-index=22
```

We have selected to run the vulnfix tool on the 22nd bug of the vulnloc benchmark. Cerberus will prepare everything needed and run the tool on the selected subject.
![Sequential Run](images/SequentialUI.png)

We have selected to run the vulnfix tool on the 22nd bug of the vulnloc benchmark. Cerberus will prepare everything needed and run the tool on the selected subject.

## Parallel

Expand All @@ -22,21 +31,17 @@ Let us run some of them - vulnfix, extractfix, f1x and fix2fit and 4 of the bugs
cerberus repair --tool-list vulnfix f1x fix2fit extractfix --benchmark=vulnloc --bug-index-list=22-26 -g
```

Cerberus will do the following actions in order:
Cerberus will allocate 4*4 = 16 tasks and display 2 windows - the first one is the log of the currently selected task, this by default is the first allocated, and the second window is a table showing all tasks. If the `--debug` flag is passed there will is a third window collecting all textual messages from all tasks, showing the messages in chronological order.

* Pull the images for each tool in the list, if they are not found locally or are outdated;
* Prepare a image for the benchmark, if one was not already created;
* Prepare an image for each subject, if one was not already created.
* Allocate 4*4 = 16 tasks and display 2 windows - the first one is the log of the currently selected task, this by default is the first allocated, and the second window is a table showing all tasks. If the `--debug` flag is passed there will is a third window collecting all textual messages from all tasks, showing the messages in chronological order.

Now Cerberus is running, one can easily filter the table by clicking on the buttons in the footer or click the accelerators or clicking on the different rows of the table to select the different text logs.
![Tool Preparation Stage](images/UIPrepareTools.png)

## When complete
![Text log](images/TextLog.png)

When running each tool, the output from the run of the tool is stored in the `output/artifacts` folder and the logs in the `output/logs`. Each experiment run has a unique identifier, which can be seen from the first lines of the output of Cerberus. The folder of the artifact contains patches, output of the tool and extra files if defined by the driver. One can also use the symbolic link `TOOL-vulnloc-bugzilla-2633-last` where `TOOL` is the respective name of the tool to more easily access the run. This is a concrete instance of the patern pattern `TOOL-BENCHMARK-EXPERIMENT-last`.
![UI Running](images/TaskList.png)

Extractfix finished. Now I would like to see what exactly happened behind the scenes. Let us run it with debug and see the output.
Now Cerberus is running, one can easily filter the table by clicking on the buttons in the footer or click the accelerators or clicking on the different rows of the table to select the different text logs. By default Cerberus uses 2 less than the maximum amount of cores for task allocation to ensure that the system is not overloaded. One can easily change this amount with the `--cpus` flag and achieve sequential execution with the UI active by setting the cpus to 1.

```bash
cerberus --tool=extractfix --benchmark=vulnloc --bug-index=22 --debug
```
## When complete

When running each tool, the output from the run of the tool is stored in the `output/artifacts` folder and the logs in the `output/logs`. Each experiment run has a unique identifier, which can be seen from the first lines of the output of Cerberus. The folder of the artifact contains patches, output of the tool and extra files if defined by the driver. One can also use the symbolic link `TOOL-vulnloc-bugzilla-2633-last` where `TOOL` is the respective name of the tool to more easily access the run. This is a concrete instance of the patern pattern `TOOL-BENCHMARK-EXPERIMENT-last`.
Binary file added doc/images/SequentialUI.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/images/TaskList.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/images/TextLog.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added doc/images/UIPrepareTools.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 3c24223

Please sign in to comment.