[Topic] Rewrite benchmark-operator and limit scope

[jtaleric]

Benchmark Operator has been the Swiss Army Knife of k8s-benchmarking. However, overtime it has become extremely difficult to debug, maintain and contribute to. Similarly, we have built some other tooling to help mitigate the need to continue enhancing benchmark-operator.

After some internal evaluation, we still see the need to keep aspects of the benchmark-operator alive -- mainly for its ability to run storage based workloads. (fio, ycsb, and hammerdb).

I am suggesting we :

1. move away from the ansible-sdk mainly for the lack of easy debug. The output from the playbook execution is very busy, and it is hard to triage where issues manifest from.
2. Much of the benchmark knowledge is in https://github.com/cloud-bulldozer/benchmark-wrapper, so we could continue to use that (or maybe specific aspects of it).
3. since we move away from ansible-sdk, rewrite the benchmark-operator using the golang framework to manage the deployment of the pods running the workloads.
4. minimize the scope of benchmark-operator to be more storage focused.

[ddomingu]

A pluggable operator for performance testing with multiple tools is exactly what's needed in the field. The cloud-benchmark operator isn’t widely used because it seems abandoned, but it’s still a solid and functional tool. The experience varies depending on the plugin: FIO works great, while others may not be as smooth. Without a centralized operator, people in the field have to juggle multiple tools instead of relying on a single solution for metric aggregation and execution control across different plugins.

Rewriting the operator in Go and simplifying it with a stronger focus on storage would be a huge help. An operator capable of running various storage benchmarking tools would be ideal. +1 to this idea!