[Benchmark] Remove Java benchmarking or pull code from arrow-java to fix failing builds

[assignUser]

Describe the bug, including details regarding any error messages, version, and platform.

Java benchmarks are failing because java/ was moved to arrow-java:

https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-amd64-m5-4xlarge-linux/builds/562#0193ac96-8042-4083-aeab-4d47355004b6
https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-amd64-c6a-4xlarge-linux/builds/564#0193ac96-729c-4009-a904-005b9401185a/6-13393

We should pull the code from arrow-java for now similar to #44980

Component(s)

Benchmarking

[assignUser]

cc @raulcd @kou

[kou]

Where are Buildkite configuration files?

[raulcd]

Does someone know where is ursabot running? I struggle to understand the workflow followed to exercise the benchmarks. The following documentation doesn't give me a clear picture to be honest:
https://github.com/apache/arrow/tree/main/dev/conbench_envs

[assignUser]

I think apache/arrow sends merge/comment events as a webhook to ursabot, not sure where it runs or which form it has (i.e. simple script or probot or something).

[assignUser]

The buildkite config is here @kou . I will disable the java benchmarks for now, I think we can implement them in the repo and upload the results to conbench similar how we did it for go/planning to do it in arrow-go

[kou]

Thanks! Cc: @singh1203