[Improvement] support different Docker versions for integration tests

[youngyjd]

What would you like to be improved?

1. List what docker versions are required to run the integration tests.
2. Support Docker version < 20.10+ if possible. Our CI runs Docker version 19.03.15 that does not have host-gateway feature required in trino integration test -

   gravitino/integration-test-common/docker-script/docker-compose.yaml

   Line 95 in e8d06f4

   - "host.docker.internal:host-gateway"

3. Support Compose V1 if possible. Our CI runs Compose V1. docker compose up -d errored out in our CI pipeline.

For 3, I managed to bypass it by installing Compose V2 through

mkdir -p ~/.docker/cli-plugins/
curl -SL https://github.com/docker/compose/releases/latest/download/docker-compose-linux-x86_64 -o ~/.docker/cli-plugins/docker-compose
chmod +x ~/.docker/cli-plugins/docker-compose

For 2, it is harder since our CI user does not have sudo access. While I am working with our CI team to find a solution, is there any plan to have the integration tests compatible with older version Docker?

How should we improve?

No response

[youngyjd]

Seems there is a potential solution for older version Docker? docker/cli#2664

[tengqm]

I'm wondering if we should try [podman](https://podman.io/] container engine. The benefits I can think of:

• The whole engine is open-source, unlike docker
• Almost full compatibility with Docker, including the command line
• There are migration path from Docker and Docker compose
• It doesn't require a daemon to run containers
• It supports running containers rootless mode
• It is lightweight
• It can be integrated seamlessly with Kubernetes