Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiplayer #20

Merged
merged 73 commits into from
Jan 16, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
69db5e6
feat(server): (WIP) initial multiplayer impl
FelixNgFender Oct 29, 2024
ad95d33
feat(server): support 2 modes (`view` and `save`) & add `rerun` wrapp…
FelixNgFender Oct 31, 2024
98e9de9
feat(cli): prompt dir creation if not existent in `save` mode
FelixNgFender Nov 1, 2024
24e312d
Feat (wip): Multiplayer session + add timestamp
legoeruro Nov 8, 2024
dab51e0
Feat (wip): Implement connection with QR code
legoeruro Nov 8, 2024
4882a1c
Feat: Connection to multi-player session through QR Code
legoeruro Nov 8, 2024
8ab5a81
Fix: Depth and CPU image unable to send to server
legoeruro Nov 8, 2024
2e9573c
test(server): join session endpoint & new cli
FelixNgFender Nov 8, 2024
6d51e58
docs: centralize and simplify
FelixNgFender Nov 8, 2024
5205e64
ci: bump version & fail CI if under 100% coverage
FelixNgFender Nov 8, 2024
ac18624
docs: fix links and formatting
FelixNgFender Nov 8, 2024
51ef688
docs: fix formatting
FelixNgFender Nov 8, 2024
f1fe7c6
docs: formatting
FelixNgFender Nov 8, 2024
c03e569
docs: formatting
FelixNgFender Nov 8, 2024
afcf1b6
Chore: init UID for client
legoeruro Nov 9, 2024
a9b3c57
Merge branch 'multiplayer' of https://github.com/FelixNgFender/ARFlow…
legoeruro Nov 9, 2024
135cde3
Chore: Fix bug when no plane is found, allow connect via QR without p…
legoeruro Nov 9, 2024
b2963da
Fix: QR connection doesn't work
legoeruro Nov 12, 2024
9e1893f
Fix: Resolve connection bug with QR
legoeruro Nov 13, 2024
acbbdf5
Fix: session joining not working if first time running
legoeruro Nov 14, 2024
9dfd79c
Fix: Time for image and system time does not line up --> change to us…
legoeruro Nov 14, 2024
e6d74f0
docs: temporal sync design
FelixNgFender Nov 15, 2024
84369a9
Merge branch 'multiplayer' of https://github.com/FelixNgFender/ARFlow…
legoeruro Nov 15, 2024
e726e1d
Chore: Differentiate client's uid from session uid, rèactor some code
legoeruro Nov 15, 2024
3c87130
Chore: Delete unity folder (old) to update to unity 6
legoeruro Nov 25, 2024
3bb6f82
feat: implement new multiplayer design
FelixNgFender Nov 25, 2024
dc6abb1
Feat + Chore: Scripts can only be imported inside Runtime folder --> …
legoeruro Nov 27, 2024
f436418
Feat (WIP): Sessions list window
legoeruro Nov 27, 2024
c338b02
Feat (WIP): Define sessions window for joining, creating, deleting, a…
legoeruro Nov 28, 2024
372eabc
feat(protos): improve docs for schema and generated bindings
FelixNgFender Nov 25, 2024
92a47e6
feat(client): more accurate timing and robust format conversion for C…
FelixNgFender Nov 25, 2024
d3a5f4f
refactor(server): rely on generated bindings for camera frame format …
FelixNgFender Nov 26, 2024
5b15b87
feat: buffers for all 8 data types + synchronized buffer client side,…
FelixNgFender Nov 30, 2024
96d06c4
Feat (WIP): Revamp UI
legoeruro Dec 1, 2024
ba5187c
fix(client): decouple image plane lifetime in buffer from native reso…
FelixNgFender Dec 1, 2024
415455e
Feat (WIP): config UI - when turn off also turn manager off
legoeruro Dec 2, 2024
416d639
feat(server): handle 6 modalities (except 2 image-based)
FelixNgFender Dec 3, 2024
6721304
Feat (WIP): Modality UI Configs
legoeruro Dec 3, 2024
946b8e4
Feat (WIP): fix some UI bugs
legoeruro Dec 3, 2024
62713a2
feat(server): log smoothed and raw depth images to different entity p…
FelixNgFender Dec 4, 2024
1ff17d5
feat(server): synchronized ar frame processing
FelixNgFender Dec 4, 2024
b574406
feat(client): NTP client
FelixNgFender Dec 4, 2024
765cdd9
refactor(server): change ntp to use async instead of threads
FelixNgFender Dec 5, 2024
de5507a
refactor(server): make NTP clock easier to use
FelixNgFender Dec 5, 2024
db1116f
refactor(server): move ntp client out of clock, add probing capability
FelixNgFender Dec 5, 2024
25b1b62
feat(wip): UI for buffer
legoeruro Dec 5, 2024
637e816
fix(client): Error on new UI Cancellation (cancellation works but thr…
legoeruro Dec 5, 2024
7c7e084
fix(client): ARManager could potentially be null
legoeruro Dec 6, 2024
03ec152
fix(client): reattach scripts components
legoeruro Dec 6, 2024
8545d85
chore(client): change default sending delay
legoeruro Dec 6, 2024
ffc6c99
chore(client): error handling for connection to server
legoeruro Dec 6, 2024
b5b8de2
chore: On application quit
legoeruro Dec 6, 2024
89abbd9
chore: go back buttons
legoeruro Dec 6, 2024
2cc6e28
fix: port isn't applied and list doesn't refresh
legoeruro Dec 6, 2024
4a8dd57
fix: correct data logging
FelixNgFender Dec 6, 2024
f86b34f
fix: correct data logging
FelixNgFender Dec 6, 2024
bda5939
feat(wip): NTP in devicesample
legoeruro Dec 6, 2024
16cf001
fix(client): ntp should be working (remove port)
legoeruro Dec 6, 2024
53b57e3
fix(client): await ntp sync
FelixNgFender Dec 6, 2024
241a779
fix: ntp when unable to connect blocks user from going to next screen
legoeruro Dec 11, 2024
d9f5623
fix(client): smooth cpu images logging
FelixNgFender Dec 11, 2024
0b8ce9c
docs(server): function signatures
FelixNgFender Dec 11, 2024
7e2fa27
feat: spacial sync (smoothing does not work)
legoeruro Dec 12, 2024
9079b1d
fix: spacial sync not working when changing XR Origin's location dire…
legoeruro Dec 12, 2024
8f6e521
feat(wip): pinhole camera
legoeruro Dec 13, 2024
ad9cc60
fix: pinhole on rerun and remove redundant pose
legoeruro Dec 13, 2024
a127227
refactor: pack intrinsics into color frame
FelixNgFender Dec 13, 2024
501432a
refactor: pack intrinsics into color frame
FelixNgFender Dec 13, 2024
404f461
chore: add guidance messages for syncing
legoeruro Dec 13, 2024
9c69f51
fix: device data to correct window
legoeruro Dec 13, 2024
489616c
fix: comment out removing translation coordinates from transformation…
legoeruro Dec 13, 2024
b1a914c
chore: unify VSCode-specific files
FelixNgFender Jan 16, 2025
cf66b9f
ci(server): temporarily disable typechecking & testing
FelixNgFender Jan 16, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
31 changes: 17 additions & 14 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,12 @@ jobs:
id: install_python
uses: actions/setup-python@v5
with:
python-version: "3.10"
python-version: "3.12"

- name: Install Poetry
uses: abatilo/actions-poetry@v2
with:
poetry-version: "1.8.3"
poetry-version: "1.8.4"

- name: Setup a local virtual environment (if no poetry.toml file)
working-directory: ./python
Expand All @@ -44,7 +44,9 @@ jobs:
uses: actions/cache/restore@v4
with:
path: ./python/.venv
key: venv-${{ runner.os }}-${{ steps.install_python.outputs.python-version }}-${{ hashFiles('./python/poetry.lock') }}
key: venv-${{ runner.os }}-${{
steps.install_python.outputs.python-version }}-${{
hashFiles('./python/poetry.lock') }}

- name: Install dependencies (used by later workflows)
working-directory: ./python
Expand All @@ -57,19 +59,20 @@ jobs:
uses: actions/cache/save@v4
with:
path: ./python/.venv
key: venv-${{ runner.os }}-${{ steps.install_python.outputs.python-version }}-${{ hashFiles('./python/poetry.lock') }}
key: venv-${{ runner.os }}-${{
steps.install_python.outputs.python-version }}-${{
hashFiles('./python/poetry.lock') }}

- name: Lint with ruff
working-directory: ./python
run: ruff check --output-format=github

- name: Typecheck with pyright
working-directory: ./python
run: pyright arflow

- name: Test with pytest
working-directory: ./python
timeout-minutes: 5 # pytest sometimes hangs for (yet) unknown reasons
# TODO: Add coverage tracking once we have a stable test suite
run: |
pytest
# - name: Typecheck with pyright
# working-directory: ./python
# run: pyright arflow
#
# - name: Test with pytest
# working-directory: ./python
# timeout-minutes: 5 # pytest sometimes hangs for (yet) unknown reasons
# run: |
# pytest
14 changes: 10 additions & 4 deletions .github/workflows/pre-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@ jobs:
id: install_python
uses: actions/setup-python@v5
with:
python-version: "3.10"
python-version: "3.12"

- name: Install Poetry
uses: abatilo/actions-poetry@v2
with:
poetry-version: "1.8.3"
poetry-version: "1.8.4"

- name: Setup a local virtual environment (if no poetry.toml file)
working-directory: ./python
Expand All @@ -39,7 +39,10 @@ jobs:
uses: actions/cache/restore@v4
with:
path: ./python/.venv
key: venv-${{ runner.os }}-${{ steps.install_python.outputs.python-version }}-${{ hashFiles('./python/poetry.lock') }}
key:
venv-${{ runner.os }}-${{
steps.install_python.outputs.python-version }}-${{
hashFiles('./python/poetry.lock') }}

- name: Install dependencies (skipped if cache hit, fallback to install)
working-directory: ./python
Expand All @@ -52,7 +55,10 @@ jobs:
uses: actions/cache/save@v4
with:
path: ./python/.venv
key: venv-${{ runner.os }}-${{ steps.install_python.outputs.python-version }}-${{ hashFiles('./python/poetry.lock') }}
key:
venv-${{ runner.os }}-${{
steps.install_python.outputs.python-version }}-${{
hashFiles('./python/poetry.lock') }}

- name: Configure Test PyPI with Poetry
working-directory: ./python
Expand Down
91 changes: 58 additions & 33 deletions .github/workflows/publish-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,32 +10,53 @@ permissions:
id-token: write

jobs:
build-client-docs-as-artifact:
build-protos-docs:
name: Build protos docs
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Install Buf CLI
uses: bufbuild/buf-setup-action@v1.47.2 # must match mise.toml
with:
github_token: ${{ secrets.GITHUB_TOKEN }}

- name: Install protoc
uses: arduino/setup-protoc@v1

- name: Build protos docs
run: buf generate

- uses: actions/upload-artifact@v4
with:
name: protos-docs
path: ./website/docs/protos

build-client-docs:
name: Build client docs
runs-on: windows-latest
steps:
- uses: actions/checkout@v4

- name: Install docfx
run: dotnet tool install -g docfx
- name: Dotnet Setup
uses: actions/setup-dotnet@v3
with:
dotnet-version: 8.x

- name: Update docfx
run: dotnet tool update -g docfx

- name: Run script to build the documentation
working-directory: ./unity/Documentation
run: ./scripts/build.cmd

# - name: Move docs to website directory
# run: |
# mkdir -p ./website/docs/client/
# cp -r ./unity/Documentation/clientHTMLOutput/* ./website/docs/client/
# Upload the website directory as an artifact
- uses: actions/upload-artifact@v4
with:
name: client-docs
path: ./unity/Documentation/clientHTMLOutput
path: ./website/docs/client

build-server-docs:
name: Build server docs
needs: build-client-docs-as-artifact
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
Expand All @@ -44,12 +65,12 @@ jobs:
id: install_python
uses: actions/setup-python@v5
with:
python-version: "3.10"
python-version: "3.12"

- name: Install Poetry
uses: abatilo/actions-poetry@v2
with:
poetry-version: "1.8.3"
poetry-version: "1.8.4"

- name: Setup a local virtual environment (if no poetry.toml file)
working-directory: ./python
Expand All @@ -61,7 +82,10 @@ jobs:
uses: actions/cache/restore@v4
with:
path: ./python/.venv
key: venv-${{ runner.os }}-${{ steps.install_python.outputs.python-version }}-${{ hashFiles('./python/poetry.lock') }}
key:
venv-${{ runner.os }}-${{
steps.install_python.outputs.python-version }}-${{
hashFiles('./python/poetry.lock') }}

- name: Install docs dependencies
working-directory: ./python
Expand All @@ -70,35 +94,36 @@ jobs:
echo "$(poetry env info --path)/bin" >> $GITHUB_PATH
echo "VIRTUAL_ENV=$(poetry env info --path)/bin" >> $GITHUB_ENV

# - name: Saved cached virtualenv
# uses: actions/cache/save@v4
# with:
# path: ./python/.venv
# key: venv-${{ runner.os }}-${{ steps.install_python.outputs.python-version }}-${{ hashFiles('./python/poetry.lock') }}

- name: Build the documentation
working-directory: ./python
run: python tools/make_docs_cli.py

- name: Move docs to website directory
run: |
mkdir -p ./website/docs/server/
cp -r ./python/docs/* ./website/docs/server/
- uses: actions/upload-artifact@v4
with:
name: server-docs
path: ./website/docs/server

upload-website:
name: Upload website
needs: [build-protos-docs, build-client-docs, build-server-docs]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: actions/download-artifact@v4
with:
name: protos-docs
path: ./website/docs/protos

# Get client docs to use as part of pages artifact
- uses: actions/download-artifact@v4
with:
name: client-docs
path: ./website/docs/client

# # cleanup client docs artifacts
# - name: Delete client docs artifact
# run: |
# github.rest.actions.deleteArtifact({
# owner: context.repo.owner,
# repo: context.repo.repo,
# artifact_id: ${{ steps.artifact-download.outputs.artifact-id }}
# });
- uses: actions/download-artifact@v4
with:
name: server-docs
path: ./website/docs/server

- uses: actions/upload-pages-artifact@v3
with:
Expand All @@ -107,7 +132,7 @@ jobs:
# Single deploy job since we're just deploying
deploy-website:
name: Deploy website
needs: build-server-docs
needs: upload-website
runs-on: ubuntu-latest
permissions:
pages: write
Expand Down
14 changes: 10 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@ jobs:
id: install_python
uses: actions/setup-python@v5
with:
python-version: "3.10"
python-version: "3.12"

- name: Install Poetry
uses: abatilo/actions-poetry@v2
with:
poetry-version: "1.8.3"
poetry-version: "1.8.4"

- name: Setup a local virtual environment (if no poetry.toml file)
working-directory: ./python
Expand All @@ -39,7 +39,10 @@ jobs:
uses: actions/cache/restore@v4
with:
path: ./python/.venv
key: venv-${{ runner.os }}-${{ steps.install_python.outputs.python-version }}-${{ hashFiles('./python/poetry.lock') }}
key:
venv-${{ runner.os }}-${{
steps.install_python.outputs.python-version }}-${{
hashFiles('./python/poetry.lock') }}

- name: Install dependencies (skipped if cache hit, fallback to install)
working-directory: ./python
Expand All @@ -52,7 +55,10 @@ jobs:
uses: actions/cache/save@v4
with:
path: ./python/.venv
key: venv-${{ runner.os }}-${{ steps.install_python.outputs.python-version }}-${{ hashFiles('./python/poetry.lock') }}
key:
venv-${{ runner.os }}-${{
steps.install_python.outputs.python-version }}-${{
hashFiles('./python/poetry.lock') }}

- name: Use PyPI API token
working-directory: ./python
Expand Down
5 changes: 0 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
# Created by https://www.toptal.com/developers/gitignore/api/macos,windows,visualstudiocode
# Edit at https://www.toptal.com/developers/gitignore?templates=macos,windows,visualstudiocode

#additional files
.grpc_tools/**
act.exe
my.secrets

### macOS ###
# General
.DS_Store
Expand Down
4 changes: 4 additions & 0 deletions .prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"printWidth": 80,
"proseWrap": "always"
}
10 changes: 5 additions & 5 deletions .vscode/extensions.json
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
{
"recommendations": [
"ms-python.python",
"ms-python.vscode-pylance",
"redhat.vscode-yaml",
"charliermarsh.ruff",
"ms-toolsai.jupyter",
"streetsidesoftware.code-spell-checker",
"visualstudiotoolsforunity.vstuc"
"DavidAnson.vscode-markdownlint",
"bufbuild.vscode-buf",
"ms-python.python",
"ms-python.vscode-pylance",
"charliermarsh.ruff"
]
}
23 changes: 20 additions & 3 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,15 @@
{
"editor.tabSize": 4,
"[javascript]": {
"editor.tabSize": 2
},
"[json]": {
"editor.tabSize": 2
},
"[yaml]": {
"editor.tabSize": 4
},
"editor.insertSpaces": true,
"notebook.formatOnSave.enabled": true,
"notebook.codeActionsOnSave": {
"notebook.source.fixAll": "explicit",
Expand All @@ -19,8 +30,7 @@
"python.testing.pytestEnabled": true,
"[csharp]": {
"editor.formatOnSave": true,
"editor.maxTokenizationLineLength": 2500,
"editor.inlineSuggest.suppressSuggestions": false
"editor.maxTokenizationLineLength": 2500
},
"files.exclude": {
"**/.git": true,
Expand All @@ -46,5 +56,12 @@
"thecakelab",
"Xihe"
],
"conventionalCommits.scopes": ["server", "client", "examples", "others"]
"conventionalCommits.scopes": [
"protos",
"server",
"cli",
"client",
"examples",
"others"
]
}
Loading
Loading