Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix/tokenizer sampling #106

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Fix/tokenizer sampling #106

wants to merge 2 commits into from

Conversation

fenneishi
Copy link

@fenneishi fenneishi commented Feb 16, 2025 •
edited
Loading

Description

This PR addresses issue #102

Comprehensive Testing

Tested all video tokenizer configurations:

  • Cosmos-Tokenizer-0.1: CV4x8x8, CV8x8x8, CV8x16x16, DV4x8x8, DV8x8x8, DV8x16x16
  • Cosmos-Tokenizer-1.0: CV8x8x8, DV8x16x16

Validation includes:

  1. Automated shape verification
    • Latent space dimensions
    • Reconstruction output dimensions
  2. Interactive quality assessment
    • Side-by-side visual comparison
    • Human evaluation interface
    • Manual quality verification

image

pytest -s -v .\cosmos1\models\tokenizer\tests\test_tokenizer.py

Tokenizer Test Results.pdf

@fenneishi
fix: correct spatial resolution tracking and assertions
605d66d 
- Only update curr_res when spatial down-sampling occurs
- Fix assertion limits for num_spatial/temporal_downs to num_resolutions-1
@fenneishi
test: add tokenizer test
a87fba5 
Model tested configs:
- Cosmos-0.1-Tokenizer-CV4x8x8
- Cosmos-0.1-Tokenizer-CV8x8x8
- Cosmos-0.1-Tokenizer-CV8x16x16
- Cosmos-0.1-Tokenizer-DV4x8x8
- Cosmos-0.1-Tokenizer-DV8x8x8
- Cosmos-0.1-Tokenizer-DV8x16x16
- Cosmos-1.0-Tokenizer-CV8x8x8
- Cosmos-1.0-Tokenizer-DV8x16x16

Automated checks:
- Validate latent space dimensions for each config.
- Verify reconstruction output shape match input for each config.

Quality assurance:
- Add interactive human evaluation interface.
- Visual comparison between input and reconstruction.
- Manual quality verification with accept/reject options.
@fenneishi fenneishi marked this pull request as ready for review February 16, 2025 09:48
@fenneishi fenneishi mentioned this pull request Feb 16, 2025
Closed
@sophiahhuang sophiahhuang added the enhancement New feature or request label Feb 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@fenneishi @sophiahhuang