Moving people and objects in the video

[gigasurgeon]

Thanks for this great work. I am working on a camera path estimation problem. I have a video where people are moving and performing some actions. The camera also is moving. What changes do I need to handle this?

[jytime]

Hi @gigasurgeon ,

We support filtering out dynamic objects using masks. Please place the image files in the SCENE_DIR/images directory and the corresponding mask files in the SCENE_DIR/masks directory. The mask files should have the same base name as the image files. You can refer to how the masks are loaded in the code here:

vggsfm/vggsfm/datasets/demo_loader.py

Line 225 in 00ddb1a

mask = Image.open(mask_path).convert("L")

The masks should be binary, with 1 indicating pixels that should be filtered out (i.e., dynamic pixels) and 0 for pixels that shouldn’t be filtered. These masks can be generated using object detection, video segmentation, or manual labeling. You can refer to the masks from DAVIS dataset as an example.

[bhack]

See also our thread at #9

[gigasurgeon]

Thank you guys.