IndexError: index 0 is out of bounds for dimension 0 with size 0

[S-Alireza-R]

Hi, I previously used this model without error, but recently, I got this error without changing the code and with exact input images. I tried to fix it by reducing max_points_num and max_tri_points_num, but it did not work. Could you help resolve this error?

Traceback (most recent call last):
File "/content/vggsfm/video_demo.py", line 66, in demo_fn
predictions = vggsfm_runner.run(
File "/content/vggsfm/vggsfm/runners/video_runner.py", line 154, in run
self.convert_pred_to_point_frame_dict(init_pred, start_idx, end_idx)
File "/content/vggsfm/vggsfm/runners/video_runner.py", line 385, in convert_pred_to_point_frame_dict
self.frame_dict[frame_idx]["extri"] = extrinsics[relative_frame_idx]
IndexError: index 0 is out of bounds for dimension 0 with size 0

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace

---

It is full output:

Poselib is available
Poselib is available
PyTorch3d is not available. Please disable visdom.
Building VGGSfM
Using cache found in /root/.cache/torch/hub/facebookresearch_dinov2_main
[2024-11-19 13:01:48,301][dinov2][INFO] - using MLP layer as FFN
VGGSfM built successfully
Data size of Sequence: 1
Run Sparse Reconstruction for Scene 200R
Predicting tracks with query_index = 15
Predict tracks in chunks to fit in memory
Predicting tracks with query_index = 0
Predict tracks in chunks to fit in memory
Predicting tracks with query_index = 31
Predict tracks in chunks to fit in memory
I20241119 13:06:04.938864 135442653024896 misc.cc:198]

==============================================================================
Global bundle adjustment

==============================================================================
I20241119 13:06:06.360823 135442653024896 timer.cc:91] Elapsed time: 0.024 [minutes]
Finished init BA
Finished init refine pose
Triangulate tracks in chunks to fit in memory
I20241119 13:06:15.501418 135442653024896 misc.cc:198]

==============================================================================
Global bundle adjustment

==============================================================================
I20241119 13:06:30.071057 135442653024896 timer.cc:91] Elapsed time: 0.243 [minutes]
Finished track triangulation and BA
Triangulate tracks in chunks to fit in memory
I20241119 13:06:41.114064 135442653024896 misc.cc:198]

==============================================================================
Global bundle adjustment

==============================================================================
I20241119 13:06:54.861339 135442653024896 timer.cc:91] Elapsed time: 0.229 [minutes]
Finished robust refine 0
Triangulate tracks in chunks to fit in memory
I20241119 13:07:05.893825 135442653024896 misc.cc:198]

==============================================================================
Global bundle adjustment

==============================================================================
I20241119 13:07:20.302343 135442653024896 timer.cc:91] Elapsed time: 0.240 [minutes]
Finished robust refine 1
Running iterative BA by 1 times
Triangulate tracks in chunks to fit in memory
I20241119 13:07:27.080671 135442653024896 misc.cc:198]

==============================================================================
Global bundle adjustment

==============================================================================
I20241119 13:07:55.638815 135442653024896 bundle_adjustment.cc:866]
Bundle adjustment report:
Residuals : 382890
Parameters : 18601
Iterations : 101
Time : 28.3568 [s]
Initial cost : 0.584149 [px]
Final cost : 0.523949 [px]
Termination : No convergence

I20241119 13:07:55.638854 135442653024896 timer.cc:91] Elapsed time: 0.476 [minutes]
Finished iterative BA 0
Error executing job with overrides: ['SCENE_DIR=/content/drive/MyDrive/Data/200R']
Traceback (most recent call last):
File "/content/vggsfm/video_demo.py", line 66, in demo_fn
predictions = vggsfm_runner.run(
File "/content/vggsfm/vggsfm/runners/video_runner.py", line 154, in run
self.convert_pred_to_point_frame_dict(init_pred, start_idx, end_idx)
File "/content/vggsfm/vggsfm/runners/video_runner.py", line 385, in convert_pred_to_point_frame_dict
self.frame_dict[frame_idx]["extri"] = extrinsics[relative_frame_idx]
IndexError: index 0 is out of bounds for dimension 0 with size 0

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace

[jytime]

Hi @S-Alireza-R ,

It appears that the issue arises because the model struggles to find a suitable reconstruction for the first window, which hence causes a problem in initialization. This could stem from the initial frames of the input sequence being challenging. I noticed you’re using a 31-frame window. Could you test the reconstruction of just the first 31 frames (0-31) by feeding them into runner instead of video_runner?

If this initial subset does not yield satisfactory results, consider adjusting the init_window_size parameter to either increase or decrease the size of the first window. This might help the model find a better starting point.