Saving predictions correctly

[lcoandrade]

I was able to make my first complete semantic segmentation pipeline using RV as a module. I made the code on this Kaggle notebook with this dataset.

The predictions are ok for the validation dataset (considering the number os epochs used), but when I use my learner to generate de predictions for my prediction dataset:

from rastervision.core.data import SemanticSegmentationLabels

predictions = learner.predict_dataset(
    dataset=prediction_dataset,
    raw_out=True,
    numpy_out=True,
    predict_kw=dict(out_shape=IMG_SIZE), # using 256x256
    progress_bar=True,
)

pred_labels = SemanticSegmentationLabels.from_predictions(
    windows=prediction_dataset.windows,
    predictions=predictions,
    smooth=True, # trying both outputs
    extent=prediction_dataset.scene.extent,
    num_classes=len(class_config),
)

The results are not good. What am I doing wrong?

Here are the predictions for my validation dataset:

And here are the geotif file saved with:

pred_labels.save(
    uri=os.path.join(OUTPUT_DIR, 'predictions'),
    crs_transformer=prediction_dataset.scene.raster_source.crs_transformer,
    class_config=class_config,
)

On the left we have the labels for the window displayed and on the right we have the same window on the geotiff saved.

What can be the problem here?

[AdeelH]

I'm not sure what I'm looking at here. As a sanity check, you can try predicting on the an image from the training or validation set. You can also inspect the predictions using pred_labels.get_label_arr() or pred_labels.get_score_arr(), to see if they look okay before saving them.

[lcoandrade]

I've just tested with pred_labels.get_label_arr() and pred_labels.get_score_arr(). As I have 13 classes, I'm more interested in the pred_labels.get_label_arr .
I've tested with my validation dataset and the outputs are quite different from learner.plot_predictions(split='valid', show=True). Not sure why.

I've tried with raw_out=False and smooth=False to see what would happen with the outputs.

Here is the window with the image file:

Here is the labels used to train:

And here is the geo tiff file saved over the image:

These are some predicted values with just one epoch:

Maybe I'm missing something silly here, but I haven't identified the source of the problem yet....

Could you take a look at my notebook to see what should be changed?

Thanks in advance!

[AdeelH]

It seems like you're re-instantiating the Learner just before you make predictions. That means you're essentially predicting with a randomly initialized model.

• You do not need to re-instantiate the Learner before making predictions if you already have it in memory.
• If you do need to load a Learner from a model-bundle, use Learner.from_model_bundle(). See this or this.

[lcoandrade]

You are completely right!!!!

When I instantiate the learner like this:

learner = SemanticSegmentationLearner(
    cfg=learner_cfg,
    output_dir=OUTPUT_DIR,
    #model=model, # Not passing a model because I'm have a model config in learner_cfg
    train_ds=concat_dataset,
    valid_ds=validation_dataset,
)

When I pass the output_dir with a model bundle saved, it loads correctly and resumes from the epoch it has stopped earlier. So, that's why I tried to load the learner in prediction mode (training=False) in the same way.

I should have seem that! Thanks and sorry to bother!

[AdeelH]

When I pass the output_dir with a model bundle saved, it loads correctly and resumes from the epoch it has stopped earlier. So, that's why I tried to load the learner in prediction mode (training=False) in the same way.

That's understandable. It only does that if training=True. And even in that case, it is not loading the state from the model bundle but rather the model file saved in the output dir.

I should have seem that! Thanks and sorry to bother!

No problem! Feel free to ask more questions!

[lcoandrade]

That's understandable. It only does that if training=True. And even in that case, it is not loading the state from the model bundle but rather the model file saved in the output dir.

Thanks! Now, I realize that.