Question about multiple identical objects within a single image

[qidihan]

Thanks for you wonderful project!!. I have question about where an image contains 4 identical objects but with different orientations, I'm unsure about the best approach for mask input during training. Would it be necessary to input 4 separate pairs of masks for this object, or should I generate a single image output that includes all the masks for these objects, each with a distinct mask number? I want to make the model to output all the masks with different mask id. The example image is below. Thanks so much!!!

[xiongxyowo]

Hi, based on your description, I think your needs fall under the category of "instance segmentation". Unfortunately, SAM2-UNet framework is based on "semantic segmentation" currently and the output does not contain the mask id. There is a temporary solution that can meet your task requirements:

1. Annotate the image, where one single mask contains all the objects you want to identify (e.g. all four objects in the image).
2. For the segmentation prediction, use OpenCV to calculate the connected domain of mask, and each connected domain is assigned an id as a different object.

A more correct solution is to wrap SAM2-UNet with a unified segmentation framework to allow instance segmentation, such as K-Net [1].

[1] K-net: Towards unified image segmentation

---

同学你好, 根据你的描述，我认为你的需求属于"实例分割"的范畴。目前我们的SAM2-UNet的框架基于"语义分割"，输出结果中不包含mask id。有一个临时的解决方案可以满足你的任务要求：

1. 对图像进行标注，其中，每张图像所对应的单张mask中包含你所需要的识别的所有对象(例如图中的全部四个物体)。
2. 对于网络预测的分割结果，使用OpenCV计算mask的连通域，每个连通域视为一个不同的对象赋予一个id。

更严谨的解决方案是使用统一分割框架包装SAM2-UNet从而实现实例分割，例如K-Net。