MedSAM2

Segment Anything in 3D Medical Images and Videos

[Paper] [Project] [Huggingface] [Dataset List] [3D Slicer] [Gradio App] [Colab] [BibTeX]

Installation

• Create a virtual environment: conda create -n medsam2 python=3.12 -y and conda activate medsam2
• Install PyTorch: pip3 install torch torchvision (Linux CUDA 12.4)
• Download code git clone https://github.com/bowang-lab/MedSAM2.git && cd MedSAM2 and run pip install -e ".[dev]"
• Download checkpoints: sh download.sh
• Optional: Please install the following dependencies for gradio

sudo apt-get update
sudo apt-get install ffmpeg
pip install gradio==3.38.0
pip install numpy==1.26.3 
pip install ffmpeg-python 
pip install moviepy

Download annotated datasets

• CT_DeepLesion-MedSAM2

• LLD-MMRI-MedSAM2

Note: Please also cite the raw DeepLesion and LLD-MMRI dataset paper when using these datasets.

Inference

3D medical image segmentation

• CMD

python medsam2_infer_3D_CT.py -i CT_DeepLesion/images -o CT_DeepLesion/segmentation

• CoLab: MedSAM2_inference_CT_Lesion_Demo.ipynb

Medical video segmentation

• CMD

python medsam2_infer_video.py -i input_video_path -m input_mask_path -o output_video_path 

• CoLab: MedSAM2_Inference_Video_Demo.ipynb

Gradio demo

python app.py

Training

Specify dataset path in sam2/configs/sam2.1_hiera_tiny_finetune512.yaml

sbatch multi_node_train.sh

Acknowledgements

• We highly appreciate all the challenge organizers and dataset owners for providing the public datasets to the community.
• We thank Meta AI for making the source code of SAM2 publicly available. Please also cite this paper when using MedSAM2.

Bibtex