[Issue]: crash using Stablity Matrix on Fedora 41 with AMD GPU #3731

panton41 · 2025-01-30T22:52:01Z

Issue Description

I had SD.Next running on Bazzite Linux (which is based on Fedora 41) until about three weeks ago when this current error began. I changed distros to Linux Mint and had no problem with SD.Next, but there were issues in other programs, so I bounced around until I was on Fedora 41 itself and the previous problem continues. I am unsure whether his has to do with a Fedora update or an update to SD.Next, but I've been unable to get previous versions installed as well with the same error.

During the installation, an exception is thrown, and the process stops.

Stability Matrix is an App Image that creates an isolated environment to install various Stable Diffusion programs without worrying too much about system libraries.

Version Platform Description

Fedora Linux 41 x86_64
AMD Ryzen 5600X
64GB DDR4
AMD Radeon 7600 GT

No logs are created that contain the application version because it fails to install.

Relevant log output

Unpacking resources
Unpacking resources
Downloading package...
/home/harasurya/AppImages/Stable-Diffusion/Data/Packages/automatic/venv/lib/python3.10/site-packages/_distutils_hack/__init__.py:53: UserWarning: Reliance on distutils from stdlib is deprecated. Users must rely on setuptools to provide the distutils module. Avoid importing distutils or import setuptools first, and avoid setting SETUPTOOLS_USE_DISTUTILS=stdlib. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
  warnings.warn(
17:34:21-631477 INFO     Starting SD.Next                                       
17:34:21-633871 INFO     Logger:                                                
                         file="/home/harasurya/AppImages/Stable-Diffusion/Data/P
                         ackages/automatic/sdnext.log" level=DEBUG size=64      
                         mode=create                                            
17:34:21-634866 INFO     Python: version=3.10.11 platform=Linux                 
                         bin="/home/harasurya/AppImages/Stable-Diffusion/Data/Pa
                         ckages/automatic/venv/bin/python3"                     
                         venv="/home/harasurya/AppImages/Stable-Diffusion/Data/P
                         ackages/automatic/venv"                                
17:34:21-647326 INFO     Version: app=sd.next updated=2025-01-29 hash=46464c4c  
                         branch=master                                          
                         url=https://github.com/vladmandic/automatic/tree/master
                         ui=main                                                
17:34:21-902721 INFO     Platform: arch=x86_64 cpu= system=Linux                
                         release=6.12.10-200.fc41.x86_64 python=3.10.11         
                         docker=False                                           
17:34:21-904451 DEBUG    Packages: venv=venv                                    
                         site=['venv/lib/python3.10/site-packages']             
17:34:21-905501 INFO     Args: ['--use-zluda', '--debug', '--test']             
17:34:21-906413 DEBUG    Setting environment tuning                             
17:34:21-907385 DEBUG    Torch allocator:                                       
                         "garbage_collection_threshold:0.80,max_split_size_mb:51
                         2"                                                     
17:34:22-036702 DEBUG    Torch overrides: cuda=False rocm=False ipex=False      
                         directml=False openvino=False zluda=True               
17:34:22-038510 INFO     Python: version=3.10.11 platform=Linux                 
                         bin="/home/harasurya/AppImages/Stable-Diffusion/Data/Pa
                         ckages/automatic/venv/bin/python3"                     
                         venv="/home/harasurya/AppImages/Stable-Diffusion/Data/P
                         ackages/automatic/venv"                                
17:34:22-040073 INFO     ROCm: AMD toolkit detected                             
17:34:22-064771 INFO     ROCm: agents=['gfx1102']                               
17:34:22-065675 INFO     ROCm: version=6.2, using agent gfx1102                 
17:34:22-066485 DEBUG    Install: package="onnxruntime-training" install        
                         required                                               
17:34:49-947027 DEBUG    Install: package="torch" install required              
17:34:49-948371 INFO     Torch: download and install in progress...             
                         cmd="torch==2.4.1+rocm6.1 torchvision==0.19.1+rocm6.1  
                         --index-url https://download.pytorch.org/whl/rocm6.1"  
17:37:08-311994 DEBUG    Install: package="onnxruntime" install required        
17:37:10-374875 INFO     Diffusers install:                                     
                         commit=7b100ce589b917d4c116c9e61a6ec46d4f2ab062        
17:37:19-574236 INFO     Install requirements: this may take a while...         
17:38:08-911568 DEBUG    Install: package="torch" install required              
17:38:08-912451 INFO     Install: verifying requirements                        
17:38:08-914168 DEBUG    Install: package="psutil" install required             
17:38:08-933648 DEBUG    Install: package="rich" install required               
17:38:08-952566 DEBUG    Install: package="requests" install required           
17:38:12-699276 INFO     Startup: standard                                      
17:38:12-700099 INFO     Verifying submodules                                   
17:38:19-247781 DEBUG    Git detached head detected:                            
                         folder="extensions-builtin/sd-extension-chainner"      
                         reattach=main                                          
17:38:19-248911 DEBUG    Git submodule: extensions-builtin/sd-extension-chainner
                         / main                                                 
17:38:19-263832 DEBUG    Git detached head detected:                            
                         folder="extensions-builtin/sd-extension-system-info"   
                         reattach=main                                          
17:38:19-264790 DEBUG    Git submodule:                                         
                         extensions-builtin/sd-extension-system-info / main     
17:38:19-276643 DEBUG    Git detached head detected:                            
                         folder="extensions-builtin/sd-webui-agent-scheduler"   
                         reattach=main                                          
17:38:19-277551 DEBUG    Git submodule:                                         
                         extensions-builtin/sd-webui-agent-scheduler / main     
17:38:19-295833 DEBUG    Git detached head detected:                            
                         folder="extensions-builtin/sdnext-modernui"            
                         reattach=main                                          
17:38:19-296800 DEBUG    Git submodule: extensions-builtin/sdnext-modernui /    
                         main                                                   
17:38:19-318016 DEBUG    Git detached head detected:                            
                         folder="extensions-builtin/stable-diffusion-webui-rembg
                         " reattach=master                                      
17:38:19-319135 DEBUG    Git submodule:                                         
                         extensions-builtin/stable-diffusion-webui-rembg /      
                         master                                                 
17:38:19-331743 DEBUG    Git detached head detected:                            
                         folder="modules/k-diffusion" reattach=master           
17:38:19-332722 DEBUG    Git submodule: modules/k-diffusion / master            
17:38:19-344228 DEBUG    Git detached head detected: folder="wiki"              
                         reattach=master                                        
17:38:19-345571 DEBUG    Git submodule: wiki / master                           
17:38:19-353227 DEBUG    Register paths                                         
17:38:19-372312 DEBUG    Installed packages: 159                                
17:38:19-373075 DEBUG    Extensions all: ['Lora', 'sd-extension-chainner',      
                         'sd-extension-system-info', 'sd-webui-agent-scheduler',
                         'sdnext-modernui', 'stable-diffusion-webui-rembg']     
17:38:19-424322 DEBUG    Extension installer:                                   
                         /home/harasurya/AppImages/Stable-Diffusion/Data/Package
                         s/automatic/extensions-builtin/sd-webui-agent-scheduler
                         /install.py                                            
17:38:21-946561 INFO     Extension installed packages: sd-webui-agent-scheduler 
                         ['SQLAlchemy==2.0.37', 'greenlet==3.1.1']              
17:38:21-965942 DEBUG    Extension installer:                                   
                         /home/harasurya/AppImages/Stable-Diffusion/Data/Package
                         s/automatic/extensions-builtin/stable-diffusion-webui-r
                         embg/install.py                                        
17:38:27-292529 INFO     Extension installed packages:                          
                         stable-diffusion-webui-rembg ['PyMatting==1.1.13',     
                         'opencv-python-headless==4.11.0.86', 'pooch==1.8.2',   
                         'rembg==2.0.62']                                       
17:38:27-293639 DEBUG    Extensions all: []                                     
17:38:27-294378 INFO     Extensions enabled: ['Lora', 'sd-extension-chainner',  
                         'sd-extension-system-info', 'sd-webui-agent-scheduler',
                         'sdnext-modernui', 'stable-diffusion-webui-rembg']     
17:38:27-295204 INFO     Install requirements: this may take a while...         
17:38:27-295798 INFO     Install: verifying requirements                        
17:38:27-296449 DEBUG    Setup complete without errors: 1738276707              
17:38:27-298651 DEBUG    Extension preload: {'extensions-builtin': 0.0,         
                         'extensions': 0.0}                                     
17:38:27-299685 INFO     Command line args: ['--use-zluda', '--debug', '--test']
                         use_zluda=True test=True debug=True args=[]            
17:38:27-300754 DEBUG    Env flags: []                                          
17:38:27-301572 DEBUG    Starting module: <module 'webui' from                  
                         '/home/harasurya/AppImages/Stable-Diffusion/Data/Packag
                         es/automatic/webui.py'>                                
amdgpu.ids: No such file or directory
17:38:30-477426 DEBUG    Reading failed: config.json [Errno 2] No such file or  
                         directory: 'config.json'                               
17:38:30-981965 INFO     Device detect: memory=16.0 default=balanced            
17:38:30-984551 DEBUG    Settings: fn="config.json" created                     
17:38:30-986301 INFO     Engine: backend=Backend.DIFFUSERS compute=rocm         
                         device=cuda attention="Scaled-Dot-Product" mode=no_grad
17:38:30-987478 DEBUG    Save: file="config.json" json=19 bytes=732 time=0.002  
17:38:30-988489 DEBUG    Migrated styles: file="styles.csv"                     
                         folder="models/styles"                                 
17:38:30-993767 INFO     Available Styles: folder="models/styles" items=288     
                         time=0.00                                              
17:38:30-994804 DEBUG    Read: file="html/reference.json" json=63 bytes=32978   
                         time=0.000 fn=_call_with_frames_removed:<module>       
Error: StabilityMatrix.Core.Exceptions.ProcessException: install script failed with code 139: '/home/harasurya/AppImages/Stable-Diffusion/Data/Packages/automatic/venv/lib/python3.10/site-packages/_distutils_hack/__init__.py:53: UserWarning: Reliance on distutils from stdlib is deprecated. Users must rely on setuptools to provide the distutils module. Avoid importing distutils or import setuptools first, and avoid setting SETUPTOOLS_USE_DISTUTILS=stdlib. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml\n  warnings.warn(\n17:34:21-631477 INFO     Starting SD.Next                                       \n17:34:21-633871 INFO     Logger:                                                \n                         file="/home/harasurya/AppImages/Stable-Diffusion/Data/P\n                         ackages/automatic/sdnext.log" level=DEBUG size=64      \n                         mode=create                                            \n17:34:21-634866 INFO     Python: version=3.10.11 platform=Linux                 \n                         bin="/home/harasurya/AppImages/Stable-Diffusion/Data/Pa\n                         ckages/automatic/venv/bin/python3"                     \n                         venv="/home/harasurya/AppImages/Stable-Diffusion/Data/P\n                         ackages/automatic/venv"                                \n17:34:21-647326 INFO     Version: app=sd.next updated=2025-01-29 hash=46464c4c  \n                         branch=master                                          \n                         url=https://github.com/vladmandic/automatic/tree/master\n                         ui=main                                                \n17:34:21-902721 INFO     Platform: arch=x86_64 cpu= system=Linux                \n                         release=6.12.10-200.fc41.x86_64 python=3.10.11         \n                         docker=False                                           \n17:34:21-904451 DEBUG    Packages: venv=venv                                    \n                         site=['venv/lib/python3.10/site-packages']             \n17:34:21-905501 INFO     Args: ['--use-zluda', '--debug', '--test']             \n17:34:21-906413 DEBUG    Setting environment tuning                             \n17:34:21-907385 DEBUG    Torch allocator:                                       \n                         "garbage_collection_threshold:0.80,max_split_size_mb:51\n                         2"                                                     \n17:34:22-036702 DEBUG    Torch overrides: cuda=False rocm=False ipex=False      \n                         directml=False openvino=False zluda=True               \n17:34:22-038510 INFO     Python: version=3.10.11 platform=Linux                 \n                         bin="/home/harasurya/AppImages/Stable-Diffusion/Data/Pa\n                         ckages/automatic/venv/bin/python3"                     \n                         venv="/home/harasurya/AppImages/Stable-Diffusion/Data/P\n                         ackages/automatic/venv"                                \n17:34:22-040073 INFO     ROCm: AMD toolkit detected                             \n17:34:22-064771 INFO     ROCm: agents=['gfx1102']                               \n17:34:22-065675 INFO     ROCm: version=6.2, using agent gfx1102                 \n17:34:22-066485 DEBUG    Install: package="onnxruntime-training" install        \n                         required                                               \n17:34:49-947027 DEBUG    Install: package="torch" install required              \n17:34:49-948371 INFO     Torch: download and install in progress...             \n                         cmd="torch==2.4.1+rocm6.1 torchvision==0.19.1+rocm6.1  \n                         --index-url https://download.pytorch.org/whl/rocm6.1"  \n17:37:08-311994 DEBUG    Install: package="onnxruntime" install required        \n17:37:10-374875 INFO     Diffusers install:                                     \n                         commit=7b100ce589b917d4c116c9e61a6ec46d4f2ab062        \n17:37:19-574236 INFO     Install requirements: this may take a while...         \n17:38:08-911568 DEBUG    Install: package="torch" install required              \n17:38:08-912451 INFO     Install: verifying requirements                        \n17:38:08-914168 DEBUG    Install: package="psutil" install required             \n17:38:08-933648 DEBUG    Install: package="rich" install required               \n17:38:08-952566 DEBUG    Install: package="requests" install required           \n17:38:12-699276 INFO     Startup: standard                                      \n17:38:12-700099 INFO     Verifying submodules                                   \n17:38:19-247781 DEBUG    Git detached head detected:                            \n                         folder="extensions-builtin/sd-extension-chainner"      \n                         reattach=main                                          \n17:38:19-248911 DEBUG    Git submodule: extensions-builtin/sd-extension-chainner\n                         / main                                                 \n17:38:19-263832 DEBUG    Git detached head detected:                            \n                         folder="extensions-builtin/sd-extension-system-info"   \n                         reattach=main                                          \n17:38:19-264790 DEBUG    Git submodule:                                         \n                         extensions-builtin/sd-extension-system-info / main     \n17:38:19-276643 DEBUG    Git detached head detected:                            \n                         folder="extensions-builtin/sd-webui-agent-scheduler"   \n                         reattach=main                                          \n17:38:19-277551 DEBUG    Git submodule:                                         \n                         extensions-builtin/sd-webui-agent-scheduler / main     \n17:38:19-295833 DEBUG    Git detached head detected:                            \n                         folder="extensions-builtin/sdnext-modernui"            \n                         reattach=main                                          \n17:38:19-296800 DEBUG    Git submodule: extensions-builtin/sdnext-modernui /    \n                         main                                                   \n17:38:19-318016 DEBUG    Git detached head detected:                            \n                         folder="extensions-builtin/stable-diffusion-webui-rembg\n                         " reattach=master                                      \n17:38:19-319135 DEBUG    Git submodule:                                         \n                         extensions-builtin/stable-diffusion-webui-rembg /      \n                         master                                                 \n17:38:19-331743 DEBUG    Git detached head detected:                            \n                         folder="modules/k-diffusion" reattach=master           \n17:38:19-332722 DEBUG    Git submodule: modules/k-diffusion / master            \n17:38:19-344228 DEBUG    Git detached head detected: folder="wiki"              \n                         reattach=master                                        \n17:38:19-345571 DEBUG    Git submodule: wiki / master                           \n17:38:19-353227 DEBUG    Register paths                                         \n17:38:19-372312 DEBUG    Installed packages: 159                                \n17:38:19-373075 DEBUG    Extensions all: ['Lora', 'sd-extension-chainner',      \n                         'sd-extension-system-info', 'sd-webui-agent-scheduler',\n                         'sdnext-modernui', 'stable-diffusion-webui-rembg']     \n17:38:19-424322 DEBUG    Extension installer:                                   \n                         /home/harasurya/AppImages/Stable-Diffusion/Data/Package\n                         s/automatic/extensions-builtin/sd-webui-agent-scheduler\n                         /install.py                                            \n17:38:21-946561 INFO     Extension installed packages: sd-webui-agent-scheduler \n                         ['SQLAlchemy==2.0.37', 'greenlet==3.1.1']              \n17:38:21-965942 DEBUG    Extension installer:                                   \n                         /home/harasurya/AppImages/Stable-Diffusion/Data/Package\n                         s/automatic/extensions-builtin/stable-diffusion-webui-r\n                         embg/install.py                                        \n17:38:27-292529 INFO     Extension installed packages:                          \n                         stable-diffusion-webui-rembg ['PyMatting==1.1.13',     \n                         'opencv-python-headless==4.11.0.86', 'pooch==1.8.2',   \n                         'rembg==2.0.62']                                       \n17:38:27-293639 DEBUG    Extensions all: []                                     \n17:38:27-294378 INFO     Extensions enabled: ['Lora', 'sd-extension-chainner',  \n                         'sd-extension-system-info', 'sd-webui-agent-scheduler',\n                         'sdnext-modernui', 'stable-diffusion-webui-rembg']     \n17:38:27-295204 INFO     Install requirements: this may take a while...         \n17:38:27-295798 INFO     Install: verifying requirements                        \n17:38:27-296449 DEBUG    Setup complete without errors: 1738276707              \n17:38:27-298651 DEBUG    Extension preload: {'extensions-builtin': 0.0,         \n                         'extensions': 0.0}                                     \n17:38:27-299685 INFO     Command line args: ['--use-zluda', '--debug', '--test']\n                         use_zluda=True test=True debug=True args=[]            \n17:38:27-300754 DEBUG    Env flags: []                                          \n17:38:27-301572 DEBUG    Starting module: <module 'webui' from                  \n                         '/home/harasurya/AppImages/Stable-Diffusion/Data/Packag\n                         es/automatic/webui.py'>                                \namdgpu.ids: No such file or directory\n17:38:30-477426 DEBUG    Reading failed: config.json [Errno 2] No such file or  \n                         directory: 'config.json'                               \n17:38:30-981965 INFO     Device detect: memory=16.0 default=balanced            \n17:38:30-984551 DEBUG    Settings: fn="config.json" created                     \n17:38:30-986301 INFO     Engine: backend=Backend.DIFFUSERS compute=rocm         \n                         device=cuda attention="Scaled-Dot-Product" mode=no_grad\n17:38:30-987478 DEBUG    Save: file="config.json" json=19 bytes=732 time=0.002  \n17:38:30-988489 DEBUG    Migrated styles: file="styles.csv"                     \n                         folder="models/styles"                                 \n17:38:30-993767 INFO     Available Styles: folder="models/styles" items=288     \n                         time=0.00                                              \n17:38:30-994804 DEBUG    Read: file="html/reference.json" json=63 bytes=32978   \n                         time=0.000 fn=_call_with_frames_removed:<module>       \n'
   at StabilityMatrix.Core.Python.PyVenvRunner.CustomInstall(ProcessArgs args, Action`1 outputDataReceived)
   at StabilityMatrix.Core.Models.Packages.VladAutomatic.InstallPackage(String installLocation, InstalledPackage installedPackage, InstallPackageOptions options, IProgress`1 progress, Action`1 onConsoleOutput, CancellationToken cancellationToken)
   at StabilityMatrix.Core.Models.Packages.VladAutomatic.InstallPackage(String installLocation, InstalledPackage installedPackage, InstallPackageOptions options, IProgress`1 progress, Action`1 onConsoleOutput, CancellationToken cancellationToken)
   at StabilityMatrix.Core.Models.PackageModification.InstallPackageStep.ExecuteAsync(IProgress`1 progress, CancellationToken cancellationToken)
   at StabilityMatrix.Core.Models.PackageModification.PackageModificationRunner.ExecuteSteps(IEnumerable`1 steps)

Backend

Diffusers

UI

Standard

Branch

Master

Model

Other

Acknowledgements

I have read the above and searched for existing issues
I confirm that this is classified correctly and its not an extension issue

Disty0 · 2025-01-31T01:46:38Z

gfx1102 / RX 7600 series are unsupported by ROCm, you have to downgrade to ROCm 5.7.

Also why are you using --use-zluda on Linux? That is mainly for Windows because native PyTorch with ROCm isn't available on Windows.

panton41 · 2025-01-31T01:59:41Z

Zulda is turned on because Stability Matrix turns it on by default. Switching to ROCm has identical results with not installing with the exact same error message.

This exact hardware has worked before on this operating system in December - and works on other Linux distros, even with ROCm 6.2 - and doesn't anymore. Something changed with an update to either Fedora or SD.Next around the New Year, and it broke. if it's Fedora I can figure it out on my end, but I need to know what the problem is in order to fix it and Error 139, from what I recall, is "segfault" which is a catchall for "Whelp, it just crashed" and from what I can see the log doesn't actually say anything useful.

The thing is, I've read the exact opposite and that you MUST have a 6.x branch of ROCm to get the hardware to work right. I honestly don't think anyone actually know what they're talking about, everyone has contradictory messages, and I include the engineers at AMD developing ROCm.

vladmandic · 2025-01-31T14:32:26Z

first thing that happens after last line in the log is setting some torch parameters.
i've just added a bit extra logging in that part of the code in dev branch, so can you try to install/use that and report updated log?
also, as soon as crash happens, run dmesg -ku so we can see the kernel log - it should show where actual crash happend.

vladmandic · 2025-02-06T14:49:15Z

@panton41 any updates?

panton41 · 2025-02-11T01:35:19Z

I am working on trying things right now. The last week has been busy at work and I haven't had a chance to do anything AI related at home.

panton41 · 2025-02-11T02:58:56Z

So, there's two things going on.

One, the install process in Stability Matrix crashes with the previous error. But, on reopening Stability Matrix it shows as installed and returns an error that it was throwing back in early January when the whole problem with it not working began.

The relevant part of the kernel dump:


[ 2485.640391] process 'Stable-Diffusion/Data/Assets/7zzs' started with executable stack
[ 2525.274690] r8169 0000:06:00.0 enp6s0: Link is Down
[ 2529.167093] r8169 0000:06:00.0 enp6s0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 2575.715062] r8169 0000:06:00.0 enp6s0: Link is Down
[ 2579.344445] r8169 0000:06:00.0 enp6s0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 3521.235737] show_signal_msg: 111 callbacks suppressed
[ 3521.235743] python3[20478]: segfault at 18 ip 00007f12928af9bd sp 00007ffeecde6e60 error 4 in libamdhip64.so[2af9bd,7f1292600000+439000] likely on CPU 10 (core 4, socket 0)
[ 3521.235754] Code: 00 31 f6 48 83 c5 08 e8 51 04 00 00 49 39 ec 75 ec 48 8b 83 48 01 00 00 48 8b a8 a0 01 00 00 48 8b 05 87 05 39 00 64 48 8b 10 <48> 8b 45 18 4c 8d 6d 18 a8 01 0f 85 8b 03 00 00 48 89 c1 48 83 c9
[ 4928.354137] python3[25586]: segfault at 18 ip 00007f180b2af9bd sp 00007ffeb4e72a10 error 4 in libamdhip64.so[2af9bd,7f180b000000+439000] likely on CPU 7 (core 1, socket 0)
[ 4928.354147] Code: 00 31 f6 48 83 c5 08 e8 51 04 00 00 49 39 ec 75 ec 48 8b 83 48 01 00 00 48 8b a8 a0 01 00 00 48 8b 05 87 05 39 00 64 48 8b 10 <48> 8b 45 18 4c 8d 6d 18 a8 01 0f 85 8b 03 00 00 48 89 c1 48 83 c9

The directory Stable-Diffusion/Data/Assets/ is related to the custom Python libraries Stability Matrix installs.

The new log:

Unpacking resources Unpacking resources Download complete! Downloading package... /home/harasurya/AppImages/Stable-Diffusion/Data/Packages/SD.Next/venv/lib/python3.10/site-packages/_distutils_hack/__init__.py:53: UserWarning: Reliance on distutils from stdlib is deprecated. Users must rely on setuptools to provide the distutils module. Avoid importing distutils or import setuptools first, and avoid setting SETUPTOOLS_USE_DISTUTILS=stdlib. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml warnings.warn( 21:49:54-191464 INFO Starting SD.Next 21:49:54-194535 INFO Logger: file="/home/harasurya/AppImages/Stable-Diffusion/Data/P ackages/SD.Next/sdnext.log" level=DEBUG size=64 mode=create 21:49:54-195583 INFO Python: version=3.10.11 platform=Linux bin="/home/harasurya/AppImages/Stable-Diffusion/Data/Pa ckages/SD.Next/venv/bin/python3" venv="/home/harasurya/AppImages/Stable-Diffusion/Data/P ackages/SD.Next/venv" 21:49:54-208617 INFO Version: app=sd.next updated=2025-02-05 hash=9f122234 branch=master url=https://github.com/vladmandic/automatic/tree/master ui=main 21:49:54-450353 INFO Platform: arch=x86_64 cpu= system=Linux release=6.12.11-200.fc41.x86_64 python=3.10.11 docker=False 21:49:54-451885 DEBUG Packages: venv=venv site=['venv/lib/python3.10/site-packages'] 21:49:54-452895 INFO Args: ['--use-rocm', '--debug', '--test'] 21:49:54-453725 DEBUG Setting environment tuning 21:49:54-454511 DEBUG Torch allocator: "garbage_collection_threshold:0.80,max_split_size_mb:51 2" 21:49:54-489253 DEBUG Torch overrides: cuda=False rocm=True ipex=False directml=False openvino=False zluda=False 21:49:54-490863 INFO ROCm: AMD toolkit detected 21:49:54-514704 INFO ROCm: agents=['gfx1102'] 21:49:54-515935 INFO ROCm: version=6.2, using agent gfx1102 21:49:54-517091 INFO Python: version=3.10.11 platform=Linux bin="/home/harasurya/AppImages/Stable-Diffusion/Data/Pa ckages/SD.Next/venv/bin/python3" venv="/home/harasurya/AppImages/Stable-Diffusion/Data/P ackages/SD.Next/venv" 21:49:54-518298 DEBUG Install: package="torch" install required 21:49:54-519150 INFO Torch: download and install in progress... cmd="torch==2.6.0+rocm6.2.4 torchvision==0.21.0+rocm6.2.4 --index-url https://download.pytorch.org/whl/rocm6.2.4" 21:51:43-770259 DEBUG Install: package="onnx" install required 21:51:46-829661 DEBUG Install: package="onnxruntime" install required 21:51:48-323628 INFO Diffusers install: commit=f63d32233f402bd603da8f3aa385aecb9c3d8809 21:51:56-145925 INFO Install requirements: this may take a while... 21:52:30-856712 DEBUG Install: package="torch" install required 21:52:30-857683 INFO Install: verifying requirements 21:52:30-860234 DEBUG Install: package="psutil" install required 21:52:30-881354 DEBUG Install: package="rich" install required 21:52:31-763578 DEBUG Install: package="requests" install required 21:52:35-509561 INFO Startup: standard 21:52:35-510373 INFO Verifying submodules 21:52:46-992252 DEBUG Git detached head detected: folder="extensions-builtin/sd-extension-chainner" reattach=main 21:52:46-993691 DEBUG Git submodule: extensions-builtin/sd-extension-chainner / main 21:52:47-010927 DEBUG Git detached head detected: folder="extensions-builtin/sd-extension-system-info" reattach=main 21:52:47-012128 DEBUG Git submodule: extensions-builtin/sd-extension-system-info / main 21:52:47-024521 DEBUG Git detached head detected: folder="extensions-builtin/sd-webui-agent-scheduler" reattach=main 21:52:47-025612 DEBUG Git submodule: extensions-builtin/sd-webui-agent-scheduler / main 21:52:47-043546 DEBUG Git detached head detected: folder="extensions-builtin/sdnext-modernui" reattach=main 21:52:47-044824 DEBUG Git submodule: extensions-builtin/sdnext-modernui / main 21:52:47-068414 DEBUG Git detached head detected: folder="extensions-builtin/stable-diffusion-webui-rembg " reattach=master 21:52:47-069506 DEBUG Git submodule: extensions-builtin/stable-diffusion-webui-rembg / master 21:52:47-083180 DEBUG Git detached head detected: folder="modules/k-diffusion" reattach=master 21:52:47-084420 DEBUG Git submodule: modules/k-diffusion / master 21:52:47-094734 DEBUG Git detached head detected: folder="wiki" reattach=master 21:52:47-095724 DEBUG Git submodule: wiki / master 21:52:47-105580 DEBUG Register paths 21:52:47-128207 DEBUG Installed packages: 156 21:52:47-129184 DEBUG Extensions all: ['Lora', 'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sdnext-modernui', 'stable-diffusion-webui-rembg'] 21:52:47-186457 DEBUG Extension installer: /home/harasurya/AppImages/Stable-Diffusion/Data/Package s/SD.Next/extensions-builtin/sd-webui-agent-scheduler/i nstall.py 21:52:49-185436 INFO Extension installed packages: sd-webui-agent-scheduler ['SQLAlchemy==2.0.38', 'greenlet==3.1.1'] 21:52:49-209580 DEBUG Extension installer: /home/harasurya/AppImages/Stable-Diffusion/Data/Package s/SD.Next/extensions-builtin/stable-diffusion-webui-rem bg/install.py 21:52:53-526940 INFO Extension installed packages: stable-diffusion-webui-rembg ['PyMatting==1.1.13', 'opencv-python-headless==4.11.0.86', 'pooch==1.8.2', 'rembg==2.0.62'] 21:52:53-528011 DEBUG Extensions all: [] 21:52:53-528593 INFO Extensions enabled: ['Lora', 'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sdnext-modernui', 'stable-diffusion-webui-rembg'] 21:52:53-529438 INFO Install requirements: this may take a while... 21:52:53-530098 INFO Install: verifying requirements 21:52:53-530697 DEBUG Setup complete without errors: 1739242374 21:52:53-532810 DEBUG Extension preload: {'extensions-builtin': 0.0, 'extensions': 0.0} 21:52:53-533873 INFO Command line args: ['--use-rocm', '--debug', '--test'] use_rocm=True test=True debug=True args=[] 21:52:53-534723 DEBUG Env flags: [] 21:52:53-535549 DEBUG Starting module: <module 'webui' from '/home/harasurya/AppImages/Stable-Diffusion/Data/Packag es/SD.Next/webui.py'> 21:52:57-572436 DEBUG System: cores=12 affinity=12 threads=6 21:52:57-575680 INFO Torch: torch==2.6.0+rocm6.2.4 torchvision==0.21.0+rocm6.2.4 21:52:57-576398 INFO Packages: diffusers==0.33.0.dev0 transformers==4.48.2 accelerate==1.3.0 gradio==3.43.2 amdgpu.ids: No such file or directory 21:53:00-215134 DEBUG Reading failed: config.json [Errno 2] No such file or directory: 'config.json' 21:53:00-217274 INFO Device detect: memory=16.0 default=balanced 21:53:00-220381 DEBUG Settings: fn="config.json" created 21:53:00-222200 INFO Engine: backend=Backend.DIFFUSERS compute=rocm device=cuda attention="Scaled-Dot-Product" mode=no_grad 21:53:00-223434 DEBUG Save: file="config.json" json=19 bytes=732 time=0.002 21:53:00-224715 DEBUG Migrated styles: file="styles.csv" folder="models/styles" 21:53:00-231265 INFO Available Styles: folder="models/styles" items=288 time=0.01 21:53:00-232599 DEBUG Read: file="html/reference.json" json=63 bytes=32978 time=0.000 fn=_call_with_frames_removed:<module> Could not install automatic (StabilityMatrix.Core.Exceptions.ProcessException: install script failed with code 139: '/home/harasurya/AppImages/Stable-Diffusion/Data/Packages/SD.Next/venv/lib/python3.10/site-packages/_distutils_hack/__init__.py:53: UserWarning: Reliance on distutils from stdlib is deprecated. Users must rely on setuptools to provide the distutils module. Avoid importing distutils or import setuptools first, and avoid setting SETUPTOOLS_USE_DISTUTILS=stdlib. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml\n warnings.warn(\n21:49:54-191464 INFO Starting SD.Next \n21:49:54-194535 INFO Logger: \n file="/home/harasurya/AppImages/Stable-Diffusion/Data/P\n ackages/SD.Next/sdnext.log" level=DEBUG size=64 \n mode=create \n21:49:54-195583 INFO Python: version=3.10.11 platform=Linux \n bin="/home/harasurya/AppImages/Stable-Diffusion/Data/Pa\n ckages/SD.Next/venv/bin/python3" \n venv="/home/harasurya/AppImages/Stable-Diffusion/Data/P\n ackages/SD.Next/venv" \n21:49:54-208617 INFO Version: app=sd.next updated=2025-02-05 hash=9f122234 \n branch=master \n url=https://github.com/vladmandic/automatic/tree/master\n ui=main \n21:49:54-450353 INFO Platform: arch=x86_64 cpu= system=Linux \n release=6.12.11-200.fc41.x86_64 python=3.10.11 \n docker=False \n21:49:54-451885 DEBUG Packages: venv=venv \n site=['venv/lib/python3.10/site-packages'] \n21:49:54-452895 INFO Args: ['--use-rocm', '--debug', '--test'] \n21:49:54-453725 DEBUG Setting environment tuning \n21:49:54-454511 DEBUG Torch allocator: \n "garbage_collection_threshold:0.80,max_split_size_mb:51\n 2" \n21:49:54-489253 DEBUG Torch overrides: cuda=False rocm=True ipex=False \n directml=False openvino=False zluda=False \n21:49:54-490863 INFO ROCm: AMD toolkit detected \n21:49:54-514704 INFO ROCm: agents=['gfx1102'] \n21:49:54-515935 INFO ROCm: version=6.2, using agent gfx1102 \n21:49:54-517091 INFO Python: version=3.10.11 platform=Linux \n bin="/home/harasurya/AppImages/Stable-Diffusion/Data/Pa\n ckages/SD.Next/venv/bin/python3" \n venv="/home/harasurya/AppImages/Stable-Diffusion/Data/P\n ackages/SD.Next/venv" \n21:49:54-518298 DEBUG Install: package="torch" install required \n21:49:54-519150 INFO Torch: download and install in progress... \n cmd="torch==2.6.0+rocm6.2.4 \n torchvision==0.21.0+rocm6.2.4 --index-url \n https://download.pytorch.org/whl/rocm6.2.4" \n21:51:43-770259 DEBUG Install: package="onnx" install required \n21:51:46-829661 DEBUG Install: package="onnxruntime" install required \n21:51:48-323628 INFO Diffusers install: \n commit=f63d32233f402bd603da8f3aa385aecb9c3d8809 \n21:51:56-145925 INFO Install requirements: this may take a while... \n21:52:30-856712 DEBUG Install: package="torch" install required \n21:52:30-857683 INFO Install: verifying requirements \n21:52:30-860234 DEBUG Install: package="psutil" install required \n21:52:30-881354 DEBUG Install: package="rich" install required \n21:52:31-763578 DEBUG Install: package="requests" install required \n21:52:35-509561 INFO Startup: standard \n21:52:35-510373 INFO Verifying submodules \n21:52:46-992252 DEBUG Git detached head detected: \n folder="extensions-builtin/sd-extension-chainner" \n reattach=main \n21:52:46-993691 DEBUG Git submodule: extensions-builtin/sd-extension-chainner\n / main \n21:52:47-010927 DEBUG Git detached head detected: \n folder="extensions-builtin/sd-extension-system-info" \n reattach=main \n21:52:47-012128 DEBUG Git submodule: \n extensions-builtin/sd-extension-system-info / main \n21:52:47-024521 DEBUG Git detached head detected: \n folder="extensions-builtin/sd-webui-agent-scheduler" \n reattach=main \n21:52:47-025612 DEBUG Git submodule: \n extensions-builtin/sd-webui-agent-scheduler / main \n21:52:47-043546 DEBUG Git detached head detected: \n folder="extensions-builtin/sdnext-modernui" \n reattach=main \n21:52:47-044824 DEBUG Git submodule: extensions-builtin/sdnext-modernui / \n main \n21:52:47-068414 DEBUG Git detached head detected: \n folder="extensions-builtin/stable-diffusion-webui-rembg\n " reattach=master \n21:52:47-069506 DEBUG Git submodule: \n extensions-builtin/stable-diffusion-webui-rembg / \n master \n21:52:47-083180 DEBUG Git detached head detected: \n folder="modules/k-diffusion" reattach=master \n21:52:47-084420 DEBUG Git submodule: modules/k-diffusion / master \n21:52:47-094734 DEBUG Git detached head detected: folder="wiki" \n reattach=master \n21:52:47-095724 DEBUG Git submodule: wiki / master \n21:52:47-105580 DEBUG Register paths \n21:52:47-128207 DEBUG Installed packages: 156 \n21:52:47-129184 DEBUG Extensions all: ['Lora', 'sd-extension-chainner', \n 'sd-extension-system-info', 'sd-webui-agent-scheduler',\n 'sdnext-modernui', 'stable-diffusion-webui-rembg'] \n21:52:47-186457 DEBUG Extension installer: \n /home/harasurya/AppImages/Stable-Diffusion/Data/Package\n s/SD.Next/extensions-builtin/sd-webui-agent-scheduler/i\n nstall.py \n21:52:49-185436 INFO Extension installed packages: sd-webui-agent-scheduler \n ['SQLAlchemy==2.0.38', 'greenlet==3.1.1'] \n21:52:49-209580 DEBUG Extension installer: \n /home/harasurya/AppImages/Stable-Diffusion/Data/Package\n s/SD.Next/extensions-builtin/stable-diffusion-webui-rem\n bg/install.py \n21:52:53-526940 INFO Extension installed packages: \n stable-diffusion-webui-rembg ['PyMatting==1.1.13', \n 'opencv-python-headless==4.11.0.86', 'pooch==1.8.2', \n 'rembg==2.0.62'] \n21:52:53-528011 DEBUG Extensions all: [] \n21:52:53-528593 INFO Extensions enabled: ['Lora', 'sd-extension-chainner', \n 'sd-extension-system-info', 'sd-webui-agent-scheduler',\n 'sdnext-modernui', 'stable-diffusion-webui-rembg'] \n21:52:53-529438 INFO Install requirements: this may take a while... \n21:52:53-530098 INFO Install: verifying requirements \n21:52:53-530697 DEBUG Setup complete without errors: 1739242374 \n21:52:53-532810 DEBUG Extension preload: {'extensions-builtin': 0.0, \n 'extensions': 0.0} \n21:52:53-533873 INFO Command line args: ['--use-rocm', '--debug', '--test'] \n use_rocm=True test=True debug=True args=[] \n21:52:53-534723 DEBUG Env flags: [] \n21:52:53-535549 DEBUG Starting module: <module 'webui' from \n '/home/harasurya/AppImages/Stable-Diffusion/Data/Packag\n es/SD.Next/webui.py'> \n21:52:57-572436 DEBUG System: cores=12 affinity=12 threads=6 \n21:52:57-575680 INFO Torch: torch==2.6.0+rocm6.2.4 \n torchvision==0.21.0+rocm6.2.4 \n21:52:57-576398 INFO Packages: diffusers==0.33.0.dev0 transformers==4.48.2 \n accelerate==1.3.0 gradio==3.43.2 \namdgpu.ids: No such file or directory\n21:53:00-215134 DEBUG Reading failed: config.json [Errno 2] No such file or \n directory: 'config.json' \n21:53:00-217274 INFO Device detect: memory=16.0 default=balanced \n21:53:00-220381 DEBUG Settings: fn="config.json" created \n21:53:00-222200 INFO Engine: backend=Backend.DIFFUSERS compute=rocm \n device=cuda attention="Scaled-Dot-Product" mode=no_grad\n21:53:00-223434 DEBUG Save: file="config.json" json=19 bytes=732 time=0.002 \n21:53:00-224715 DEBUG Migrated styles: file="styles.csv" \n folder="models/styles" \n21:53:00-231265 INFO Available Styles: folder="models/styles" items=288 \n time=0.01 \n21:53:00-232599 DEBUG Read: file="html/reference.json" json=63 bytes=32978 \n time=0.000 fn=_call_with_frames_removed:<module> \n' at StabilityMatrix.Core.Python.PyVenvRunner.CustomInstall(ProcessArgs args, Action1 outputDataReceived)
at StabilityMatrix.Core.Models.Packages.VladAutomatic.InstallPackage(String installLocation, InstalledPackage installedPackage, InstallPackageOptions options, IProgress1 progress, Action1 onConsoleOutput, CancellationToken cancellationToken)
at StabilityMatrix.Core.Models.Packages.VladAutomatic.InstallPackage(String installLocation, InstalledPackage installedPackage, InstallPackageOptions options, IProgress1 progress, Action1 onConsoleOutput, CancellationToken cancellationToken)
at StabilityMatrix.Core.Models.PackageModification.InstallPackageStep.ExecuteAsync(IProgress1 progress, CancellationToken cancellationToken) at StabilityMatrix.Core.Models.PackageModification.PackageModificationRunner.ExecuteSteps(IEnumerable1 steps))
`

The other error from when it seems to be installed and fails to run is:

/home/harasurya/AppImages/Stable-Diffusion/Data/Packages/SD.Next/venv/lib/python3.10/site-packages/_distutils_hack/__init__.py:53: UserWarning: Reliance on distutils from stdlib is deprecated. Users must rely on setuptools to provide the distutils module. Avoid importing distutils or import setuptools first, and avoid setting SETUPTOOLS_USE_DISTUTILS=stdlib. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml warnings.warn( 21:58:00-688495 INFO Starting SD.Next 21:58:00-691219 INFO Logger: file="/home/harasurya/AppImages/Stable-Diffusion/Data/P ackages/SD.Next/sdnext.log" level=INFO size=8300 mode=append 21:58:00-692154 INFO Python: version=3.10.11 platform=Linux bin="/home/harasurya/AppImages/Stable-Diffusion/Data/Pa ckages/SD.Next/venv/bin/python3" venv="/home/harasurya/AppImages/Stable-Diffusion/Data/P ackages/SD.Next/venv" 21:58:00-705398 INFO Version: app=sd.next updated=2025-02-05 hash=9f122234 branch=master url=https://github.com/vladmandic/automatic/tree/master ui=main 21:58:00-936754 INFO Platform: arch=x86_64 cpu= system=Linux release=6.12.11-200.fc41.x86_64 python=3.10.11 docker=False 21:58:00-938282 INFO Args: ['--gradio-allowed-path', '/home/harasurya/AppImages/Stable-Diffusion/Data/Images '] 21:58:00-994345 INFO ROCm: AMD toolkit detected 21:58:01-018055 INFO ROCm: agents=['gfx1102'] 21:58:01-019070 INFO ROCm: version=6.2, using agent gfx1102 21:58:01-020068 INFO Python: version=3.10.11 platform=Linux bin="/home/harasurya/AppImages/Stable-Diffusion/Data/Pa ckages/SD.Next/venv/bin/python3" venv="/home/harasurya/AppImages/Stable-Diffusion/Data/P ackages/SD.Next/venv" 21:58:01-021340 INFO Diffusers update: package=diffusers 0.33.0.dev0 current= target=f63d32233f402bd603da8f3aa385aecb9c3d8809 21:58:08-643941 WARNING Modified files: ['models/VAE-approx/model.pt', 'models/karlo/ViT-L-14_stats.th'] 21:58:08-668244 INFO Install: verifying requirements 21:58:08-675332 INFO Extensions: disabled=[] 21:58:08-676033 INFO Extensions: enabled=['Lora', 'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sdnext-modernui', 'stable-diffusion-webui-rembg'] extensions-builtin 21:58:08-677210 INFO Extensions: enabled=[] extensions 21:58:08-678076 INFO Startup: quick launch 21:58:08-678931 INFO Extensions: disabled=[] 21:58:08-679657 INFO Extensions: enabled=['Lora', 'sd-extension-chainner', 'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sdnext-modernui', 'stable-diffusion-webui-rembg'] extensions-builtin 21:58:08-680976 INFO Extensions: enabled=[] extensions usage: [-h] [--backend {original,diffusers}] [--config CONFIG] [--ui-config UI_CONFIG] [--medvram] [--lowvram] [--freeze] [--device-id DEVICE_ID] [--ckpt CKPT] [--data-dir DATA_DIR] [--models-dir MODELS_DIR] [--no-hashing] [--no-metadata] [--server-name SERVER_NAME] [--tls-keyfile TLS_KEYFILE] [--tls-certfile TLS_CERTFILE] [--tls-selfsign] [--cors-origins CORS_ORIGINS] [--cors-regex CORS_REGEX] [--subpath SUBPATH] [--autolaunch] [--auth AUTH] [--auth-file AUTH_FILE] [--api-only] [--allowed-paths ALLOWED_PATHS [ALLOWED_PATHS ...]] [--share] [--insecure] [--listen] [--port PORT] [--reset] [--upgrade] [--requirements] [--reinstall] [--optional] [--uv] [--quick] [--skip-requirements] [--skip-extensions] [--skip-git] [--skip-torch] [--skip-all] [--skip-env] [--use-directml] [--use-openvino] [--use-ipex] [--use-cuda] [--use-nightly] [--use-rocm] [--use-zluda] [--use-xformers] [--safe] [--experimental] [--test] [--version] [--ignore] [--log LOG] [--debug] [--profile] [--docs] [--api-log] [--agent-scheduler-sqlite-file AGENT_SCHEDULER_SQLITE_FILE] [args ...] : error: unrecognized arguments: --gradio-allowed-path

Which began at the same time the previous error started to happen so I imagine they're related in some way.

vladmandic · 2025-02-11T17:22:10Z

[ 3521.235743] python3[20478]: segfault at 18 ip 00007f12928af9bd sp 00007ffeecde6e60 error 4 in libamdhip64.so[2af9bd,7f1292600000+439000] likely on CPU 10 (core 4, socket 0)

so its crashing inside rocm. this can be due to any number of reasons, but its is typical for amd if its attempting to use wrong functions intended for wrong gpu subtype. search previous issues for HSA_OVERRIDE_GFX_VERSION (i don't know why is rocm so silly that it cannot autodetect its own gpu type)

Disty0 · 2025-02-11T19:20:26Z

It detected the gpu fine, the issue is this:

gfx1102 / RX 7600 series are unsupported by ROCm, you have to downgrade to ROCm 5.7.

panton41 · 2025-02-11T23:32:02Z

But, repeating yet again, this was working back in December, on a Fedora 41 derivative, and stopped working in early January with these exact errors. Fedora 41 shipped with ROCm 6.2, so it was working until that point.

Edit: From the looks of it while searching, AMD considers the incompatibility a regression and seems to be working on fixing it. Breaking compatibility on their latest generation of parts is a bad look, but AMD seems to be masters of snatching defeat from the jaws of victory when it comes to competing with nVidia so it seems par for the course.

vladmandic added question Further information is requested platform Platform specific problem labels Jan 31, 2025

vladmandic changed the title ~~[Issue]: Unable to install under Stablity Matrix on Fedora 41 with AMD GPU~~ [Issue]: crash using Stablity Matrix on Fedora 41 with AMD GPU Jan 31, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Issue]: crash using Stablity Matrix on Fedora 41 with AMD GPU #3731

[Issue]: crash using Stablity Matrix on Fedora 41 with AMD GPU #3731

panton41 commented Jan 30, 2025

Disty0 commented Jan 31, 2025 •

edited

Loading

panton41 commented Jan 31, 2025 •

edited

Loading

vladmandic commented Jan 31, 2025

vladmandic commented Feb 6, 2025

panton41 commented Feb 11, 2025

panton41 commented Feb 11, 2025

vladmandic commented Feb 11, 2025

Disty0 commented Feb 11, 2025

panton41 commented Feb 11, 2025 •

edited

Loading

[Issue]: crash using Stablity Matrix on Fedora 41 with AMD GPU #3731

[Issue]: crash using Stablity Matrix on Fedora 41 with AMD GPU #3731

Comments

panton41 commented Jan 30, 2025

Issue Description

Version Platform Description

Relevant log output

Backend

UI

Branch

Model

Acknowledgements

Disty0 commented Jan 31, 2025 • edited Loading

panton41 commented Jan 31, 2025 • edited Loading

vladmandic commented Jan 31, 2025

vladmandic commented Feb 6, 2025

panton41 commented Feb 11, 2025

panton41 commented Feb 11, 2025

vladmandic commented Feb 11, 2025

Disty0 commented Feb 11, 2025

panton41 commented Feb 11, 2025 • edited Loading

Disty0 commented Jan 31, 2025 •

edited

Loading

panton41 commented Jan 31, 2025 •

edited

Loading

panton41 commented Feb 11, 2025 •

edited

Loading