Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

vkDestroyCommandPool: Invalid device [VUID-vkDestroyCommandPool-device-parameter] starting with Release r9_mod_v7 #16

Open
Selur opened this issue Dec 3, 2023 · 23 comments
Open

vkDestroyCommandPool: Invalid device [VUID-vkDestroyCommandPool-device-parameter] starting with Release r9_mod_v7 #16

Selur opened this issue Dec 3, 2023 · 23 comments

Comments

@Selur
Copy link

Selur commented Dec 3, 2023

I'm using r9_mod_v10 through:

from FillDuplicateFrames import FillDuplicateFrames
fdf = FillDuplicateFrames(clip=clip, method="RIFE")

which calls

class FillDuplicateFrames:
  # constructor
  def __init__(self, clip: vs.VideoNode, thresh: float=0.001, method: str='SVP', debug: bool=False, rifeSceneThr: float=0.15, device_index: int=0):
      self.clip = core.std.PlaneStats(clip, clip[0]+clip)
      self.thresh = thresh
      self.debug = debug
      self.method = method
      self.smooth = None
      self.rifeSceneThr = rifeSceneThr
      self.device_index = device_index
          
  def interpolate(self, n, f):
    out = self.get_current_or_interpolate(n)
    if self.debug:
      return out.text.Text(text="avg: "+str(f.props['PlaneStatsDiff']),alignment=8)            
    return out

  def interpolateWithRIFE(self, clip, n, start, end, rifeModel=22, rifeTTA=False, rifeUHD=False):
    if clip.format.id != vs.RGBS:
      raise ValueError(f'FillDuplicateFrames: "clip" needs to be RGBS when using \'{self.method}\'!')
      
    if self.rifeSceneThr != 0:
      clip = core.misc.SCDetect(clip=clip,threshold=self.rifeSceneThr)
    
    num = end - start
    self.smooth = core.rife.RIFE(clip, model=rifeModel, factor_num=num, tta=rifeTTA,uhd=rifeUHD,gpu_id=self.device_index)
    self.smooth_start = start
    self.smooth_end   = end
    return self.smooth[n-start]
   
  def interpolateWithMV(self, clip, n, start, end):   
    num = end - start
    sup = core.mv.Super(clip, pel=2, hpad=0, vpad=0)
    bvec = core.mv.Analyse(sup, blksize=16, isb=True, chroma=True, search=3, searchparam=1)
    fvec = core.mv.Analyse(sup, blksize=16, isb=False, chroma=True, search=3, searchparam=1)
    self.smooth = core.mv.FlowFPS(clip, sup, bvec, fvec, num=num, den=1, mask=2)
    self.smooth_start = start
    self.smooth_end   = end
    out = self.smooth[n-start]
    if self.debug:
      return out.text.Text(text="MV",alignment=9)
    return out

  def interpolateWithSVP(self, clip, n, start, end):   
    if clip.format.id != vs.YUV420P8:
      raise ValueError(f'FillDuplicateFrames: "clip" needs to be YUV420P8 when using \'{self.method}\'!')
    if self.method == 'SVP':
      super = core.svp1.Super(clip,"{gpu:1}")
    else: # self.method == 'SVPCPU':
      super = core.svp1.Super(clip,"{gpu:0}")
    vectors = core.svp1.Analyse(super["clip"],super["data"],clip,"{}")
    num = end - start
    self.smooth = core.svp2.SmoothFps(clip,super["clip"],super["data"],vectors["clip"],vectors["data"],f"{{rate:{{num:{num},den:1,abs:true}}}}")
    self.smooth_start = start
    self.smooth_end   = end
    out = self.smooth[n-start]
    if self.debug:
      return out.text.Text(text="SVP",alignment=9)
    return out
  
  def get_current_or_interpolate(self, n):
    if self.is_not_duplicate(n):
      #current non dublicate selected
      if self.debug:
        return self.clip[n].text.Text(text="Input (1)", alignment=9)
      return self.clip[n]

    #dublicate frame, frame is interpolated
    for start in reversed(range(n+1)):
      if self.is_not_duplicate(start):
        break
    else: #there are all black frames preceding n, return current n frame // will be executed then for-look does not end with a break
      if self.debug:
        return self.clip[n].text.Text(text="Input (2)", alignment=9)
      return self.clip[n]
  
    for end in range(n, len(self.clip)):
      if self.is_not_duplicate(end):
        break
    else:
      #there are all black frames to the end, return current n frame
      if self.debug:
        return self.clip[n].text.Text(text="Input(3)", alignment=9)
      return self.clip[n]

    #does interpolated smooth clip exist for requested n frame? Use n frame from it.
    if self.smooth is not None and start >= self.smooth_start and end <= self.smooth_end:
      if self.debug:
        return self.smooth[n-start].text.Text(text=self.method, alignment=9)
      return self.smooth[n-start]

    #interpolating two frame clip  into end-start+1 fps
    clip = self.clip[start] + self.clip[end]
    clip = clip.std.AssumeFPS(fpsnum=1, fpsden=1)
    if self.method == 'SVP' or self.method == 'SVPcpu':  
      return self.interpolateWithSVP(clip, n, start, end)
    elif self.method == 'RIFE':
      return self.interpolateWithRIFE(clip, n, start, end)
    elif self.method == 'MV':
      return self.interpolateWithMV(clip, n, start, end)
    else:
      raise ValueError(f'ReplaceBlackFrames: "method" \'{self.method}\' is not supported atm.')

  def is_not_duplicate(self, n):
    return self.clip.get_frame(n).props['PlaneStatsDiff'] > self.thresh
  
  @property
  def out(self):
    self.clip = core.std.PlaneStats(self.clip, self.clip[0] + self.clip)
    return core.std.FrameEval(self.clip, self.interpolate, prop_src=self.clip)

which sadly throws the above error:

VSPipe.exe  c:\Users\Selur\Desktop\Testing.vpy -c y4m NUL
[0 NVIDIA GeForce RTX 4080]  queueC=2[8]  queueG=0[16]  queueT=1[2]
[0 NVIDIA GeForce RTX 4080]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[0 NVIDIA GeForce RTX 4080]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[0 NVIDIA GeForce RTX 4080]  subgroup=32  basic/vote/ballot/shuffle=1/1/1/1
[0 NVIDIA GeForce RTX 4080]  fp16-matrix-16_8_8/16_8_16/16_16_16=1/1/1
[1 Intel(R) Arc(TM) A380 Graphics]  queueC=1[1]  queueG=0[1]  queueT=2[1]
[1 Intel(R) Arc(TM) A380 Graphics]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[1 Intel(R) Arc(TM) A380 Graphics]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[1 Intel(R) Arc(TM) A380 Graphics]  subgroup=32  basic/vote/ballot/shuffle=1/1/1/1
[1 Intel(R) Arc(TM) A380 Graphics]  fp16-matrix-16_8_8/16_8_16/16_16_16=0/0/0
[2 AMD Radeon(TM) Graphics]  queueC=1[2]  queueG=0[1]  queueT=2[1]
[2 AMD Radeon(TM) Graphics]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[2 AMD Radeon(TM) Graphics]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[2 AMD Radeon(TM) Graphics]  subgroup=64  basic/vote/ballot/shuffle=1/1/1/1
[2 AMD Radeon(TM) Graphics]  fp16-matrix-16_8_8/16_8_16/16_16_16=0/0/0
FATAL ERROR! reclaim_blob_allocator get wild allocator 00000298BE73F700
FATAL ERROR! reclaim_staging_allocator get wild allocator 00000298BE73FC10
FATAL ERROR! reclaim_blob_allocator get wild allocator 00000298BE73FDC0
FATAL ERROR! reclaim_staging_allocator get wild allocator 00000298BE740240
ERROR:            vkDestroyCommandPool: Invalid device [VUID-vkDestroyCommandPool-device-parameter]
ERROR:            vkDestroyCommandPool: Invalid device [VUID-vkDestroyCommandPool-device-parameter]

I tested some older librife versions and with Release r9_mod_v6 it works, all versions after that one produce the crash.

Would be nice if this could be fixed.
@styler00dollar
Copy link
Owner

I tested r9_mod_v10 on my own system and it works fine on Manjaro with vspipe. All I did since v6 was to update ncnn submodules. I don't change code and only add models. That means that it is either Windows related or related to your local system or scripts. Try out all gpu ids, since it says the device you used is invalid and try minimal code. Make sure your drivers are set up correctly, since you have multiple gpus from different vendors. The above example is way too big. I can't fix if I can't reproduce.

@Selur
Copy link
Author

Selur commented Dec 4, 2023

Strange thing is that if I use just RIFE it works fine, only if I use it through FillDuplicateFrames I get the crash.
I adjusted the code to support RGBH:

import vapoursynth as vs
from vapoursynth import core

'''
call using:

from FillDuplicateFrames import FillDuplicateFrames
fdf = FillDuplicateFrames(clip, debug=True, thresh=0.001, method='SVP')
//fdf = FillDuplicateFrames(clip, debug=True, thresh=0.001, method='MV')
//fdf = FillDuplicateFrames(clip, debug=True, thresh=0.001, method='RIFE', rifeSceneThr=0.15)
clip = fdf.out

Replaces duplicate frames with interpolations.
v0.0.2
'''

class FillDuplicateFrames:
  # constructor
  def __init__(self, clip: vs.VideoNode, thresh: float=0.001, method: str='SVP', debug: bool=False, rifeSceneThr: float=0.15, device_index: int=0):
      fp16 = clip.format.id == vs.RGBH
      if (fp16):
        clip = core.resize.Bicubic(clip=clip,format=vs.RGBS)
        self.clip = core.std.PlaneStats(clip, clip[0]+clip)
        clip = core.resize.Bicubic(clip=clip,format=vs.RGBH)
      else:
        self.clip = core.std.PlaneStats(clip, clip[0]+clip)
      self.thresh = thresh
      self.debug = debug
      self.method = method
      self.smooth = None
      self.rifeSceneThr = rifeSceneThr
      self.device_index = device_index
          
  def interpolate(self, n, f):
    out = self.get_current_or_interpolate(n)
    if self.debug:
      return out.text.Text(text="avg: "+str(f.props['PlaneStatsDiff']),alignment=8)            
    return out

  def interpolateWithRIFE(self, clip, n, start, end, rifeModel=22, rifeTTA=False, rifeUHD=False):
    if clip.format.id != vs.RGBS and clip.format.id != vs.RGBH:
      raise ValueError(f'FillDuplicateFrames: "clip" needs to be RGBS or RGBH when using \'{self.method}\'!')
      
    if self.rifeSceneThr != 0:
      fp16 = clip.format.id == vs.RGBH
      if (fp16):
        clip = core.resize.Bicubic(clip=clip,format=vs.RGBS)
      clip = core.misc.SCDetect(clip=clip,threshold=self.rifeSceneThr)
      if (fp16):
        clip = core.resize.Bicubic(clip=clip,format=vs.RGBH)
    
    num = end - start
    self.smooth = core.rife.RIFE(clip, model=rifeModel, factor_num=num, tta=rifeTTA,uhd=rifeUHD,gpu_id=self.device_index)
    self.smooth_start = start
    self.smooth_end   = end
    return self.smooth[n-start]
   
  def interpolateWithMV(self, clip, n, start, end):   
    num = end - start
    sup = core.mv.Super(clip, pel=2, hpad=0, vpad=0)
    bvec = core.mv.Analyse(sup, blksize=16, isb=True, chroma=True, search=3, searchparam=1)
    fvec = core.mv.Analyse(sup, blksize=16, isb=False, chroma=True, search=3, searchparam=1)
    self.smooth = core.mv.FlowFPS(clip, sup, bvec, fvec, num=num, den=1, mask=2)
    self.smooth_start = start
    self.smooth_end   = end
    out = self.smooth[n-start]
    if self.debug:
      return out.text.Text(text="MV",alignment=9)
    return out

  def interpolateWithSVP(self, clip, n, start, end):   
    if clip.format.id != vs.YUV420P8:
      raise ValueError(f'FillDuplicateFrames: "clip" needs to be YUV420P8 when using \'{self.method}\'!')
    if self.method == 'SVP':
      super = core.svp1.Super(clip,"{gpu:1}")
    else: # self.method == 'SVPCPU':
      super = core.svp1.Super(clip,"{gpu:0}")
    vectors = core.svp1.Analyse(super["clip"],super["data"],clip,"{}")
    num = end - start
    self.smooth = core.svp2.SmoothFps(clip,super["clip"],super["data"],vectors["clip"],vectors["data"],f"{{rate:{{num:{num},den:1,abs:true}}}}")
    self.smooth_start = start
    self.smooth_end   = end
    out = self.smooth[n-start]
    if self.debug:
      return out.text.Text(text="SVP",alignment=9)
    return out
  
  def get_current_or_interpolate(self, n):
    if self.is_not_duplicate(n):
      #current non dublicate selected
      if self.debug:
        return self.clip[n].text.Text(text="Input (1)", alignment=9)
      return self.clip[n]

    #dublicate frame, frame is interpolated
    for start in reversed(range(n+1)):
      if self.is_not_duplicate(start):
        break
    else: #there are all black frames preceding n, return current n frame // will be executed then for-look does not end with a break
      if self.debug:
        return self.clip[n].text.Text(text="Input (2)", alignment=9)
      return self.clip[n]
  
    for end in range(n, len(self.clip)):
      if self.is_not_duplicate(end):
        break
    else:
      #there are all black frames to the end, return current n frame
      if self.debug:
        return self.clip[n].text.Text(text="Input(3)", alignment=9)
      return self.clip[n]

    #does interpolated smooth clip exist for requested n frame? Use n frame from it.
    if self.smooth is not None and start >= self.smooth_start and end <= self.smooth_end:
      if self.debug:
        return self.smooth[n-start].text.Text(text=self.method, alignment=9)
      return self.smooth[n-start]

    #interpolating two frame clip  into end-start+1 fps
    clip = self.clip[start] + self.clip[end]
    clip = clip.std.AssumeFPS(fpsnum=1, fpsden=1)
    if self.method == 'SVP' or self.method == 'SVPcpu':  
      return self.interpolateWithSVP(clip, n, start, end)
    elif self.method == 'RIFE':
      return self.interpolateWithRIFE(clip, n, start, end)
    elif self.method == 'MV':
      return self.interpolateWithMV(clip, n, start, end)
    else:
      raise ValueError(f'ReplaceBlackFrames: "method" \'{self.method}\' is not supported atm.')

  def is_not_duplicate(self, n):
    return self.clip.get_frame(n).props['PlaneStatsDiff'] > self.thresh
  
  @property
  def out(self):
    self.clip = core.std.PlaneStats(self.clip, self.clip[0] + self.clip)
    return core.std.FrameEval(self.clip, self.interpolate, prop_src=self.clip)

but that didn't help.
Using vs-rife (https://github.com/HolyWu/vs-rife) the script works fine.
I'm on Windows 11 using a Geforce RTX 4080 with NVIDIA Studio drivers 546.01.
I adjusted the VSPipe call to 'VSPipe.exe c:\Users\Selur\Desktop\Testing.vpy -c y4m --progress NUL'
Using different gpu_ids, I still get a crash, but no error message.
(explicitly setting gpu_thread=1 doesn't help either)
Using r9_mod6 works for all my gpus.
Maybe I find some time on the weekend to try different driver versions, but since all GPUs fail I doubt that it's the drivers, but atm. it seems to me like there is a problem with ncnn.

@mafiosnik777
Copy link
Contributor

I adjusted the code to support RGBH

The plugin never did and still doesn't support RGBH lol

@Selur
Copy link
Author

Selur commented Dec 4, 2023

No problem, did that mainly to speed up vs-rife. (in my test script, I did use RGBS; yup, tested again to be sure)

@styler00dollar
Copy link
Owner

styler00dollar commented Apr 3, 2024 •
edited
Loading

Does that error still happen? Otherwise I will just close the issue if it doesn't appear anymore.

@Selur
Copy link
Author

Selur commented Apr 3, 2024

Just reverted my FillDuplicateFrames version and tested it.
With the latest version it works with RGBH too.

@Selur Selur closed this as completed Apr 3, 2024
@Selur
Copy link
Author

Selur commented Apr 3, 2024

Correction!
Nope still fails, using:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import ctypes
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("F:/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
# Loading Plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/FrameFilter/RIFE/librife.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/EEDI3m_opencl.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/NNEDI3CL.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/scenechange.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DeinterlaceFilter/Bwdif/Bwdif.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
# Import scripts
import havsfunc
# source: 'G:\TestClips&Co\files\test.avi'
# current color space: YUV420P8, bit depth: 8, resolution: 640x352, fps: 25, scanorder: top field first, yuv luminance scale: limited, matrix: 470bg
# Loading G:\TestClips&Co\files\test.avi using LWLibavSource
clip = core.lsmas.LWLibavSource(source="G:/TestClips&Co/files/test.avi", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
frame = clip.get_frame(0)
# Setting detected color matrix (470bg).
clip = core.std.SetFrameProps(clip, _Matrix=5)
# Setting color transfer (170), if it is not set.
if '_Transfer' not in frame.props or not frame.props['_Transfer']:
  clip = core.std.SetFrameProps(clip, _Transfer=6)
# Setting color primaries info (to 470), if it is not set.
if '_Primaries' not in frame.props or not frame.props['_Primaries']:
  clip = core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=2) # tff
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=False, opencl=True) # new fps: 25
# Making sure content is preceived as frame based
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive
clip = clip[::2] # selecting previously even frames
# adjusting color space from YUV420P8 to RGBH for vsFillDuplicateFrames
clip = core.resize.Bicubic(clip=clip, format=vs.RGBH, matrix_in_s="470bg", range_s="limited")
from FillDuplicateFrames import FillDuplicateFrames
fdf = FillDuplicateFrames(clip=clip, method="RIFE")
clip = fdf.out
# adjusting output color from: RGBH to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited", dither_type="error_diffusion")
# set output frame rate to 25fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Output
clip.set_output()

and:

import vapoursynth as vs
from vapoursynth import core

'''
call using:

from FillDuplicateFrames import FillDuplicateFrames
fdf = FillDuplicateFrames(clip, debug=True, thresh=0.001, method='SVP')
//fdf = FillDuplicateFrames(clip, debug=True, thresh=0.001, method='MV')
//fdf = FillDuplicateFrames(clip, debug=True, thresh=0.001, method='RIFE', rifeSceneThr=0.15)
clip = fdf.out

Replaces duplicate frames with interpolations.
v0.0.3
0.0.4 removed and added back RGBH support or RIFE
0.0.3 allow to set device_index for RIFE and support RGBH input for RIFE
0.0.4 removed RGBH since 
'''

class FillDuplicateFrames:
  # constructor
  def __init__(self, clip: vs.VideoNode, thresh: float=0.001, method: str='SVP', debug: bool=False, rifeSceneThr: float=0.15, device_index: int=0):
      fp16 = clip.format.id == vs.RGBH
      if (fp16):
        clip = core.resize.Bicubic(clip=clip,format=vs.RGBS)
        self.clip = core.std.PlaneStats(clip, clip[0]+clip)
        clip = core.resize.Bicubic(clip=clip,format=vs.RGBH)
      else:
        self.clip = core.std.PlaneStats(clip, clip[0]+clip)
      self.thresh = thresh
      self.debug = debug
      self.method = method
      self.smooth = None
      self.rifeSceneThr = rifeSceneThr
      self.device_index = device_index
          
  def interpolate(self, n, f):
    out = self.get_current_or_interpolate(n)
    if self.debug:
      return out.text.Text(text="avg: "+str(f.props['PlaneStatsDiff']),alignment=8)            
    return out

  def interpolateWithRIFE(self, clip, n, start, end, rifeModel=22, rifeTTA=False, rifeUHD=False):
    if clip.format.id != vs.RGBS:
      raise ValueError(f'FillDuplicateFrames: "clip" needs to be RGBS when using \'{self.method}\'!')
      
    if self.rifeSceneThr != 0:
      fp16 = clip.format.id == vs.RGBH
      if (fp16):
        clip = core.resize.Bicubic(clip=clip,format=vs.RGBS)
      clip = core.misc.SCDetect(clip=clip,threshold=self.rifeSceneThr)
      if (fp16):
        clip = core.resize.Bicubic(clip=clip,format=vs.RGBH)
        
    num = end - start
    
    self.smooth = core.rife.RIFE(clip, model=rifeModel, factor_num=num, tta=rifeTTA,uhd=rifeUHD,gpu_id=self.device_index)
    self.smooth_start = start
    self.smooth_end   = end
    return self.smooth[n-start]
   
  def interpolateWithMV(self, clip, n, start, end):   
    num = end - start
    sup = core.mv.Super(clip, pel=2, hpad=0, vpad=0)
    bvec = core.mv.Analyse(sup, blksize=16, isb=True, chroma=True, search=3, searchparam=1)
    fvec = core.mv.Analyse(sup, blksize=16, isb=False, chroma=True, search=3, searchparam=1)
    self.smooth = core.mv.FlowFPS(clip, sup, bvec, fvec, num=num, den=1, mask=2)
    self.smooth_start = start
    self.smooth_end   = end
    out = self.smooth[n-start]
    if self.debug:
      return out.text.Text(text="MV",alignment=9)
    return out

  def interpolateWithSVP(self, clip, n, start, end):   
    if clip.format.id != vs.YUV420P8:
      raise ValueError(f'FillDuplicateFrames: "clip" needs to be YUV420P8 when using \'{self.method}\'!')
    if self.method == 'SVP':
      super = core.svp1.Super(clip,"{gpu:1}")
    else: # self.method == 'SVPCPU':
      super = core.svp1.Super(clip,"{gpu:0}")
    vectors = core.svp1.Analyse(super["clip"],super["data"],clip,"{}")
    num = end - start
    self.smooth = core.svp2.SmoothFps(clip,super["clip"],super["data"],vectors["clip"],vectors["data"],f"{{rate:{{num:{num},den:1,abs:true}}}}")
    self.smooth_start = start
    self.smooth_end   = end
    out = self.smooth[n-start]
    if self.debug:
      return out.text.Text(text="SVP",alignment=9)
    return out
  
  def get_current_or_interpolate(self, n):
    if self.is_not_duplicate(n):
      #current non dublicate selected
      if self.debug:
        return self.clip[n].text.Text(text="Input (1)", alignment=9)
      return self.clip[n]

    #dublicate frame, frame is interpolated
    for start in reversed(range(n+1)):
      if self.is_not_duplicate(start):
        break
    else: #there are all black frames preceding n, return current n frame // will be executed then for-look does not end with a break
      if self.debug:
        return self.clip[n].text.Text(text="Input (2)", alignment=9)
      return self.clip[n]
  
    for end in range(n, len(self.clip)):
      if self.is_not_duplicate(end):
        break
    else:
      #there are all duplicate frames to the end, return current n frame
      if self.debug:
        return self.clip[n].text.Text(text="Input(3)", alignment=9)
      return self.clip[n]

    #does interpolated smooth clip exist for requested n frame? Use n frame from it.
    if self.smooth is not None and start >= self.smooth_start and end <= self.smooth_end:
      if self.debug:
        return self.smooth[n-start].text.Text(text=self.method, alignment=9)
      return self.smooth[n-start]

    #interpolating two frame clip  into end-start+1 fps
    clip = self.clip[start] + self.clip[end]
    clip = clip.std.AssumeFPS(fpsnum=1, fpsden=1)
    if self.method == 'SVP' or self.method == 'SVPcpu':  
      return self.interpolateWithSVP(clip, n, start, end)
    elif self.method == 'RIFE':
      return self.interpolateWithRIFE(clip, n, start, end)
    elif self.method == 'MV':
      return self.interpolateWithMV(clip, n, start, end)
    else:
      raise ValueError(f'FillDuplicateFrames: "method" \'{self.method}\' is not supported atm.')

  def is_not_duplicate(self, n):
    return self.clip.get_frame(n).props['PlaneStatsDiff'] > self.thresh
  
  @property
  def out(self):
    self.clip = core.std.PlaneStats(self.clip, self.clip[0] + self.clip)
    return core.std.FrameEval(self.clip, self.interpolate, prop_src=self.clip)

it still crashes when called through:

VSPipe.exe c:\Users\Selur\Desktop\test.vpy -c y4m NUL --progress

Script evaluation done in 0.69 seconds
[0 NVIDIA GeForce RTX 4080]  queueC=2[8]  queueG=0[16]  queueT=1[2]
[0 NVIDIA GeForce RTX 4080]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[0 NVIDIA GeForce RTX 4080]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[0 NVIDIA GeForce RTX 4080]  subgroup=32  basic/vote/ballot/shuffle=1/1/1/1
[0 NVIDIA GeForce RTX 4080]  fp16-matrix-16_8_8/16_8_16/16_16_16=1/1/1
[1 AMD Radeon(TM) Graphics]  queueC=1[2]  queueG=0[1]  queueT=2[1]
[1 AMD Radeon(TM) Graphics]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[1 AMD Radeon(TM) Graphics]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[1 AMD Radeon(TM) Graphics]  subgroup=64  basic/vote/ballot/shuffle=1/1/1/1
[1 AMD Radeon(TM) Graphics]  fp16-matrix-16_8_8/16_8_16/16_16_16=0/0/0
[2 Intel(R) Arc(TM) A380 Graphics]  queueC=1[1]  queueG=0[1]  queueT=2[1]
[2 Intel(R) Arc(TM) A380 Graphics]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[2 Intel(R) Arc(TM) A380 Graphics]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[2 Intel(R) Arc(TM) A380 Graphics]  subgroup=32  basic/vote/ballot/shuffle=1/1/1/1
[2 Intel(R) Arc(TM) A380 Graphics]  fp16-matrix-16_8_8/16_8_16/16_16_16=0/0/0
[0 NVIDIA GeForce RTX 4080]  queueC=2[8]  queueG=0[16]  queueT=1[2]
[0 NVIDIA GeForce RTX 4080]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[0 NVIDIA GeForce RTX 4080]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[0 NVIDIA GeForce RTX 4080]  subgroup=32  basic/vote/ballot/shuffle=1/1/1/1
[0 NVIDIA GeForce RTX 4080]  fp16-matrix-16_8_8/16_8_16/16_16_16=1/1/1
[1 AMD Radeon(TM) Graphics]  queueC=1[2]  queueG=0[1]  queueT=2[1]
[1 AMD Radeon(TM) Graphics]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[1 AMD Radeon(TM) Graphics]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[1 AMD Radeon(TM) Graphics]  subgroup=64  basic/vote/ballot/shuffle=1/1/1/1
[1 AMD Radeon(TM) Graphics]  fp16-matrix-16_8_8/16_8_16/16_16_16=0/0/0
[2 Intel(R) Arc(TM) A380 Graphics]  queueC=1[1]  queueG=0[1]  queueT=2[1]
[2 Intel(R) Arc(TM) A380 Graphics]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[2 Intel(R) Arc(TM) A380 Graphics]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[2 Intel(R) Arc(TM) A380 Graphics]  subgroup=32  basic/vote/ballot/shuffle=1/1/1/1
[2 Intel(R) Arc(TM) A380 Graphics]  fp16-matrix-16_8_8/16_8_16/16_16_16=0/0/0

it just doesn't show the error message.

@Selur Selur reopened this Apr 3, 2024
@Selur
Copy link
Author

Selur commented Apr 21, 2024

does also happen in: https://github.com/Selur/VapoursynthScriptsInHybrid/blob/master/filldrops.py
works fine with r9_mod_v6, any version later crashes

@TNTwise
Copy link
Contributor

TNTwise commented Apr 21, 2024

This is most likely due to a ncnn update causing simplevk to be used by default. You have to disable NCNN_SIMPLEVK in the meson file. I cant seem to get this to work across macos, windows, and ubuntu as macos seems to need simplevk in order to compile.

@TNTwise
Copy link
Contributor

TNTwise commented Apr 21, 2024

I got it to compile https://github.com/TNTwise/VapourSynth-RIFE-ncnn-Vulkan/releases/tag/V1
Please try with this build and see if it fixes it or not.

@Selur
Copy link
Author

Selur commented Apr 21, 2024 •
edited
Loading

sadly no, it works on the first frame that gets interpolated, but crashes on the next
(so it's kind of better, since it does not crash on the first interpolated frame,...)

@TNTwise
Copy link
Contributor

TNTwise commented Apr 21, 2024

With what error?

@Selur
Copy link
Author

Selur commented Apr 21, 2024

No error message:

F:\Hybrid\64bit\Vapoursynth>VSPipe.exe c:\Users\Selur\Desktop\test_1.vpy -c y4m NUL --progress
Script evaluation done in 0.43 seconds
[0 NVIDIA GeForce RTX 4080]  queueC=2[8]  queueG=0[16]  queueT=1[2]
[0 NVIDIA GeForce RTX 4080]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[0 NVIDIA GeForce RTX 4080]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[0 NVIDIA GeForce RTX 4080]  subgroup=32  basic/vote/ballot/shuffle=1/1/1/1
[0 NVIDIA GeForce RTX 4080]  fp16-matrix-16_8_8/16_8_16/16_16_16=1/1/1
[1 AMD Radeon(TM) Graphics]  queueC=1[2]  queueG=0[1]  queueT=2[1]
[1 AMD Radeon(TM) Graphics]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[1 AMD Radeon(TM) Graphics]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[1 AMD Radeon(TM) Graphics]  subgroup=64  basic/vote/ballot/shuffle=1/1/1/1
[1 AMD Radeon(TM) Graphics]  fp16-matrix-16_8_8/16_8_16/16_16_16=0/0/0
[2 Intel(R) Arc(TM) A380 Graphics]  queueC=1[1]  queueG=0[1]  queueT=2[1]
[2 Intel(R) Arc(TM) A380 Graphics]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[2 Intel(R) Arc(TM) A380 Graphics]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[2 Intel(R) Arc(TM) A380 Graphics]  subgroup=32  basic/vote/ballot/shuffle=1/1/1/1
[2 Intel(R) Arc(TM) A380 Graphics]  fp16-matrix-16_8_8/16_8_16/16_16_16=0/0/0
[0 NVIDIA GeForce RTX 4080]  queueC=2[8]  queueG=0[16]  queueT=1[2]
[0 NVIDIA GeForce RTX 4080]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[0 NVIDIA GeForce RTX 4080]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[0 NVIDIA GeForce RTX 4080]  subgroup=32  basic/vote/ballot/shuffle=1/1/1/1
[0 NVIDIA GeForce RTX 4080]  fp16-matrix-16_8_8/16_8_16/16_16_16=1/1/1
[1 AMD Radeon(TM) Graphics]  queueC=1[2]  queueG=0[1]  queueT=2[1]
[1 AMD Radeon(TM) Graphics]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[1 AMD Radeon(TM) Graphics]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[1 AMD Radeon(TM) Graphics]  subgroup=64  basic/vote/ballot/shuffle=1/1/1/1
[1 AMD Radeon(TM) Graphics]  fp16-matrix-16_8_8/16_8_16/16_16_16=0/0/0
[2 Intel(R) Arc(TM) A380 Graphics]  queueC=1[1]  queueG=0[1]  queueT=2[1]
[2 Intel(R) Arc(TM) A380 Graphics]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[2 Intel(R) Arc(TM) A380 Graphics]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
[2 Intel(R) Arc(TM) A380 Graphics]  subgroup=32  basic/vote/ballot/shuffle=1/1/1/1
[2 Intel(R) Arc(TM) A380 Graphics]  fp16-matrix-16_8_8/16_8_16/16_16_16=0/0/0

just closes,...
script I used:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
#core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/FrameFilter/RIFE/librife_r9_mod_v6.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/FrameFilter/RIFE/librife_windows_x86-64.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/DGDecNV/DGDecodeNV.dll")
# Source: 'G:\clips\duplicate frames\duplicate frames.mkv'
# Current color space: YUV420P8, bit depth: 8, resolution: 1920x1080, frame rate: 24fps, scanorder: progressive, yuv luminance scale: limited, matrix: 709
# Loading G:\clips\duplicate frames\duplicate frames.mkv using DGSource
clip = core.dgdecodenv.DGSource("J:/tmp/mkv_25bb11a27d290a7f1cab5f76c3573145_853323747.dgi")# 24 fps, scanorder: progressive
frame = clip.get_frame(0)
# Setting detected color matrix (709).
clip = core.std.SetFrameProps(clip, _Matrix=1)
# setting color transfer (709), if it is not set.
if '_Transfer' not in frame.props or not frame.props['_Transfer']:
  clip = core.std.SetFrameProps(clip, _Transfer=1)
# setting color primaries info (to 709), if it is not set.
if '_Primaries' not in frame.props or not frame.props['_Primaries']:
  clip = core.std.SetFrameProps(clip, _Primaries=1)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 24fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24, fpsden=1)
# making sure the detected scan type is set
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive
# adjusting color space from YUV420P8 to RGBS for vsFillDuplicateFrames
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="709", range_s="limited")
from FillDuplicateFrames import FillDuplicateFrames
fdf = FillDuplicateFrames(clip=clip, method="RIFE", debug=True)
clip = fdf.out
# adjusting output color from: RGBS to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="709", range_s="limited", dither_type="error_diffusion")
# set output frame rate to 24fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=24, fpsden=1)
# output
clip.set_output()

@TNTwise
Copy link
Contributor

TNTwise commented Apr 21, 2024

I'm updating ncnn right now, that might fix it but I don't know at this point.

@Selur
Copy link
Author

Selur commented Apr 21, 2024 •
edited
Loading

Did call it again now I additionally at the end got:

ERROR:             vkDestroyBuffer: Invalid device [VUID-vkDestroyBuffer-device-parameter]

(next few times calling the script again, this does not appear and the script just closed without an error message)

@TNTwise
Copy link
Contributor

TNTwise commented Apr 21, 2024

See if this works....
Idk how to reproduce so I'm kind of just guessing at this point... Sorry https://github.com/TNTwise/VapourSynth-RIFE-ncnn-Vulkan/releases/tag/V2

@Selur
Copy link
Author

Selur commented Apr 21, 2024

Nope, still the same.
https://github.com/Selur/VapoursynthScriptsInHybrid/blob/master/FillDuplicateFrames.py
is the FillDuplicateFrames scripts I use with:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# loading plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/FrameFilter/RIFE/librife_windows_x86-64.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/DGDecNV/DGDecodeNV.dll")
# Source: 'G:\clips\duplicate frames\duplicate frames.mkv'
# Current color space: YUV420P8, bit depth: 8, resolution: 1920x1080, frame rate: 24fps, scanorder: progressive, yuv luminance scale: limited, matrix: 709
# Loading G:\clips\duplicate frames\duplicate frames.mkv using DGSource
clip = core.dgdecodenv.DGSource("J:/tmp/mkv_25bb11a27d290a7f1cab5f76c3573145_853323747.dgi")# 24 fps, scanorder: progressive
frame = clip.get_frame(0)
# Setting detected color matrix (709).
clip = core.std.SetFrameProps(clip, _Matrix=1)
# setting color transfer (709), if it is not set.
if '_Transfer' not in frame.props or not frame.props['_Transfer']:
  clip = core.std.SetFrameProps(clip, _Transfer=1)
# setting color primaries info (to 709), if it is not set.
if '_Primaries' not in frame.props or not frame.props['_Primaries']:
  clip = core.std.SetFrameProps(clip, _Primaries=1)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 24fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24, fpsden=1)
# making sure the detected scan type is set
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive
# adjusting color space from YUV420P8 to RGBS for vsFillDuplicateFrames
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="709", range_s="limited")
from FillDuplicateFrames import FillDuplicateFrames
fdf = FillDuplicateFrames(clip=clip, method="RIFE", debug=True)
clip = fdf.out
# adjusting output color from: RGBS to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="709", range_s="limited", dither_type="error_diffusion")
# set output frame rate to 24fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=24, fpsden=1)
# output
clip.set_output()

here is a link to my source file (I usually check around frame 450+): https://drive.google.com/file/d/1HsbgMlJ8iNzGYdWrEuKsYpdvrn4Jvj0m/view?usp=drive_link
I then call VSPipe.exe c:\Users\Selur\Desktop\test_1.vpy -c y4m NUL --progress`

Maybe that helps to reproduce the problem.
I'm using a Geforce RTX 4080 with 552.22 studio drivers.

@TNTwise
Copy link
Contributor

TNTwise commented Apr 21, 2024

I can try a few things, but imo you should just use the latest vapoursynth that works and just use model_path to load whatever rife model you want. This will work with all of them except rife 4.14 lite.

@Selur
Copy link
Author

Selur commented Apr 21, 2024

Strange thing is that when I use RIFE just normally it works. (all versions)
only if I use it with FrameEval and only on some frames it seems to crash.
-> will try to do some more testing tomorrow, going to bed now :)

@Selur
Copy link
Author

Selur commented Apr 22, 2024 •
edited
Loading

Okay, I think I found the culprit and it isn't RIFE. It's the combination of RIFE and misc.SCDetect.
Moving misc.SCDetect out of the FrameEval seems to fix the problem.
Thanks again, for looking into it.
(strangely, it only works with models <= 4.6, but thats enough for my scripts)

@Selur Selur closed this as completed Apr 22, 2024
@Selur
Copy link
Author

Selur commented May 7, 2024

Sadly I was wrong, issue is still there and happens if I replace just 15, 17, 19. My guess is that is somehow related to loading and unloading RIFE or something similar. :/ (v6 works fine, everything after it not)

@Selur Selur reopened this May 7, 2024
@TNTwise
Copy link
Contributor

TNTwise commented May 7, 2024

Until this is fixed, I would just use model_path with a path to the model, every model in here will work except 4.14-lite. this is most likely a ncnn issue, but I will have to test more later.

@Selur
Copy link
Author

Selur commented Sep 8, 2024

might be related to AmusementClub/vs-mlrt#72

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Selur @styler00dollar @TNTwise @mafiosnik777