VMAF and SD Footage

[e-from-ct]

I was wondering if you could provide further guidance with VMAF scores and SD footage. We are digitizing old beta tapes to ProRes 422 interlaced, bottom field first. I've used Claude and ChatGPT to create a script that takes the .mov file, runs ab-av1 on it to find the optimal CRF, then takes that CRF and passes it to an ffmpeg encode.

I have min-vmaf set to 70 because the footage is SD and Netflix told Jan Ozer SD video score between 40-70. However, based on other threads here, I think ab-av1 may default "scale" the SD, 720x480 interlaced footage to 1080? Is that correct or do I need to set --vmaf-scale? If ab-av1 defaults to scale, then I need to keep min-vmaf to 95, yes?

Thanks, code currently in script is below.

crf-search --encoder libx264 --min-vmaf 95 --pix-format yuv420p10le -i

ffmpeg -y -nostdin -hwaccel auto \ -i "$input_file" \ -c:v libx264 \ -pix_fmt yuv422p10le \ -flags +ilme+ildct \ -field_order bt \ -crf "$crf_value" \ -preset slower \ -profile:v high422 \ -level:v 5.1 \ -x264-params "weightp=0:bff=1" \ -c:a copy \ -map 0 \ "$output_file"

[veikk0]

VMAF isn't great for targeting visually lossless results. Its strengths lie in evaluating low-to-good quality video (the quality range used in streaming video over the internet, basically).

I'd recommend trying XPSNR instead for your use case. I'm not all that familiar with it (only began using it when ab-av1 added support a month ago), but according to this wiki, 42 should be visually lossless. So start with --min-xpsnr 42, see how it looks compared to the source, and try lower and lower values until you land in a value that looks good enough for you.

If you end up using VMAF, I'd suggest using the same process to find a value that suits your needs. Since your use-case is fairly specific, I doubt there's anyone who can tell you exactly what to do. Trying things out is the best way of doing things in this field anyway.

As for scaling, according to Netflix themselves, upscaling lower-than-1080p footage to 1080p is the correct way to calculate VMAF (assuming you're using the default model, which was trained on 1080p footage and a viewing distance of three times the screen height).

ab-av1 does this automatically and uses the default 1080p model for any video that's a certain amount below 1080p in resolution (you can find the exact amount in the --help output, but VHS is certainly below that).

[e-from-ct]

Thanks, that is very helpful. I appreciate the full response. I'm not familiar with XPSNR but will look into it.

After further trial and error it appears like running --vfilter "scale=1440:1080:flags=bicubic,pad=1920:1080:240:0:black" --min-vmaf 95 produced a CRF value that created 720x480 files with bits/pixel/frame between .1 and .3.

I haven't been able to visually confirm yet as I've been just remoting into the machine, but by the numbers I'd be happy with that. Thanks

[stderr-to-devnull]

VMAF isn't great for targeting visually lossless results. Its strengths lie in evaluating low-to-good quality video (the quality range used in streaming video over the internet, basically).

Citation needed here, based on what analysis/tests is this concluded?

I'd recommend trying XPSNR instead for your use case. I'm not all that familiar with it (only began using it when ab-av1 added support a month ago), but according to this wiki, 42 should be visually lossless. So start with --min-xpsnr 42, see how it looks compared to the source, and try lower and lower values until you land in a value that looks good enough for you.

From the very same wiki, we read:

XPSNR uses a PSNR-like scoring system: a logarithmic scale in range 0..inf. This is considerably harder to interpret than other metrics (e.g. SSIMULACRA 2 or VMAF); and the threshold to what constitutes good quality is content-dependent. (Preliminary testing seems to indicate that anything above a XPSNR score of 42.00 is visually lossless.)

So XPSNR uses a logarithmic scale in range 0..inf which is considerably harder to interpret than other metrics and furthermore the threshold to what constitutes good quality is content-dependent... which means it is very difficult to establish a good baseline value for a specific quality level.

The 42.00 value cited is based on some Preliminary testing which seems to indicate things. Preliminary and seems are keywords here.

So from the information in that wiki, XPSNR cannot be used reliably using a baseline quality score value that would provide repeatable results. This can infer that using 42.00 would most likely waste a lot of bitrate to attain that score.

[e-from-ct]

Update: I did try XPSNR, but no CRF value could be found at value 42, 32, or 30.

Once I tried a min-xpsnr value of 28 it found: crf 38.8 XPSNR 28.07.
Obviously way too high of a CRF value.

As of now I'm running
crf-search --encoder libx264 --vfilter "scale=1440:1080:flags=bicubic,pad=1920:1080:240:0:black" --min-vmaf 98 --pix-format yuv420p10le -i

I tried a min-vmaf value of 95, but I could actually see the difference between 95 and 98; 98, obviously, being visually very close to the source file. Fine details like letters or window panes were evident at 98, when they were blurry at 95.

[mr44er]

interlaced, bottom field first

Are you deinterlacing first or is it intended, that the material should be encoded that way?

[e-from-ct]

Purposely keeping it interlaced.