Desampling functions for Avisynth+, replaces DebilinearM and lineart_rpow2

Desample.avsi

/*Desample.avsi - v1.3

---Included script functions---
DesampleX: Wrapper for doing various things involving DesampleMT, deinterlacers, resizers, and masks
           Default behavior identical to DebilinearM, can also function as lineart_rpow2
DeCrossConversionMT: Replacement for ccc, ccc_720

---Requirements--- 
Avisynth+ r2455-MT or newer

Formats: YUV, Y
Plugins: masktools2-pfmod, ResampleMT
Plugins (DesampleX): RgTools-pfmod, Dither, nnedi3, nnedi3ocl, eedi2, eedi3
Scripts (DesampleX): edi_rpow2, ResizeX

---Parameters---
dw, dh                            | "Native" width & height
kernel                            | Desampling kernel. Takes string or integer. Mostly corresponds to the kernel strings for Dither Tools, plus "ccc" for cross-conversions
b, c, p, taps, cc                 | Parameters for Debicubic (b, c), Degauss (p), Delanczos/blackman/sinc (taps), and DeCrossConversion (cc)
                                  | Note that "c" will be calculated automatically based on "b" according to b+2c=1, if not supplied (i.e. 0/0.5, 0.2/0.4, 0.333/0.333)

ow, oh                            | Output width & height. Default is same as dw, dh
kerneld                           | Post-Desample/edi kernel. Mostly like Dither
a1d, a2d, tapsd                   | Parameters to go with kerneld
edi                               | Set deinterlacer for height doubling
ratiothr                          | If scaling factor >= ratiothr, use edi_rpow2 to upscale, otherwise scale normally. This was copied from nnedi3_resize16 so assume this works the same

^ these parameters can be abused to do antialiasing while still downscaling. 
ratiothr=0.5 will usually work right, don't set it to 0.000001 or it'll try to double the height a million or so times

kernely, kernelc                  | When output is resized, this controls the resizer(s) for the base clip
a1y, a2y, tapsy, a1c, a2c, tapsc  | Parameters for kernely/kernelc
yuv444                            | Scale chroma to 4:4:4 - this does nothing when output and input resolutions are the same. Use a boolean or set to "nnedi3" to upscale chroma with nnedi3_rpow2 (don't use nnedi3 if your input is float!)

edgemask                          | Enable or disable edge masking. Default: enabled only when edi_rpow2 is used
thr                               | Cutoff for the absolute error of descaling -> rescaling for a pixel to be considered distorted and included in protection mask. Set to 0 to disable
expand, inflate                   | Iterations of mt_expand/inflate. mt_expand uses the pattern both/both/square for "mode"
paramscale                        | masktools2 parameter, applies to "thr". Basically, mt_binarize defaults to an 8 bit scale, regardless of input depth, unless you change this
rgmode                            | Which removegrain mode to use to soften the line mask
blur_first                        | Whether to use removegrain on the line mask before or after subtracting the error mask
exmask                            | External edgemask string/clip. Renders rgmode and blur_first moot. Error mask is still subtracted from this clip
                                  | i.e. a clip: exmask=last.mt_edge("hprewitt") or a string: exmask="""des.mt_edge("hprewitt")""" - desampled clip is "des", or normal clip is "src" (needs ExtractY)
show                              | Display mask (0 = normal output, 1 = credits mask, 2 = full mask) Mask is overlayed 50% onto the output clip: set negative values for the raw mask clip

(DesampleMask)
ow, oh                            | Mask "output" width & height, for masking at a different resolution
dw, dh                            | Descaled clip width & height. You don't usually need to set this

---Notes---
Desample can have a sharpening effect on low detail areas; grain added after upscaling, blocking, noise, etc. It may be a good idea to enable the edgemask even when not using edi_rpow2 

Due to its very destructive nature, using an error mask with ccc can give a very high error, causing false positives which will be much more noticeable than usual. To deal with this, I would recommend scene-
filtering things like text overlays manually to only mask those scenes, and if possible limiting the mask to those black and white areas only, using the input clip's luma, DesampleMask with thr=0, and mt_lutxy

---Changelog---
v1.3.0 - DeCrossConversionMT actually works now
v1.2.2 - Set DeCrossConversionMT "order" default to 1 (vertical first)
v1.2.1 - Fix YV12 chroma center shift
v1.2.0 - Add nnedi3 YV12 -> YV24
v1.1.0 - Fixes and "exmask" enhancement
v1.0.0 - Initial release

*/

# TODO: saner controls over antialiasing features. maybe rfactor override
#       allow setting a second, lower thr value to smooth 0 - 255 from thr1 -> thr2
Function DesampleX(clip src, int "dw", int "dh", val "kernel", float "b", float "c", float "p", int "taps", float "cc", \
                   int "ow", int "oh", string "kerneld", float "a1d", float "a2d", int "tapsd", string "edi", float "ratiothr", \
                   string "kernely", float "a1y", float "a2y", int "tapsy", string "kernelc", float "a1c", float "a2c", int "tapsc", val "yuv444", \
                   bool "edgemask", float "thr", int "expand", int "inflate", string "paramscale", int "rgmode", bool "blur_first", val "exmask", int "show", \
                   int "nsize", int "nns", int "qual", int "etype", int "pscrn", int "edi_threads", int "opt", int "fapprox", \
                   float "alpha", float "beta", float "gamma", int "nrad", int "mdis", bool "hp", bool "ucubic", bool "cost3", \
                   int "vcheck", float "vthresh0", float "vthresh1", float "vthresh2", string "sclip", string "sclip_params", clip "mclip", \
                   int "mthresh", int "lthresh", int "vthresh", int "estr", int "dstr", int "maxd", int "map", int "nt", int "pp", bool "mt", string "mt_params", \
                   int "threads", bool "logicalCores", bool "MaxPhysCore", bool "SetAffinity", bool "sleep", int "prefetch", int "range", int "accuracy", int "order"){
                   # Row 1: Desample parameters
                   # Row 2: Post-Desample/edi resizing parameters
                   # Row 3: Base clip resizing parameters
                   # Row 4: masking parameters
                   # Rows 5-8: edi parameters
                   # Row 9: More Desample parameters
    
    assert(!src.IsRGB, "DesampleX: YUV and Y format only")
    
    bits = src.BitsPerComponent
    none = Undefined()
    
    ratiothr = Default(ratiothr, 1.125)
    
    iw = src.Width
    w = default(dw,1280)
    ow = default(ow, w)
    wratio = (float(ow) / float(w))/ratiothr
    rfactorX = wratio > 1 ? Round(Pow(2, Ceil( log(wratio) / log(2) ))) : 1
    
    ih = src.Height
    h = default(dh,720)
    oh = default(oh, h)
    hratio = (float(oh) / float(h))/ratiothr
    rfactorY = hratio > 1 ? Round(Pow(2, Ceil( log(hratio) / log(2) ))) : 1
    
    kernel = default(kernel, "bilinear")
    kernel = isint(kernel) ? select(kernel, "ccc", "bilinear", "bicubic", "lanczos", "spline16", "spline36", "spline64", "blackman", "gauss", "sinc") : kernel
    kerneld = default(kerneld, "spline36")
    kernely = default(kernely, "spline36")
    kernelc = default(kernelc, kernely)
    b = default(b, 1.0/3.0)
    c = default(c, (1.0-b)/2.0)
    p = default(p, 30.0)
    yuv444 = default(yuv444, ow%2==1 || oh%2==1)
    
    edi = default(edi, "nnedi3")
    
    edgemask = default(edgemask, edi!="" && (rfactorX>1 || rfactorY>1))
    thr = default(thr, 10)
    expand = default(expand, min(max((iw - 960)/320, (ih - 540)/180), 6))
    inflate = default(inflate, max(expand, 2))
    rgmode = default(rgmode, iw < 960 || ih < 720 ? 11 : 20)
    blur_first = default(blur_first, true)
    show = default(show,0)
    
    usemask = Defined(exmask) || edgemask || thr>0
    
    show!=0 ? assert(usemask, "DesampleX: Mask cannot be previewed while disabled") : NOP()
    
    srcy = src.ExtractY()
    
    src = iw==ow && ih==oh ? src : src.Resize(ow, oh, kernely, a1y, a2y, tapsy, kernelc, a1c, a2c, tapsc, yuv444, !usemask)
    
    down = srcy.Desample_internal(w, h, none, none, none, none, kernel, b, c, p, taps, cc, threads, logicalCores, MaxPhysCore, SetAffinity, sleep, prefetch, range, accuracy, order)
    errmask = thr>0 ? down.DesampleMask(srcy, ow, oh, w, h, thr, expand, inflate, kernel, b, c, p, taps, cc, paramscale) : NOP()
    
    (rfactorX<2 && rfactorY<2) || edi=="" ? Eval("""
        des = ow==w && oh==h ? down : down.Resize(ow, oh, kerneld, a1d, a2d, tapsd)
    """) : Eval("""
        des = bits>8 ? down.ConvertBits(8, dither=1) : down
        des = des.edi_rpow2(rfactorX, rfactorY, edi, kerneld+"resize", ow, oh, tapsd, a1d, a2d, none, true, false, false, bits>8, true, none, 
                           \nsize, nns, qual, etype, pscrn, edi_threads, opt, fapprox, alpha, beta, gamma, nrad, mdis, hp, ucubic, cost3, 
                           \vcheck, vthresh0, vthresh1, vthresh2, sclip, sclip_params, mclip, mthresh, lthresh, vthresh, estr, dstr, maxd, 
                           \map, nt, pp, mt, mt_params)
        des = bits>8 ? des.ConvertFromStacked(16) : des
        des = bits==8 || bits==16 ? des : des.ConvertBits(bits)
    """)
    
    Defined(exmask) ? Eval("""
        exmask = IsString(exmask) ? Eval(exmask) : exmask
        mask = thr>0 ? exmask.mt_lutxy(errmask, "x y -") : exmask
    """) : edgemask ? Eval("""
        lines = des.mt_edge("prewitt", 4, 24)
        
        lines = (kernel=="bicubic" && c > 0.5) || kernel=="lanczos" ? lines.mt_expand() : lines
        lines = kernel=="blackman" || kernel=="spline16" || kernel=="spline36" || kernel=="spline64" ? lines.mt_inflate() : lines
        lines = kernel=="sinc" ? lines.mt_expand().mt_expand().mt_inflate().mt_inflate() : lines
        
        lines = blur_first ? lines.removegrain(rgmode) : lines
        
        mask = thr>0 ? blur_first || inflate>0 ? lines.mt_lutxy(errmask, "x y -") : errmask.mt_logic(lines, "andn") : lines
        mask = blur_first ? mask : mask.removegrain(rgmode)
    """) : thr > 0 ? Eval("""
        mask = errmask
    """) : NOP()
    
    return usemask ? show>0 ? mt_merge(src, src.BlankClip(color=$35FF11), show==1?errmask.clamp():mask.clamp(), luma=true)
    \                       : show<0 ? show==-1 ? errmask : mask
    \                                : edgemask || Defined(exmask) ? mt_merge(src, des, mask)
    \                                                              : mt_merge(des.csp(src), src, mask, u=4, v=4)
    \              : src.IsYUV ? CombinePlanes(des, src, planes="YUV", source_planes="YUV", sample_clip=src) : des
    
    Function clamp(clip clip){clip.mt_clamp(clip.BlankClip(color=$808080), clip.BlankClip(), overshoot=0, undershoot=0)}
    Function csp(clip clip, clip ref){return ref.IsY ? clip : ref.Is420 ? clip.converttoyuv420() : ref.Is422 ? clip.converttoyuv422() : ref.Is444 ? clip.converttoyuv444() : clip.converttoyv411()}
    Function Desample_internal(clip src, int w, int h, float "src_left", float "src_top", float "src_width", float "src_height", string "kernel", float "b", float "c", float "p", int "taps", float "cc", \
                      int "threads", bool "logicalCores", bool "MaxPhysCore", bool "SetAffinity", bool "sleep", int "prefetch", int "range", int "accuracy", int "order"){
        src
        return kernel=="point"                     ? pointresize(w, h, src_left, src_top, src_width, src_height)
        \ : kernel=="bilinear" || kernel=="linear" ? DeBilinearResizeMT(w, h, src_left, src_top, src_width, src_height, threads, logicalCores, MaxPhysCore, SetAffinity, sleep, prefetch, range, accuracy, order)
        \ : kernel=="bicubic" || kernel=="cubic"   ? DeBicubicResizeMT(w, h, b, c, src_left, src_top, src_width, src_height, threads, logicalCores, MaxPhysCore, SetAffinity, sleep, prefetch, range, accuracy, order)
        \ : kernel=="lanczos"                      ? DeLanczosResizeMT(w, h, src_left, src_top, src_width, src_height, taps, threads, logicalCores, MaxPhysCore, SetAffinity, sleep, prefetch, range, accuracy, order)
        \ : kernel=="lanczos4"                     ? DeLanczos4ResizeMT(w, h, src_left, src_top, src_width, src_height, threads, logicalCores, MaxPhysCore, SetAffinity, sleep, prefetch, range, accuracy, order)
        \ : kernel=="spline16"                     ? DeSpline16ResizeMT(w, h, src_left, src_top, src_width, src_height, threads, logicalCores, MaxPhysCore, SetAffinity, sleep, prefetch, range, accuracy, order)
        \ : kernel=="spline36"                     ? DeSpline36ResizeMT(w, h, src_left, src_top, src_width, src_height, threads, logicalCores, MaxPhysCore, SetAffinity, sleep, prefetch, range, accuracy, order)
        \ : kernel=="spline64"                     ? DeSpline64ResizeMT(w, h, src_left, src_top, src_width, src_height, threads, logicalCores, MaxPhysCore, SetAffinity, sleep, prefetch, range, accuracy, order)
        \ : kernel=="blackman"                     ? DeBlackmanResizeMT(w, h, src_left, src_top, src_width, src_height, taps, threads, logicalCores, MaxPhysCore, SetAffinity, sleep, prefetch, range, accuracy, order)
        \ : kernel=="gauss" || kernel=="gaussian"  ? DeGaussResizeMT(w, h, src_left, src_top, src_width, src_height, p, threads, logicalCores, MaxPhysCore, SetAffinity, sleep, prefetch, range, accuracy, order)
        \ : kernel=="sinc"                         ? DeSincResizeMT(w, h, src_left, src_top, src_width, src_height, taps, threads, logicalCores, MaxPhysCore, SetAffinity, sleep, prefetch, range, accuracy, order)
        \ : kernel=="ccc"                          ? DeCrossConversionMT(w, h, cc, taps, threads, logicalCores, MaxPhysCore, SetAffinity, sleep, prefetch, range, accuracy, order)
        \                                          : assert(false, "DesampleX: Invalid kernel setting")
        }
    }

# TODO: Implement option for luma-matching to catch only black/white credits
#       see if abusing floats when binarizing and downscaling the mask works in Avisynth
#       Add src_left/top
Function DesampleMask(clip descaled, clip input, int "ow", int "oh", float "dw", float "dh", float "thr", int "expand", int "inflate", val "kernel", float "b", float "c", float "p", int "taps", float "cc", string "paramscale"){
    iw = input.Width
    ih = input.Height
    dw = default(dw, descaled.Width)
    dh = default(dh, descaled.Height)
    ow = default(ow, round(dw))
    oh = default(oh, round(dh))
    thr = default(thr, 10)
    expand = default(expand, min(max((iw - 960)/320, (ih - 540)/180), 6))
    inflate = default(inflate, max(expand, 2))
    kernel = default(kernel, "bilinear")
    b = default(b, 1.0/3.0)
    c = default(c, (1.0-b)/2.0)
    paramscale = default(paramscale, "i8")
    
    kernel = isint(kernel) ? select(kernel, "ccc", "bilinear", "bicubic", "lanczos", "spline16", "spline36", "spline64", "blackman", "gauss", "sinc") : kernel
    
    resc = descaled.DeDesample(iw, ih, dw, dh, kernel, b, c, p, taps, cc)
    
    error = mt_lutxy(input, resc, "x y - abs", chroma="process")
    
    mask = thr>0 ? error.mt_binarize(thr, chroma="process", paramscale=paramscale) : error
    mask = ow==iw && oh==ih ? mask : mask.bilinearresize(ow, oh).mt_binarize(3, chroma="process")
    mask = expand>0 ? mask.expand_rec(expand) : mask
    mask = inflate>0 ? mask.inflate_rec(inflate) : mask
    
    return mask
    
    Function expand_rec(clip clp, int iter){return iter==0 ? clp : clp.mt_expand(chroma="process", mode=iter%3==0?"square":"both").expand_rec(iter-1)}
    Function inflate_rec(clip clp, int iter){return iter==0 ? clp : clp.mt_inflate(chroma="process").inflate_rec(iter-1)}
    Function DeDesample(clip src, int iw, int ih, float dw, float dh, string "kernel", float "b", float "c", float "p", int "taps", float "cc"){
        src
        return kernel=="point"                        ? pointresize(iw, ih, 0, 0, dw, dh)
        \    : kernel=="bilinear" || kernel=="linear" ? bilinearresize(iw, ih, 0, 0, dw, dh)
        \    : kernel=="bicubic" || kernel=="cubic"   ? bicubicresize(iw, ih, b, c, 0, 0, dw, dh)
        \    : kernel=="lanczos"                      ? lanczosresize(iw, ih, 0, 0, dw, dh, taps)
        \    : kernel=="lanczos4"                     ? lanczos4resize(iw, ih, 0, 0, dw, dh)
        \    : kernel=="spline16"                     ? spline16resize(iw, ih, 0, 0, dw, dh)
        \    : kernel=="spline36"                     ? spline36resize(iw, ih, 0, 0, dw, dh)
        \    : kernel=="spline64"                     ? spline64resize(iw, ih, 0, 0, dw, dh)
        \    : kernel=="blackman"                     ? blackmanresize(iw, ih, 0, 0, dw, dh, taps)
        \    : kernel=="gauss" || kernel=="gaussian"  ? gaussresize(iw, ih, 0, 0, dw, dh, p)
        \    : kernel=="sinc"                         ? sincresize(iw, ih, 0, 0, dw, dh, taps)
        \    : kernel=="ccc"                          ? crossconversion(iw, ih, cc)
        \                                             : assert(false, "DesampleMask: invalid kernel")
        }
    }


# Unlike CCC, this doesn't scale chroma normally, so you'll have to do that yourself
Function DeCrossConversionMT(clip src, int "target_width", int "target_height", float "cc", \
                    int "threads", bool "logicalCores", bool "MaxPhysCore", bool "SetAffinity", \
                    bool "sleep", int "prefetch", int "range", int "accuracy", int "order"){
    
    target_width = default(target_width, 1280)
    target_height = default(target_height, 720)
    order = default(order, 1)
    cc = default(cc, 0.0)
    
    assert(target_height % 2 == 0, "DeCrossConversionMT: Height must be even")
    
    fh = float(target_height/2)
    sfac = float(target_height) / float(src.Height)
    cc = -cc * sfac
    
    sf = src.separatefields()
    
    e = sf.selecteven().DeBilinearResizeMT(target_width, target_height/2, src_top=-cc, src_height=fh-cc, \
                                          threads=threads, logicalCores=logicalCores, MaxPhysCore=MaxPhysCore, SetAffinity=SetAffinity, \
                                          sleep=sleep, prefetch=prefetch, range=range, accuracy=accuracy, order=order)
    o = sf.selectodd().DeBilinearResizeMT(target_width, target_height/2, src_top=cc, src_height=fh+cc, \
                                          threads=threads, logicalCores=logicalCores, MaxPhysCore=MaxPhysCore, SetAffinity=SetAffinity, \
                                          sleep=sleep, prefetch=prefetch, range=range, accuracy=accuracy, order=order)
    return interleave(e,o).weave()
    }

Function crossconversion(clip src, int w, int h, float cc){
    src.separatefields()
    fh = float(last.Height)
    sfac = (fh * 2.) / float(h)
    cc = -cc * sfac
    e = selecteven().bilinearresize(w, h/2, src_top=-cc, src_height=fh-cc)
    o = selectodd().bilinearresize(w, h/2, src_top=cc, src_height=fh+cc)
    return interleave(e, o).weave()
    }
    
Function Resize(clip src, int "w", int "h", string "kernely", float "a1y", float "a2y", int "tapsy", string "kernelc", float "a1c", float "a2c", int "tapsc", val "yuv444", bool "trashluma"){
    bits = src.BitsPerComponent
    
    yuv444 = Default(yuv444, false)
    trashluma = Default(trashluma, false)
    
    nnedi3_uv = IsString(yuv444)
    yuv444 = nnedi3_uv ? true : yuv444
    
    # love that trend of never adding plane parameters to core filters
    src8   = nnedi3_uv && bits>8 ? bits==16 ? src.ConvertToStacked().DitherPost(mode=6, y=1)  
    \                                       : src.ConvertBits(8, dither=1)
    \                            : Undefined()
    u_clip = nnedi3_uv && bits>8 ? src8.ExtractU() : src.ExtractU()
    v_clip = nnedi3_uv && bits>8 ? src8.ExtractV() : src.ExtractV()
    
    yuv = src.IsYUV
    hss = src.IsYV411 ? 4 : src.Is420 || src.Is422 ? 2 : 1
    vss = src.Is420 ? 2 : 1
    
    cw = yuv444 ? w : w/hss
    ch = yuv444 ? h : h/vss
    sx = hss>1 ? yuv444 ? 0.25 : calign(src.Width, w, hss) : 0
    
    return trashluma ? yuv ? CombinePlanes(u_clip.Resize_internal_uv(cw, ch, sx, kernelc, a1c, a2c, tapsc, nnedi3_uv, bits), 
                                          \v_clip.Resize_internal_uv(cw, ch, sx, kernelc, a1c, a2c, tapsc, nnedi3_uv, bits),
                                          \planes="UV", source_planes="YY", sample_clip=src)
    \                      : Undefined()
    \      : !yuv || hss==1 ? src.Resize_internal(w, h, 0,  0, kernely, a1y, a2y, tapsy)
    \                       : CombinePlanes(src.ExtractY().Resize_internal(w, h,  0,  0, kernely, a1y, a2y, tapsy),
                                           \u_clip.Resize_internal_uv(cw, ch, sx, kernelc, a1c, a2c, tapsc, nnedi3_uv, bits), 
                                           \v_clip.Resize_internal_uv(cw, ch, sx, kernelc, a1c, a2c, tapsc, nnedi3_uv, bits),
                                           \planes="YUV", source_planes="YYY", sample_clip=src)
    
    Function calign(float in, float out, int hss){return hss==2 ? 0.25*(1.0-in/out) : 0.375*(1.0-in/out)}
    
    Function Resize_internal_uv(clip src, int w, int h, float sx, string "kernel", float "a1", float "a2", int "taps", bool "usennedi3", int "bits"){
        src
        usennedi3           ? nnedi3_rpow2(2)   : last
        usennedi3 && bits>8 ? ConvertBits(bits) : last
        Resize_internal(w, h, usennedi3 ? 0 : sx, usennedi3 ? -0.5 : 0, kernel, a1, a2, taps)
        }
    
    Function Resize_internal(clip src, int w, int h, float sx, float sy, string "kernel", float "a1", float "a2", int "taps"){
        src
        return kernel=="point"                        ? pointresize(w, h, sx, sy)
        \    : kernel=="bilinear" || kernel=="linear" ? bilinearresize(w, h, sx, sy)
        \    : kernel=="bicubic" || kernel=="cubic"   ? bicubicresize(w, h, a1, a2, sx, sy)
        \    : kernel=="lanczos"                      ? lanczosresize(w, h, sx, sy, taps=taps)
        \    : kernel=="lanczos4"                     ? lanczos4resize(w, h, sx, sy)
        \    : kernel=="spline16"                     ? spline16resize(w, h, sx, sy)
        \    : kernel=="spline36"                     ? spline36resize(w, h, sx, sy)
        \    : kernel=="spline64"                     ? spline64resize(w, h, sx, sy)
        \    : kernel=="blackman"                     ? blackmanresize(w, h, sx, sy, taps=taps)
        \    : kernel=="gauss" || kernel=="gaussian"  ? gaussresize(w, h, sx, sy, p=a1)
        \    : kernel=="sinc"                         ? sincresize(w, h, sx, sy, taps=taps)
        \                                             : assert(false, "DesampleX: invalid kernely or kernelc")
        }
    }