1
0
mirror of https://git.ffmpeg.org/ffmpeg.git synced 2026-10-24 02:09:03 +03:00
Commit Graph

12878 Commits

Author SHA1 Message Date
zhang xingxing
ddb610f9fe avfilter/qrencode: free the rendered width expressions
This is not a security issue.

Fixes: oFKoC1YimRPu
Fixes: swarm-Memory-leak-qr--text-and-pexpr-fields-not-freed-in-uninit-5cc723

Found-by: zhang xingxing <2388969553@qq.com>

Memory leak Replicated through UnModified FFmpeg with LSAN

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2026-10-20 11:36:57 +00:00
Michael Niedermayer
341553de8a avfilter/dnn_backend_openvino: do not free the model when updating it fails
Fixes: double free
Fixes: QIOnvFKEz80N
Regression since: 22652b576c
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24617
Double free Replicated through UnModified FFmpeg with ASAN
2026-10-20 02:25:09 +00:00
Michael Niedermayer
21c4aad654 avfilter/dnn_backend_openvino: set the number of outputs together with the output ports
Fixes: NULL pointer dereference
Fixes: gJMHsT6GErl5
Regression since: 22652b576c
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24617
NULL pointer dereference Replicated through UnModified FFmpeg with ASAN
2026-10-20 02:25:09 +00:00
Michael Niedermayer
94e1680dbf avfilter/vf_drawbox: check the expression results before converting them to int
Fixes: out of range float to int conversion
Fixes: signed integer overflow
Fixes: 4jtp2TTOoEsQ
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24765
Out of range float to int conversion Replicated through UnModified FFmpeg with UBSAN
Signed integer overflow Replicated through UnModified FFmpeg with UBSAN
2026-10-20 02:25:09 +00:00
Michael Niedermayer
86b257b4da avfilter/motion_estimation: use separate linesizes for the current and reference frame
Fixes: out of array read
Fixes: Au41rSBM5kGS
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24765
Out of array read Replicated through UnModified FFmpeg with ASAN
2026-10-20 02:25:09 +00:00
Michael Niedermayer
6a025044c7 avfilter/vf_photosensitivity: blend only the visible part of the lines
Fixes: out of array read
Fixes: Au41rSBM5kGS
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24765
Out of array read Replicated through UnModified FFmpeg with ASAN
2026-10-20 02:25:09 +00:00
Michael Niedermayer
27f61c0ff5 avfilter/vf_idet: use the linesize of the previous and next frame
Fixes: out of array read
Fixes: Au41rSBM5kGS
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24765
Out of array read Replicated through UnModified FFmpeg with ASAN
2026-10-20 02:25:09 +00:00
Michael Niedermayer
5cd8e58b94 avfilter/vf_deshake: use the linesize of the reference frame
Fixes: out of array read
Fixes: Au41rSBM5kGS
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24765
Out of array read Replicated through UnModified FFmpeg with ASAN
2026-10-20 02:25:09 +00:00
Michael Niedermayer
f703292c42 avfilter/vf_libopencv: reject shapes too big for OpenCV
Fixes: out of array write
Fixes: EVA3Y4e693ul
Found during the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24753
Out of array write Replicated through UnModified FFmpeg with ASAN
2026-10-20 02:25:09 +00:00
Michael Niedermayer
c23f784451 avfilter/vidstabutils: do not offer packed RGB formats with vid.stab before 1.4.0
Fixes: assertion failure
Fixes: CSKmF1Dx1jig
Found during the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24608
Assertion failure Replicated through UnModified FFmpeg
Crash Replicated through UnModified FFmpeg with ASAN
2026-10-20 02:25:09 +00:00
Michael Niedermayer
8a729ff8ce avfilter/af_sofalizer: check the size of the delay data
Fixes: NULL pointer dereference
Fixes: d80uw1EIP3Pa
Regression since: ee64b64c02
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24624
NULL pointer dereference Replicated through UnModified FFmpeg with ASAN
2026-10-20 02:25:09 +00:00
Michael Niedermayer
d9c9bebe78 avfilter/af_sofalizer: check the IR length against the IR data
Fixes: out of array read
Fixes: 65nc8s5OIq4Z
Regression since: 4096c670ab
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24624
Out of array read Replicated through UnModified FFmpeg with ASAN
2026-10-20 02:25:09 +00:00
Michael Niedermayer
0f5ebb1eb6 avfilter/af_sofalizer: reject SOFA files with an IR length of 0
Fixes: undefined clz of 0
Fixes: jj9j0M05l3SP
Regression since: 7ea4b928a2
Found during the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24624
Undefined clz of 0 Replicated through UnModified FFmpeg with UBSAN
2026-10-20 02:25:09 +00:00
Vincent Herbst
7f2e5cdd7c avfilter/vf_yadif_videotoolbox: fix over-release of autoreleased Metal objects
call_kernel() obtains the command buffer and compute encoder from
property getters (mtlQueue.commandBuffer, buffer.computeCommandEncoder),
which return autoreleased references under the non-ARC build, and then
releases them via ff_objc_release(). That is an over-release: when the
enclosing autorelease pool is drained, the objects are released a second
time and the process crashes with EXC_BAD_ACCESS in objc_release.

On current macOS this is reproducible with the ffmpeg CLI itself. The
filter thread's thread-local autorelease pool is popped during pthread
TSD cleanup at thread exit, so any run that used yadif_videotoolbox
crashes when the filter thread terminates:

  ffmpeg -init_hw_device videotoolbox=vt -filter_hw_device vt \
    -f lavfi -i testsrc2=size=640x480:rate=30 \
    -vf setfield=tff,format=nv12,hwupload,yadif_videotoolbox,\
  hwdownload,format=nv12 -frames:v 20 -f null -

  Thread 11 Crashed:: vf#0:0
  0 libobjc.A.dylib  objc_release + 16
  1 libobjc.A.dylib  AutoreleasePoolPage::releaseUntil(objc_object**) + 204
  2 libobjc.A.dylib  objc_autoreleasePoolPop + 244
  3 libobjc.A.dylib  objc_tls_direct_base<...>::dtor_(void*) + 156
  4 libsystem_pthread.dylib  _pthread_tsd_cleanup + 620
  5 libsystem_pthread.dylib  _pthread_exit + 84

Host applications hit the same crash earlier, for example when a
dispatch queue drains its last-resort pool after a work item that ran
the filter.

Wrap the kernel invocation in @autoreleasepool and drop the manual
releases, making the pool pop the single balanced release and draining
the Metal transients once per frame. The ff_objc_release() calls on the
s->mtl* members in do_uninit() stay: those hold owned references
obtained from new* methods.
2026-10-20 01:57:09 +00:00
Michael Niedermayer
38ac3a9237 avfilter/vf_codecview: compute the arrow head length in 64bit
Fixes: division by zero
Fixes: signed integer overflow
Fixes: NaN to int conversion
Fixes: 2qHh10XPbgiO
Fixes: swarm-Signed-integer-overflow-in-draw_arrow-length-calc-8f1f15
Division by zero Replicated through UnModified FFmpeg with ASAN
Signed integer overflow Replicated through UnModified FFmpeg with UBSAN
NaN to int conversion Replicated through UnModified FFmpeg with UBSAN

Found-by: zhang xingxing <2388969553@qq.com>
The report suggested doing the shift in a wider unsigned type.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2026-10-19 23:51:00 +00:00
Michael Niedermayer
3f00282939 avfilter/vf_stack: reject a grid with more than INT_MAX cells
Regression since: aa8905a1b1
NULL pointer dereference Replicated through UnModified FFmpeg with ASAN

Found during review of the fixes for the security report GIRN0tvt58WQ

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2026-10-19 23:51:00 +00:00
Michael Niedermayer
13133e0b32 avfilter: use ff_slice_pos() in the remaining slice threaded filters
This is a behavior preserving change for all non-overflowing cases.

Found during triage of the security report GIRN0tvt58WQ

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2026-10-19 23:51:00 +00:00
zhang xingxing
8a976f7275 avfilter/vf_stack: use ff_slice_pos() for the per-slice input range
Fixes: GIRN0tvt58WQ
Fixes: swarm-Signed-integer-overflow-in-process_slice-slice-computation-b29bec
Regression since: 631f7acc6c
Out of array read Replicated through UnModified FFmpeg with ASAN

Found-by: zhang xingxing <2388969553@qq.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2026-10-19 23:51:00 +00:00
Michael Niedermayer
63a9ce3921 avfilter/vf_pad_vaapi: do the x/y range check and centering in 64bit
Fixes: Ilayk1MzOixN
Fixes: swarm-Signed-integer-overflow-in-pad-x-y-bounds-check-499261
Regression since: 42eb10ecc6

Found-by: zhang xingxing <2388969553@qq.com>
The report suggested, among other options, widening the comparison.
The overflow in the centering was found during review of the fixes for the report.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2026-10-19 23:51:00 +00:00
Michael Niedermayer
12ca9b6890 avfilter/vf_pad: do the x/y range check and centering in 64bit
Fixes: Ilayk1MzOixN
Fixes: swarm-Signed-integer-overflow-in-pad-x-y-bounds-check-499261
Regression since: 57c3670896
Signed integer overflow Replicated through UnModified FFmpeg with UBSAN

Found-by: zhang xingxing <2388969553@qq.com>
The report suggested, among other options, widening the comparison.
The overflow in the centering was found during review of the fixes for the report.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2026-10-19 23:51:00 +00:00
zhang xingxing
eccf4fadf1 avfilter/vf_avgblur: compute max * area in 64bit
Fixes: out of array read
Fixes: BxsBdfl8050I
Fixes: swarm-Signed-integer-overflow-in-build_lut-slow-check-enables-heap-OOB-read-ba3309
Regression since: a7fc78c1a6
Out of array read Replicated through UnModified FFmpeg with ASAN

Found-by: zhang xingxing <2388969553@qq.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2026-10-19 23:51:00 +00:00
Steven Xiao
07ab463f9f avfilter/dnn: implement D3D12 DirectML zero-copy for ONNX Runtime backend
This commit enables the DNN video filter to run entirely on the GPU
for the full D3D12 pipeline (decode, AI inference, and encode)
without ever copying frames back to system memory, making it
significantly faster on D3D12 hardware.

Benchmarked the new zero-copy D3D12/DirectML DNN pipeline against
the existing CPU round-trip pipeline on three major GPU vendors; up
to 10x throughput improvement was measured across every vendor.

Add dnn_onnx_d3d12, a helper that binds the DirectML execution
provider to a D3D12VA input frame's own device and does NV12
texture<->tensor conversion on the GPU (CopyTextureRegion + compute
shaders), avoiding a host round-trip.

dnn_processing negotiates this automatically for D3D12 NV12 input
with device=dml. Software-frame input retains the host I/O path.
D3D12 input requires successful DirectML negotiation; unsupported
hardware-input configurations fail during configuration. Currently
limited to single-channel FLOAT/NCHW models; chroma bypasses the
model and is copied/upscaled on the GPU. At runtime, DirectML.dll
and d3dcompiler_47.dll must be loadable from the application
directory or System32.

Example usage:

  # Zero-copy pipeline
  ffmpeg.exe -y -loglevel info \
    -hwaccel d3d12va -hwaccel_output_format d3d12 \
    -i input-1920x1080.mp4 \
    -vf "dnn_processing=dnn_backend=onnx:device=dml:model=ecbsrnet_y_fused.onnx:input=input:output=output" \
    -c:v h264_d3d12va -bf 0 -c:a copy \
    output_2x_sr_zero_copy.mp4

  # CPU round-trip pipeline
  ffmpeg.exe -y -loglevel info \
    -init_hw_device d3d12va=dev -filter_hw_device dev -hwaccel d3d12va \
    -i input-1920x1080.mp4 \
    -vf "dnn_processing=dnn_backend=onnx:device=dml:model=ecbsrnet_y_fused.onnx:input=input:output=output,hwupload" \
    -c:v h264_d3d12va -bf 0 -c:a copy \
    output_2x_sr_round_trip.mp4

Signed-off-by: younengxiao <steven.xiao@amd.com>
2026-10-19 21:08:42 +00:00
dhalsimax
fd9a8f25a0 avfilter/vf_tonemap_opencl: zero the peak detection buffer
The kernel keeps its peak detection state in util_mem (frame index,
frame count, running totals of peak and average) and expects it to
start at zero. The buffer is created without a host pointer, so per the
OpenCL spec its contents are undefined. Intel's compute-runtime (NEO
24.35, legacy branch, on a Gen9 Apollo Lake) hands out recycled memory,
and the totals can start as garbage: the detected peak is then huge and
the whole stream comes out black (Y=16), or with the wrong brightness.
The same command gave black video every time on one film and a correct
picture on another; with the buffer zeroed both are correct.

Create the buffer from a zeroed array with CL_MEM_COPY_HOST_PTR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VB8Q6zwZGmj1CET8B15HLg
2026-10-19 14:58:14 +00:00
romainhedouin
8dba37253f avfilter/vf_scale_vulkan: fix height check for debayering
The check compared the width twice so a height-only change was not
rejected when debayering.

Found during a code review sweep of ffmpeg using an LLM.

Signed-off-by: romainhedouin <romain.hedouin@epitech.eu>
2026-10-19 07:32:29 +00:00
zhang xingxing
5a2f87c190 avfilter/vf_libvmaf: reject a feature config without a name
This is not a security issue.

Fixes: 1QLxiyziAd9N
Fixes: swarm-NULL-feature_name-dereferenced-in-parse_features-129e69
Regression since: 3d29724c00
NULL feature name Replicated through UnModified FFmpeg (the error message read "(null)")
Found-by: zhang xingxing <2388969553@qq.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2026-10-18 22:47:35 +00:00
zhang xingxing
65ca8b1f50 avfilter/vf_crop: keep the x/y expressions when the new ones fail to parse
This is not a security issue.

Fixes: 3yfFBPXxhEts
Fixes: swarm-NULL-pexpr-dereference-after-failed-x-y-runtime-reconfiguration-e3e672
Regression since: d2b78fe6b7
NULL pointer dereference Replicated through UnModified FFmpeg with ASAN
Found-by: zhang xingxing <2388969553@qq.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2026-10-18 22:47:35 +00:00
Michael Niedermayer
9cdc4989f1 avfilter/vf_atadenoise: filter the 4th plane with its own thresholds and sigma
Fixes: NULL pointer dereference
Fixes: xIzHmB8ju8Nq
Fixes: swarm-NULL-dereference-on-YUVA-alpha-plane-filter-5f974b
Regression since: a50bd3a50e
NULL pointer dereference Replicated through UnModified FFmpeg with ASAN
Found-by: zhang xingxing <2388969553@qq.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2026-10-18 22:47:35 +00:00
zhang xingxing
62cf1e1de2 avfilter/vf_unsharp: initialize the alpha filter parameters
Fixes: NULL pointer dereference
Fixes: cRljeq44cXxS
Fixes: swarm-Null-deref-when-sharpening-alpha-plane-a2779b
Regression since: 84f5583078
NULL pointer dereference Replicated through UnModified FFmpeg with ASAN
Found-by: zhang xingxing <2388969553@qq.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2026-10-18 22:47:35 +00:00
Martin Mirchev
6076399974 avfilter/vf_colormap: fix single-patch coefficient index
With one patch, the per-channel gains were written to transposed
coefficient indices. Subsequent channel initialization cleared the first
two misplaced gains, while the third was stored in an unused column,
leaving the identity transform in place: mapping 0xA06040 to 0x503020
turned 0xA06040 into (160,97,63) instead of (80,48,32).

Store each gain at icoeff[1+c][c], matching the pixel loop and the
two-patch case.

Signed-off-by: Martin Mirchev <mmirchev@comp.nus.edu.sg>
2026-10-18 19:07:29 +00:00
Zhao Zhili
7045cc9b80 avfilter/vf_signature: fix inverted %d check in get_frame_filename()
The first %d was rejected instead of a second one.

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
2026-10-18 15:35:57 +00:00
Zhao Zhili
9d662af7ef avfilter/vf_signature: fix error check of get_frame_filename()
It returns a negative AVERROR code on failure, not -1.

Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
2026-10-18 15:35:57 +00:00
lusoris
ffb15c94c2 avfilter/Makefile: link cuda/load_helper.o into three CUDA filters
bilateral_cuda, bwdif_cuda and chromakey_cuda call ff_cuda_load_module(),
which lives in cuda/load_helper.c, but their OBJS lines do not pull that
object in; the other CUDA filters that load a module list it. A build
where one of these is the only CUDA filter fails to link, for example:

  vf_bwdif_cuda.c:312: undefined reference to `ff_cuda_load_module'

Reproduced for each filter with --disable-everything --enable-avfilter
--enable-filter=<name> --enable-cuda-llvm --enable-ffnvcodec; each links
after this change.

Signed-off-by: lusoris <lusoris@proton.me>
2026-10-18 14:52:35 +02:00
Sam.Richards@taurich.org
26a05be656 avfilter/vf_opencolorio: negotiate the output format properly
The filter picked its output format in config_props(), after format
negotiation had already settled the output link. The frames it emitted
could therefore disagree with the negotiated link format, for example
when a downstream zscale had negotiated something else.

Switch to FILTER_QUERY_FUNC2() and decide the output format during
negotiation instead:

- with format=, the output is constrained to that single format, which
  is now also validated in init() against the supported output list;
- otherwise, when no half-float candidate remains on the input, the
  output list is the input list, so the filter passes the format
  through and downstream constraints can still narrow the input;
- otherwise the output is restricted to float, and negotiation picks
  GBRPF32 or GBRAPF32 to match the input's alpha.

Half-float input has to be promoted because swscale cannot read or
write half-float in its new backend, and its legacy code clamps
half-float to [0,1], which would destroy values outside that range.
Half-float is therefore no longer offered as an output format.

The default output depends on the input format, so the query waits for
the input list to be merged with the upstream one, which it detects by
comparing it against the input link's other end. The merged list need
not be a singleton. Waiting for a singleton would reject valid graphs,
because a filter such as zscale offers half-float and integer formats
on its output independently of its input, so that list never narrows
to one of the two.

While the query is delayed it publishes the input color space, color
range, alpha mode and chroma location lists, because a converter
inserted on the input requires them, but the matching output lists are
only published together with the output pixel formats, as a converter
inserted on the output would otherwise find them without a format list
and crash.

Finally, supply opaque input alpha when the output format has alpha and
the input format does not. Prepend an alpha-initialization matrix to the
resolved OCIO processor, allowing OCIO to optimize it with the user's
transform. This avoids a separate per-frame alpha fill and ensures that
transforms which use alpha in RGB calculations or modify alpha receive
the opaque value before processing and retain their transformed alpha.
Filling output alpha after processing would instead leave alpha-dependent
RGB incorrect and overwrite alpha produced by the user's transform.

Note, this includes Fairys fairy/pr21799-replace patch (Thank-you Fairy!).

Signed-off-by: Sam Richards <Sam.Richards@taurich.org>
Assisted-by: Fairy
2026-10-18 00:45:18 +00:00
romainhedouin
32b9574ae4 avfilter/vf_drawbox: also hoist loop invariants in draw_region
Same as the previous commit, for the planar YUV path. Hoisting only the
alpha made 4:2:0 and 4:2:2 about 30% slower on arm64 with clang, so the
colour and subsampling reads are hoisted as well.

Output is identical: 1794 drawbox/drawgrid combinations, all 256 alpha
values in 15 pixel formats, and fate-filter-drawbox pass.

Full-frame fill, 1080p, arm64 with clang, CPU time: YUV 4:2:0 -13%,
YUV 4:4:4 -23%. Not measured on x86.

Found during a code review sweep of ffmpeg using an LLM.

Signed-off-by: romainhedouin <romain.hedouin@epitech.eu>
2026-10-17 18:41:36 +00:00
romainhedouin
6f7ca90701 avfilter/vf_drawbox: compute alpha once in draw_region_rgb_packed
alpha only depends on the box colour, so compute it once per call
instead of for every pixel. The output is unchanged.

About 10-25% less CPU on full-frame boxes and grids (rgb24/rgba,
1080p, arm64). Small boxes are not affected.

Found during a code review sweep of ffmpeg using an LLM.
2026-10-17 18:41:36 +00:00
romainhedouin
94b8431a55 avfilter/vf_ssim360: compute the barrel density once per row
The cos() value only depends on the row, but it was computed for every
pixel in the row. No change in output.

Found during a code review sweep of ffmpeg using an LLM.
2026-10-17 18:40:29 +00:00
romainhedouin
2193727b88 avfilter/vf_dblur: Copy float rows once instead of once per pixel
The float input and output paths called memcpy() for a whole row inside
the per-pixel loop, so every row was copied width times.

1080p gbrpf32le: 1.84 s -> 37 ms per frame on a single thread. Output is
unchanged.

Found during a code review sweep of ffmpeg using an LLM.

Signed-off-by: romainhedouin <romain.hedouin@epitech.eu>
2026-10-17 18:39:21 +00:00
Timo Rothenpieler
447b72d90f avfilter/vf_scale_cuda: support resizing during YUV<->RGB conversion
Resizing is done on the RGB side: YUV to RGB conversion converts at the
visible input size before scaling, RGB to YUV conversion converts after
scaling at the output size. Every scaling pass then works on a single
packed plane and reuses the generic RGB kernels, and the conversion
kernels are reused with a float RGB side.

The RGB intermediates hold normalized floats, so filter overshoot and
out of gamut colors are not clipped before the scaler rounds and clamps
once. As conversion and filtering are linear, the result matches
scaling on the YUV side, apart from rounding and the chroma upsampling
of the conversion kernels.

Resizing during conversion always uses the generic filters with the
selected interp_algo, with nearest using point sampling weights.
Conversion without resizing keeps using the direct conversion kernels.

Assisted-By: Claude Opus 5.5
2026-10-16 13:13:51 +00:00
Timo Rothenpieler
fcf1a35c4d avfilter/vf_scale_cuda: describe kernel passes independently
Each pass took its kernel, filter weights and planes from its position
in the chain, and the sample format parameters of its kernels from the
filter's input and output formats. Only the geometry came from the
buffers it reads and writes.

Describe every pass explicitly instead: its kernels, weights, written
plane groups, source and destination buffers, and the formats it reads
and writes. The fixed function and generic paths both build such a
plan, and processing a frame just runs its passes in order, passing
plane groups a pass does not write through from its source. This
makes adding passes that do not just read the input or write the
output, such as for color conversion, possible without special cases.

No functional change.

Assisted-By: Claude Opus 5.5
2026-10-16 13:13:51 +00:00
Zhao Zhili
9bbe8397e2 avfilter/vf_scale_cuda: add RGB to YUV color space conversion
The output YCbCr matrix and range are negotiated, and selectable via
the new out_color_matrix and out_range options. Without them, the
output prefers BT.601 (SMPTE 170M) limited range, matching swscale
behavior, while still following a single matrix and range negotiated
for the input, as buffersrc does for ordinary YUV. Since RGB input is
not constrained, RGB to YUV conversion defaults to BT.601 limited
range; RGB output prefers full range without forcing it, so that it
does not carry over into YUV output converted from it again. The
output frames are tagged with the negotiated properties, and YUV output
never carries the RGB tag of the input. As without a requested format
the options are negotiated before the output is known to be RGB, they
are rejected for RGB output.

Without RGB to YUV conversion, the output description must not
contradict that of the input, as the samples cannot be converted; an
unspecified input takes on the negotiated description, also when
frames are passed through.

Chroma is averaged over each subsampled block before clamping, high bit
depth output uses the code maximum for full range and keeps padding
bits clear, and limited range YCgCo uses the luma excursion for chroma.
The averaging centers the chroma samples, so with a requested format,
the chroma location is negotiated independently of the input, and
subsampled output must allow centered chroma.

Co-authored-by: Timo Rothenpieler <timo@rothenpieler.org>
Co-authored-by: Reinhard Nißl <rnissl@gmx.de>
Assisted-By: Claude Opus 5.5
2026-10-16 13:13:51 +00:00
Zhao Zhili
6ba18c7b7e avfilter/vf_scale_cuda: add YUV to RGB color space conversion
When the input colorspace is unspecified, BT.601 (SMPTE 170M) is used
as the default, matching swscale behavior. Both limited and full range
inputs are handled, with full range high bit depth input normalized by
the code maximum. BT.2020 constant luminance is rejected, as it is not
a plain matrix transform. Results are rounded to the nearest value.

The conversion kernels use direct texture reads for luma and simple
coordinate mapping for chroma upsampling, avoiding unnecessary
Subsample_Nearest overhead since color conversion operates at the same
resolution. Simultaneous resize and color conversion is rejected
with an error directing users to chain separate scale_cuda instances.
Hardware cropping is honored, by checking the visible frame size
against the output size per frame.

As the software format of CUDA frames is unknown during negotiation,
the input and output color properties are negotiated independently.
RGB output is always negotiated as full range RGB, so that following
filters see the properties of the converted frames, while other output
follows the input. Plain scaling rejects a matrix or range on the
output that differs from the input, as it cannot convert the samples.

Co-authored-by: Timo Rothenpieler <timo@rothenpieler.org>
Co-authored-by: Reinhard Nißl <rnissl@gmx.de>
Assisted-By: Claude Opus 5.5
2026-10-16 13:13:51 +00:00
Jake Fineman
3e6cd5e760 avfilter/dnn_io_proc: fix stack overflow in NCHW<->frame sws_scale calls
ff_proc_from_dnn_to_frame() and ff_proc_from_frame_to_dnn() each declare a
single `void *middle_data` local and, on the DL_NCHW layout branch, take its
address (`dst_data = &middle_data;` / `src_data = &middle_data;`) and pass
that as sws_scale()'s dst/src plane-array argument after casting to
`uint8_t *const *`. sws_scale()'s dst/src parameters are a 4-element array
(uint8_t *const dst[4]/src[4]), and scale_internal() copies all four
pointers (memcpy of srcSlice/dstSlice into src2/dst2) regardless of how many
planes the actual conversion uses. Since `&middle_data` only ever points at
one pointer-sized stack slot, that copy reads past it into whatever is next
on the stack: an out-of-bounds stack read on
every dnn_processing call whose model uses NCHW layout output or input into
an RGB24/BGR24 frame.

Fix both by giving dst_data/src_data real 4-element backing storage
(dst_data_buf/src_data_buf, zero-initialized) with the allocated buffer in
slot 0, instead of aliasing the single pointer.

Tested on arm64 macOS 27.0 (M1 Ultra), clang-asan toolchain, with
--enable-libopenvino (2026.4.0), --enable-libonnxruntime (1.30.0) and
--enable-libtensorflow, against unmodified upstream master (ff2059afd9) and
against this commit applied on top of the same base:

- Reproduced the stack-buffer-overflow on unmodified master two ways:
  - ff_proc_from_dnn_to_frame(): `ffmpeg -threads 1 -i puppies.ogv -an -vf
    "fps=2,format=rgb24,dnn_processing=dnn_backend=openvino:
    model=proc_valid_720.onnx:input=input:output=output" -f framemd5 out.md5`
    aborts under ASan on the first frame (offset-72 read into the
    `middle_data`/`planar_data` stack slots, from the OpenVINO async
    inference callback thread).
  - ff_proc_from_frame_to_dnn(): the same dnn_processing filter with two real
    ONNX Model Zoo fast-neural-style models (candy-9.onnx,
    sha256 9d11a3529d1e547da6ae07201d93484dbab2ec0a3614535752c8f40f0fe2968a;
    mosaic-9.onnx,
    sha256 fa646dedade881243f8d5a2ceb7de2b93675b21fc24f7482894ac4851a9a0a47;
    both NCHW 1x3x224x224 in/out) via dnn_backend=onnx aborts under ASan the
    same way, this time with the overflow located directly in
    ff_proc_from_frame_to_dnn's `middle_data` frame (offset 72), confirming
    the sibling instance flagged at review is independently live, not just
    structurally identical.
  With this commit applied, all three of the above runs are clean of the
  stack-buffer-overflow report (an unrelated, pre-existing OpenVINO/TBB
  at-exit SEGV under ASan remains on this machine/toolchain/OpenVINO version;
  it reproduces identically before and after this commit, and even on plain
  `ffmpeg -version` with no DNN filter at all, so it is unrelated to this fix
  and not addressed here).
- Whole-clip (puppies.ogv, 589 frames) framemd5 output from a release
  (non-ASan) build is byte-for-byte identical before/after this commit for:
  the NCHW OpenVINO identity model (proc_valid_720.onnx) into rgb24, and for
  both candy-9.onnx and mosaic-9.onnx into rgb24 via dnn_backend=onnx. This
  is expected: the aliasing read pulls garbage bytes that sws_scale never
  writes to a packed destination/never reads for these plane counts, so the
  visible output is unaffected; only ASan's redzone instrumentation makes the
  read observable.
- Unchanged paths are also byte-for-byte identical before/after: the NHWC
  TensorFlow path (tf_rgb_valid.pb into rgb24, which never takes the
  DL_NCHW/middle_data branch) and the planar/gray output path (tf_y_valid.pb
  into gray8, which uses frame->data directly and never allocates
  middle_data).
- `make fate-source` passes with this commit. This tree has no fate-dnn*
  tests to run.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Jake Fineman <jake@wave.online>
2026-10-16 01:49:26 +00:00
Jake Fineman
b2f8ebcd27 avfilter/dnn: reject channel-mismatched model output in dnn_processing
ff_proc_from_dnn_to_frame()'s AV_PIX_FMT_RGB24/BGR24 case assumes the
model's output has exactly 3 channels, but never checks it.

The first sws_scale() in that case reads from output->data with a
hardcoded source stride of frame->width * 3 * src_datatype_size over
frame->height rows, i.e. frame->width * 3 * frame->height *
src_datatype_size bytes. For a model whose output has fewer channels
than that -- e.g. the single-channel alpha matte an RGB-in
background-removal model produces -- the real tensor only holds
frame->width * frame->height * channels * src_datatype_size bytes, so
the call reads roughly 3x past the end of it. This affects both
DL_NCHW and DL_NHWC output, since it happens before the layout-specific
planar-to-packed branch.

For DL_NCHW there is a second, independent overflow on the write side:
middle_data is allocated plane_size * output->dims[1] bytes, but
linesize[0] is set to frame->width * 3 unconditionally, so the same
sws_scale() also writes past that allocation. This reproduces as a
SIGABRT (glibc "free(): invalid pointer", or a malloc.c assertion,
depending on allocator state) inside dnn_processing, after inference
has already completed successfully in the backend.

The input side is guarded -- check_modelinput_inlink() in
vf_dnn_processing.c -- but the output side is not. dnn_processing only
supports channel-preserving models for these pixel formats by design,
so reject a mismatched channel count with a clear error instead of
reading and writing out of bounds. The channel dimension is resolved
via dnn_get_channel_idx_by_layout() so the check covers both layouts.

This does not add support for rendering a 1- or 2-channel model output
through this filter; that would need a different pixel-format target
and is left as future work.

Signed-off-by: Jake Fineman <jake@wave.online>
2026-10-16 01:45:15 +00:00
Jake Fineman
459436357d avfilter/dnn: zero-initialize the output-probe task
get_output_tf() (dnn_backend_tf.c) and get_output_ov()
(dnn_backend_openvino.c) both declare `TaskItem task;` uninitialized
and then unconditionally free task.in_frame/task.out_frame on their
error path:

    ret = ff_dnn_fill_gettingoutput_task(&task, ...);
    if (ret != 0)
        goto err;
    ...
err:
    av_frame_free(&task.out_frame);
    av_frame_free(&task.in_frame);

ff_dnn_fill_gettingoutput_task() (dnn_backend_common.c) allocates
in_frame first and, only if that succeeds, allocates out_frame and
assigns both pointers into exec_params (and from there into the
task via ff_dnn_fill_task()). If the first av_frame_alloc() fails,
the function returns AVERROR(ENOMEM) immediately, before task.in_frame
or task.out_frame is ever set. The two probes then reach `err:` with
task's frame fields holding whatever garbage was on the stack, and
av_frame_free() dereferences those pointers.

Fix by zero-initializing the declaration, matching get_output_onnx()
(dnn_backend_onnx.c) and get_output_th() (dnn_backend_torch.cpp),
which already use `TaskItem task = { 0 };` and are not affected.

Reported by Forgejo_Fairy, review 8971 on #24761, which also
confirmed the defect predates that series and is already present at
6246f7a4cd.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Jake Fineman <jake@wave.online>
2026-10-16 01:43:20 +00:00
Michael Niedermayer
ec33c154fa avfilter/vf_yadif_cuda: Pass 0 or 1 as skip_spatial_check
The kernel takes a bool, but got the value 2 for the modes that skip
the spatial interlacing check, which with a cuda-llvm build was taken
as false, so modes 2 and 3 gave the output of modes 0 and 1.

Ignored mode Replicated through UnModified FFmpeg
Found during triage of the security report LQ8PDMaG6fL1
2026-10-14 17:59:13 +00:00
Michael Niedermayer
a6c7ff3f4e avfilter/yadif: cosmetics: Reindent after the previous commit 2026-10-14 17:59:13 +00:00
Michael Niedermayer
2b99c068db avfilter/yadif: propagate errorsl
Not a security issue. Not tested on VideoToolbox.

Unprocessed output frames Replicated through UnModified FFmpeg with CUDA and Vulkan call failures injected by gdb
Found-by: Joshua Rogers <joshua.rogers@aisle.com>
Suggested-by: Joshua Rogers <joshua.rogers@aisle.com>
2026-10-14 17:59:13 +00:00
Ramiro Polla
ef72fa74c0 avfilter/af_lv2: support plugins built with DPF
Plugins built with DPF require the LV2 options feature to be passed to
lilv_plugin_instantiate(), otherwise they fail to instantiate with the
error:
Options feature missing, cannot continue!

Only the maxBlockLength option seems to be necessary.

Tested with all plugins from dpf-plugins-lv2 from Debian 13.

Fixes #24272.

Reported-by: hellocatfood <hellocatfoodsocial@gmail.com>
2026-10-12 18:00:45 +00:00
Ramiro Polla
a1a95df500 avfilter/af_lv2: move the bounded block check earlier in config_output()
This will simplify the following commit.
2026-10-12 18:00:45 +00:00
Ramiro Polla
4f09194ac2 avfilter/af_lv2: NULL-terminate array of features the host supports
The documentation for lilv_plugin_instantiate() specifies that the array
of features the host supports should be NULL-terminated.
2026-10-12 18:00:45 +00:00