The whisper filter exposes segment timestamps and text in JSON but omits the language selected by whisper.cpp. Clients consuming multilingual transcripts cannot distinguish the language of each transcription chunk without detecting it again.
Read whisper_full_lang_id() after each successful transcription and include its language code in JSON segments when available. This leaves the configured language and audio-frame metadata unchanged. Document that the code applies to the whole chunk, not to individual segments.
Tested with a minimal --enable-whisper build, make fate (without external SAMPLES), and a local WAV transcription using a multilingual tiny model.
Signed-off-by: Stefan Singer <stefan.singer@quartr.se>
The nsamples_max doubling can overflow int for an accepted delay, causing an undersized RDFT and an out-of-bounds write during kernel generation. Compare in int64_t so excessive delays take the existing error path.
Add a FATE check for the reported delay.
Fixes: #24344
Reported-by: yijan4845
Assisted-by: Fairy
The frames a filter in front of hwupload_cuda produces are allocated
with av_hwframe_get_transfer_buffer(), so that the upload reads from
memory the device transfers directly from.
Assisted-by: Claude
Check the request before reading its fields and use a context-free log for
the NULL case. This makes the documented EINVAL return reachable.
Assisted-by: Fairy
Fixes: WDZyf7w7BNU1
Fixes: NULL pointer dereference
NULL dereference after a failed allocation Replicated through API with ASAN
Found during the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24765
Fixes: A3O7F4s10WXJ
Fixes: EINVAL on a format change
EINVAL on a format change Replicated through UnModified FFmpeg with a plain output diff
Found during the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24765
Fixes: MxIcMabVhhFR
Fixes: command without effect
Command without effect Replicated through UnModified FFmpeg with a plain output diff
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24672
Fixes: fZnlZKlJmMn0
Fixes: wrong release time and asc level after a command
Wrong release time and asc level after a command Replicated through UnModified FFmpeg with a plain output diff
Found during the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24672
The fate-filter-metadata-cropdetect1/2 references change as the scans
now stop at the edge free lines next to the box.
Fixes: s7zfKOCXo4MJ
Fixes: wrong crop area with mode=mvedges
Wrong crop area with mode=mvedges Replicated through UnModified FFmpeg with a plain output diff
Found during triage of the security report VoiiaDYyhdfz
Found again by the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24623
This is not a security issue.
Fixes: 6HSeekmc6mV7
Fixes: 8YwigyLd85Gn
Fixes: swarm-Memory-leak-of-input-output-frames-on-ff_dnn_execute_model-failure-3a5cbe
Fixes: memleak
Memory leak Replicated through UnModified FFmpeg with LSan
Found-by: zhang xingxing <2388969553@qq.com>
Fixes: O0Cm42dvrP4l
Fixes: NULL pointer dereference
NULL pointer dereference Replicated through UnModified FFmpeg with UBSAN and gdb
Found during the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24617
Fixes: nC3u8khGyx61
Fixes: hang on teardown
Hang on teardown Replicated through UnModified FFmpeg with gdb
Hang on teardown Replicated through UnModified FFmpeg with gdb, a crafted 10240x10240 PNG, input_resizable=1 and vm.overcommit_memory=2
Found during the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24617
Fixes: cE2BMN6svM0X
Fixes: success reported after a failure
Success reported after a failure Replicated through UnModified FFmpeg with LSan
Found during the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24617
Fixes: o7gnt6oof6Qd
Fixes: use after free
Fixes: double free
Heap use after free after an allocation failure Replicated through UnModified FFmpeg with ASAN, the allocation failure injected through an LD_PRELOAD shim
Double free Replicated through UnModified FFmpeg with gdb, a crafted 8192x8192 PNG, input_resizable=1 and vm.overcommit_memory=2
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24617
Fixes: XPtsDsDMo2NW
Fixes: pointer formed two past the mapping
Pointer formed two past the mapping Replicated through Modified FFmpeg with an assertion
Found during the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24753
Fixes: qchet66gqCBs
Fixes: memleak
Memleak Replicated through UnModified FFmpeg with LSan, LCEVC_SendDecoderBase() failing through an LD_PRELOAD shim
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24765
Fixes: lJDYwsOtMQXY
Fixes: EINVAL on every hardware frame
EINVAL on every hardware frame Replicated through UnModified FFmpeg with plain output
Found during the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24765
This is not a security issue.
Fixes: oFKoC1YimRPu
Fixes: swarm-Memory-leak-qr--text-and-pexpr-fields-not-freed-in-uninit-5cc723
Found-by: zhang xingxing <2388969553@qq.com>
Memory leak Replicated through UnModified FFmpeg with LSAN
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: NULL pointer dereference
Fixes: gJMHsT6GErl5
Regression since: 22652b576c
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24617
NULL pointer dereference Replicated through UnModified FFmpeg with ASAN
Fixes: out of range float to int conversion
Fixes: signed integer overflow
Fixes: 4jtp2TTOoEsQ
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24765
Out of range float to int conversion Replicated through UnModified FFmpeg with UBSAN
Signed integer overflow Replicated through UnModified FFmpeg with UBSAN
Fixes: out of array read
Fixes: Au41rSBM5kGS
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24765
Out of array read Replicated through UnModified FFmpeg with ASAN
Fixes: out of array read
Fixes: Au41rSBM5kGS
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24765
Out of array read Replicated through UnModified FFmpeg with ASAN
Fixes: out of array read
Fixes: Au41rSBM5kGS
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24765
Out of array read Replicated through UnModified FFmpeg with ASAN
Fixes: out of array read
Fixes: Au41rSBM5kGS
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24765
Out of array read Replicated through UnModified FFmpeg with ASAN
Fixes: assertion failure
Fixes: CSKmF1Dx1jig
Found during the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24608
Assertion failure Replicated through UnModified FFmpeg
Crash Replicated through UnModified FFmpeg with ASAN
Fixes: NULL pointer dereference
Fixes: d80uw1EIP3Pa
Regression since: ee64b64c02
Found during triage of the review of https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/24624
NULL pointer dereference Replicated through UnModified FFmpeg with ASAN
call_kernel() obtains the command buffer and compute encoder from
property getters (mtlQueue.commandBuffer, buffer.computeCommandEncoder),
which return autoreleased references under the non-ARC build, and then
releases them via ff_objc_release(). That is an over-release: when the
enclosing autorelease pool is drained, the objects are released a second
time and the process crashes with EXC_BAD_ACCESS in objc_release.
On current macOS this is reproducible with the ffmpeg CLI itself. The
filter thread's thread-local autorelease pool is popped during pthread
TSD cleanup at thread exit, so any run that used yadif_videotoolbox
crashes when the filter thread terminates:
ffmpeg -init_hw_device videotoolbox=vt -filter_hw_device vt \
-f lavfi -i testsrc2=size=640x480:rate=30 \
-vf setfield=tff,format=nv12,hwupload,yadif_videotoolbox,\
hwdownload,format=nv12 -frames:v 20 -f null -
Thread 11 Crashed:: vf#0:0
0 libobjc.A.dylib objc_release + 16
1 libobjc.A.dylib AutoreleasePoolPage::releaseUntil(objc_object**) + 204
2 libobjc.A.dylib objc_autoreleasePoolPop + 244
3 libobjc.A.dylib objc_tls_direct_base<...>::dtor_(void*) + 156
4 libsystem_pthread.dylib _pthread_tsd_cleanup + 620
5 libsystem_pthread.dylib _pthread_exit + 84
Host applications hit the same crash earlier, for example when a
dispatch queue drains its last-resort pool after a work item that ran
the filter.
Wrap the kernel invocation in @autoreleasepool and drop the manual
releases, making the pool pop the single balanced release and draining
the Metal transients once per frame. The ff_objc_release() calls on the
s->mtl* members in do_uninit() stay: those hold owned references
obtained from new* methods.
Fixes: division by zero
Fixes: signed integer overflow
Fixes: NaN to int conversion
Fixes: 2qHh10XPbgiO
Fixes: swarm-Signed-integer-overflow-in-draw_arrow-length-calc-8f1f15
Division by zero Replicated through UnModified FFmpeg with ASAN
Signed integer overflow Replicated through UnModified FFmpeg with UBSAN
NaN to int conversion Replicated through UnModified FFmpeg with UBSAN
Found-by: zhang xingxing <2388969553@qq.com>
The report suggested doing the shift in a wider unsigned type.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Regression since: aa8905a1b1
NULL pointer dereference Replicated through UnModified FFmpeg with ASAN
Found during review of the fixes for the security report GIRN0tvt58WQ
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This is a behavior preserving change for all non-overflowing cases.
Found during triage of the security report GIRN0tvt58WQ
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: GIRN0tvt58WQ
Fixes: swarm-Signed-integer-overflow-in-process_slice-slice-computation-b29bec
Regression since: 631f7acc6c
Out of array read Replicated through UnModified FFmpeg with ASAN
Found-by: zhang xingxing <2388969553@qq.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: Ilayk1MzOixN
Fixes: swarm-Signed-integer-overflow-in-pad-x-y-bounds-check-499261
Regression since: 42eb10ecc6
Found-by: zhang xingxing <2388969553@qq.com>
The report suggested, among other options, widening the comparison.
The overflow in the centering was found during review of the fixes for the report.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: Ilayk1MzOixN
Fixes: swarm-Signed-integer-overflow-in-pad-x-y-bounds-check-499261
Regression since: 57c3670896
Signed integer overflow Replicated through UnModified FFmpeg with UBSAN
Found-by: zhang xingxing <2388969553@qq.com>
The report suggested, among other options, widening the comparison.
The overflow in the centering was found during review of the fixes for the report.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array read
Fixes: BxsBdfl8050I
Fixes: swarm-Signed-integer-overflow-in-build_lut-slow-check-enables-heap-OOB-read-ba3309
Regression since: a7fc78c1a6
Out of array read Replicated through UnModified FFmpeg with ASAN
Found-by: zhang xingxing <2388969553@qq.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit enables the DNN video filter to run entirely on the GPU
for the full D3D12 pipeline (decode, AI inference, and encode)
without ever copying frames back to system memory, making it
significantly faster on D3D12 hardware.
Benchmarked the new zero-copy D3D12/DirectML DNN pipeline against
the existing CPU round-trip pipeline on three major GPU vendors; up
to 10x throughput improvement was measured across every vendor.
Add dnn_onnx_d3d12, a helper that binds the DirectML execution
provider to a D3D12VA input frame's own device and does NV12
texture<->tensor conversion on the GPU (CopyTextureRegion + compute
shaders), avoiding a host round-trip.
dnn_processing negotiates this automatically for D3D12 NV12 input
with device=dml. Software-frame input retains the host I/O path.
D3D12 input requires successful DirectML negotiation; unsupported
hardware-input configurations fail during configuration. Currently
limited to single-channel FLOAT/NCHW models; chroma bypasses the
model and is copied/upscaled on the GPU. At runtime, DirectML.dll
and d3dcompiler_47.dll must be loadable from the application
directory or System32.
Example usage:
# Zero-copy pipeline
ffmpeg.exe -y -loglevel info \
-hwaccel d3d12va -hwaccel_output_format d3d12 \
-i input-1920x1080.mp4 \
-vf "dnn_processing=dnn_backend=onnx:device=dml:model=ecbsrnet_y_fused.onnx:input=input:output=output" \
-c:v h264_d3d12va -bf 0 -c:a copy \
output_2x_sr_zero_copy.mp4
# CPU round-trip pipeline
ffmpeg.exe -y -loglevel info \
-init_hw_device d3d12va=dev -filter_hw_device dev -hwaccel d3d12va \
-i input-1920x1080.mp4 \
-vf "dnn_processing=dnn_backend=onnx:device=dml:model=ecbsrnet_y_fused.onnx:input=input:output=output,hwupload" \
-c:v h264_d3d12va -bf 0 -c:a copy \
output_2x_sr_round_trip.mp4
Signed-off-by: younengxiao <steven.xiao@amd.com>
The kernel keeps its peak detection state in util_mem (frame index,
frame count, running totals of peak and average) and expects it to
start at zero. The buffer is created without a host pointer, so per the
OpenCL spec its contents are undefined. Intel's compute-runtime (NEO
24.35, legacy branch, on a Gen9 Apollo Lake) hands out recycled memory,
and the totals can start as garbage: the detected peak is then huge and
the whole stream comes out black (Y=16), or with the wrong brightness.
The same command gave black video every time on one film and a correct
picture on another; with the buffer zeroed both are correct.
Create the buffer from a zeroed array with CL_MEM_COPY_HOST_PTR.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VB8Q6zwZGmj1CET8B15HLg
The check compared the width twice so a height-only change was not
rejected when debayering.
Found during a code review sweep of ffmpeg using an LLM.
Signed-off-by: romainhedouin <romain.hedouin@epitech.eu>
This is not a security issue.
Fixes: 1QLxiyziAd9N
Fixes: swarm-NULL-feature_name-dereferenced-in-parse_features-129e69
Regression since: 3d29724c00
NULL feature name Replicated through UnModified FFmpeg (the error message read "(null)")
Found-by: zhang xingxing <2388969553@qq.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
