Fixes ticket #8160.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 324487b596)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Affected 61 FATE-tests: 60 vsynth tests and lavf-mxf_opatom.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b7f156e8cb)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Affected the FATE-tests vsynth_lena-dv-411, vsynth1-dv-411,
vsynth2-dv-411 and hevc-paramchange-yuv420p.yuv420p10.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e2646e23be)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

This affected many FATE-tests: The number of failing tests went down
from 663 to 344. (Both numbers exclude tests that failed because of
unaligned accesses in code that is inside #if HAVE_FAST_UNALIGNED.)

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 736c7c20e7)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Affected the FATE-tests exr-rgb-scanline-pxr24-half-uint32-13x9 and
exr-rgb-scanline-pxr24-uint32.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8b0f949906)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Fixes the movenc FATE-test.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 646799b42f)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Fixes the acodec-pcm-u16[lb]e FATE-tests.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 69473bec6f)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Fixes ticket #8161 and the acodec-wavpack FATE-test.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 84974c6fb5)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Fixes the FATE-tests unknown_layout-ac3, ac3-fixed-encode, ac3-encode
and eac3-encode. It furthermore fixes the ac3-encoder bugs mentioned in
tickets #7994, #8144 and #8159.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2f289ec914)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Fixes the tdsc FATE-test.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 86bef10e7a)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Fixes ticket #8154.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2b1fcba8dd)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Only read str_size bytes from offset 30 of extradata if the extradata is
indeed at least 30 + str_size bytes long.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit ff3fad6b0e)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

mov_read_custom tries to read three strings belonging to three different
tags. When an already encountered tag is encountered again, a new buffer
for the string to be read is allocated and stored in the pointer
destined for this particular tag. But in this scenario, said pointer
already holds the address of the string read earlier, leading to a leak.

This commit therefore aborts the reading process upon encountering
an already encountered tag.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit dfef1d5e3c)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The argument pertaining to a printf %s conversion specifier must not
be NULL, even if the precision (i.e. the number of characters to write)
is zero. If it is NULL, it is undefined behaviour.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 6de6ce7bc8)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon reading a subsequent subtitle.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit c784fe8b86)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon reading a subsequent subtitle.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 67434afa7f)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if allocating the AVStream for the subtitles fails.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 337783b118)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon reading a subsequent subtitle.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit a708f65273)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon reading a subsequent subtitle.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 9751d75152)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon reading a subsequent subtitle.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit e13874b9ea)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon reading a subsequent subtitle.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit c70409957c)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon reading a subsequent subtitle.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit f3c63e67bb)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon reading a subsequent subtitle
or when creating extradata.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit f161f8e4ad)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon reading a subsequent subtitle.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 9df560e898)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon creating an AVStream.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit a5ed8aeea4)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon reading a subsequent subtitle.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 331799747e)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon reading a subsequent subtitle
or when allocating extradata.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit b12014a5b8)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon reading a subsequent subtitle.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit d38694cea9)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon reading a subsequent subtitle.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit c13a752733)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon reading a subsequent subtitle
or if creating the extradata failed.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 5ab39c2d8c)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The already parsed subtitles (contained in an FFDemuxSubtitlesQueue)
would leak if an error happened upon reading a subsequent subtitle.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit a86a5d06d8)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

By default, a demuxer's read_close function is not called automatically
if an error happens when reading the header; instead it is up to the
demuxer to clean up after itself in this case. The mov demuxer did this
by calling its read_close function when it encountered some errors when
reading the header. Yet for other errors (mostly adding side-data to
streams) this has been forgotten, so that all the internal structures
of the demuxer leak.

This commit fixes this by making sure mov_read_close is called when
necessary.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit ac378c535b)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Fixes possible leaks of id3v2 metadata as well as an AVDES struct in
case the content is encrypted and an error happens lateron.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 3d3ba43bc6)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

When demuxing a Matroska/WebM file, streams are added for tracks and for
attachments, so that the array containing the former can be NULL even
when the corresponding AVFormatContext has streams. So check for there
to be tracks in the MatroskaDemuxContext instead of just streams in the
AVFormatContext before dereferencing the pointer to the tracks.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 1ef30571a0)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

matroska_parse_block currently asserts that the duration is not equal to
AV_NOPTS_VALUE, but there is nothing that actually guarantees this. It
is easy to create (spec-compliant) files which run into this assert;
so replace it and instead cap the duration to INT64_MAX, as the duration
field of an AVPacket is an int64_t.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 3714d452b8)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

If adding two ints overflows, it doesn't matter whether the result will
be stored in an unsigned or not; and checking afterwards does not make it
retroactively defined.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 28a078eded)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit cb255b616c)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The data of an attachment file is put into an AVCodecParameter's
extradata. The corresponding size field has type int, yet there was no
check for the size to fit into an int. As a consequence, it was possible
to create extradata with negative size (by using a big enough max_alloc).

Other errors were also possible: If SIZE_MAX < INT64_MAX (e.g. on 32bit
systems) then the file size might be truncated before the allocation;
and avio_read() takes an int, too, so one would not have read as much
as one desired.

Furthermore, the extradata is now padded as is required.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 889ad93c88)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The WebM DASH Manifest muxer only supports VP8, VP9, Vorbis and Opus,
but there was no check for this. The codec type is used to get a pointer
to a string containing the codec name or NULL if it is not one of those
four codecs. Said pointer has then been used without further checks as
string for the %s conversion specifier in an avio_printf()) call which
is undefined behaviour.

This commit adds a check for the supported codec types.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit cbea58b2b3)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The classical ptr = av_realloc(ptr, size).

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 418e468699)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit ba36a07734)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

and also add padding to it; moreover, don't use memcpy to write one byte
to extradata.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9e0f3352d1)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Do it only if the packet has been successfully allocated in
av_new_packet() -- otherwise on error a completely uninitialized packet
would be unreferenced later.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6bd8bcc2ac)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

The classical ptr = av_realloc(ptr, size), just with av_fast_realloc().

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5acef12061)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

If an AVFormatContext could be allocated, but white-/blacklists couldn't
be copied, the AVFormatContext would leak as it was only accessible
through a local variable that goes out of scope when one goes to fail.

Furthermore, in case writing a header of a submuxer failed, the options
used for said call could leak.

Both of these memleaks have been fixed.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit abbb466368)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Don't read a 64bit number before having checked that the data is at
least 8 bytes long.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6e9cc96429)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

A demuxer might have allocated memory while reading the header. If
reading the header was successfull and an error happens before returning
(e.g. when queueing the attached pictures), the read_close function
would have never been called, so that all those allocations would leak.
This commit changes this.

Furthermore, there would be even more memleaks if the error level was
set to AV_EF_EXPLODE in case there is both metadata and id3v2 metadata.
This has been fixed, too.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Marton Balint <cus@passwd.hu>
(cherry picked from commit e2307f4ff1)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Affected the ffmpeg-filter_colorkey FATE-test (but only if the C version
of idct8_add is used and not e.g. the x86 SSE2 version).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

Fixes: left shift of negative value -107
Fixes: 20398/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CAVS_fuzzer-5725389278412800

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0f0f2ab0c3)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

This is needed especially for AV1: If a reformatting error happens (e.g.
if the length field of an OBU contained in the current packet indicates
that said OBU extends beyond the current packet), the data pointer is
still NULL, yet the size is unchanged, so that writing the data leads
to a segmentation fault.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 58428bef4b)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

by replacing it with a multiplication. Said multiplication can't
overflow an int32_t because lpc_coefs is limited to 16 bit precision.

Fixes the FACE-test acodec-ra144 as well as part of #8217.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e3fb9af6f1)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>