Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f96f9d111)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Also check for possible overflows.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit b0a043f51b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Negative values don't make sense and too large values can cause
overflows. For AV_CODEC_ID_ADPCM_THP this leads to a too small extradata
buffer being allocated, causing out-of-bounds writes.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ee5f0f1d35)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes a heap-buffer-overflow in ff_er_frame_end when decoding mss2
with coded_width/coded_height larger than width/height.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 2566ad98b0)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
floats are not necessarily normalized, so a normalized softfloat needs
MIN_EXP lowered by 23 to cover that range.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 2d6f46d801)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes an out-of-bounds read if avc->channels is 0.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 8c8f543b81)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Rects with positive w/h/linesize but no data are invalid.
Reviewed-by: Petri Hintukainen <phintuka@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 995512328e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Josh de Kock <josh@itanimul.li>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit dbefbb61b7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes segmentation faults due to stack-overflow caused by too deep
recursion.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 946ecd19ea)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Metadata streams have priv_data set to NULL.
Reviewed-by: Josh de Kock <josh@itanimul.li>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit fdb8c455b6)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 3c0328d58d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
They are not valid and can cause problems/crashes for API users.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a86ebbf7f6)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This is similar to commit 9ac61e73d0.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 0edd569466)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This prevents a division by zero in read_packet.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 25012c5644)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Otherwise put_bits can be called with a value that doesn't fit in the
sample_len, causing an assertion failure.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit cdb5479c9d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes a heap-buffer-overflow detected by AddressSanitizer.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 360bc0d90a)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
It can read less than the requested amount, in which case buf contains
uninitialized data, causing problems like segmentation faults later on.
Also make sure that image->size is positive, so that it can't match a
negative error code.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 89eb398c7f)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This problem was introduced in commit
4b90dcb849.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit c82b8ef0e4)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The implicit checks via v_data_size and a_data_size don't work in the case
'(hdr_size > 7) && !ctx->alpha_info'.
This fixes segmentation faults due to invalid reads.
This problem was introduced in commit
547c2f002a.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1e33035ee7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
It causes a cb_depth of 32, leading to assertion failures in get_bits.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 226d35c845)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit d54c95a143)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 467eece1be)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1bbb18fe82)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The code assumes that s->streams[0] is valid.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ff100c9dd9)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The API does not allow returning AVERROR codes.
It triggers an assert in av_parser_parse2.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5249706e9d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Metadata streams have priv_data set to NULL.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 0efb610611)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes heap-use-after-free detected by AddressSanitizer.
Reviewed-by: Luca Barbato <lu_zero@gentoo.org>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit bb6a7b6f75)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This was broken by the following Libav commit:
4c387c7 ppc: dsputil: do unaligned block accesses correctly
The following tests fail due to this:
fate-checkasm
fate-vsynth1-dnxhd-2k-hr-hq fate-vsynth1-dnxhd-edge1-hr
fate-vsynth1-dnxhd-edge2-hr fate-vsynth1-dnxhd-edge3-hr
fate-vsynth1-dnxhd-hr-sq-mov fate-vsynth1-dnxhd-hr-hq-mov
fate-vsynth2-dnxhd-2k-hr-hq fate-vsynth2-dnxhd-edge1-hr
fate-vsynth2-dnxhd-edge2-hr fate-vsynth2-dnxhd-edge3-hr
fate-vsynth2-dnxhd-hr-sq-mov fate-vsynth2-dnxhd-hr-hq-mov
fate-vsynth3-dnxhd-2k-hr-hq fate-vsynth3-dnxhd-edge1-hr
fate-vsynth3-dnxhd-edge2-hr fate-vsynth3-dnxhd-edge3-hr
fate-vsynth3-dnxhd-hr-sq-mov fate-vsynth3-dnxhd-hr-hq-mov
Fixes trac ticket #5508.
Reviewed-by: Carl Eugen Hoyos <ceffmpeg@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 3932ccc472)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes out-of-bounds reads by the bitstream reader.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 60178e78f2)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The number of channels is used as divisor in decode_frame, so it must
not be zero to avoid SIGFPE crashes.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5540d6c134)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes out-of-bounds reads.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 14e4e26559)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Otherwise another frame gets referenced into picture, triggering an assert
(from commit 13aae8) in av_frame_ref.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a92f8edf0c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes asserts (from commit 13aae8) in av_frame_ref and
av_frame_move_ref.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1966ea012f)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If probe_codec is called with pkt == NULL, it sets probe_packets to 0
and request_probe to -1.
However, request_probe can change when calling s->iformat->read_packet
and thus a probe_packets value of 0 doesn't guarantee a request_probe
value of -1.
In that case calling probe_codec again is necessary to prevent
triggering the assert.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a5b4476a60)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If realloc fails, the pointer is overwritten and the previously allocated
buffer is leaked, which goes against the expected behavior of keeping the
packet unchanged in case of error.
Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 574929d8b6)
This code is not correct for git master
Reviewed-by: Stefano Sabatini <stefasab@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2bd9956454)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The calculation of width/height should round up, not round down to
prevent setting width or height to 0.
Also image->comps[compno].w is unsigned (at least in openjpeg2), so the
calculation could silently wrap around without the explicit cast to int.
Reviewed-by: Michael Bradshaw <mjbshaw@gmail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 56706ac0d5)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
openjpeg 2 sets the data pointers of the image components to NULL,
causing segfaults if the image is reused.
Reviewed-by: Michael Bradshaw <mjbshaw@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 69c8505f3b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Use check_lib2 to test the header together with the function. This is
necessary, because '-DOPJ_STATIC' changes what the included header does.
Also add '-DOPJ_STATIC' to CPPFLAGS, so that it isn't necessary to
hardcode this in libavcodec/libopenjpeg{dec,enc}.c.
Finally, check for non-static openjpeg 2.1, too.
Reviewed-by: Michael Bradshaw <mjbshaw@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7a65aef00d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3e5d27d7a7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f4e4bde1f4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3305f71025)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Moritz Barsnick <barsnick@gmx.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f71c98ee12)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Decoders have previously not used AVFrame.pts, and with the upcoming
deprecation of pkt_pts (in favor of pts), this would lead to an errorneous
interpration of timestamps.
(cherry picked from commit 04a3577263)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Michael Niedermayer
Mark Harris
Andreas Cadhalpun
James Almer
Moritz Barsnick
Hendrik Leppkes
Shivraj Patil