Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5f0bc0215a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 161ccdaa06)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 16793504df)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7d196f2a5a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a6330119a0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d99101d09)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a2b8dde659)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 121be31060)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes out of array read

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 042faa847f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes out of array read

Found-by: Thomas Garnier using libFuzzer
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c92f55847a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes out of array access

Found-by: Thomas Garnier using libFuzzer
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 25ab1a65f3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This is intended to workaround bug "665 Integer Divide Instruction May
Cause Unpredictable Behavior" on some early AMD CPUs, which causes a
div-by-zero in this codepath, such as reported in Mozilla bug #1293996.

Note that this isn't guaranteed to fix the bug, since a compiler is free
to reorder instructions that don't depend on each other. However, it
appears to fix the bug in Firefox, and a similar patch was applied to
libvpx also (see Chrome bug #599899).

(cherry picked from commit be885da342)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes out of array read

Found-by: Thomas Garnier using libFuzzer
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 85d23e5cbc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fecb3e82a4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This function must be called from the mb or slice encoding loop and MMX state may not
be clean there

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 03ec6b780c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f96f9d111)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Avoid invalid memory read/crash when frame offset >= 0xfffffff8.
Base64-encoded example: AAABADAwMDAwMAAAMAAwMDAw/P///w==
(The previous commit verifies that p->buf_size >= 22.)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 56e2cd9c04)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Also check for possible overflows.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit b0a043f51b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Negative values don't make sense and too large values can cause
overflows. For AV_CODEC_ID_ADPCM_THP this leads to a too small extradata
buffer being allocated, causing out-of-bounds writes.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ee5f0f1d35)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

This fixes a heap-buffer-overflow in ff_er_frame_end when decoding mss2
with coded_width/coded_height larger than width/height.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 2566ad98b0)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

floats are not necessarily normalized, so a normalized softfloat needs
MIN_EXP lowered by 23 to cover that range.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 2d6f46d801)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

This fixes an out-of-bounds read if avc->channels is 0.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 8c8f543b81)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Rects with positive w/h/linesize but no data are invalid.

Reviewed-by: Petri Hintukainen <phintuka@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 995512328e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Reviewed-by: Josh de Kock <josh@itanimul.li>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit dbefbb61b7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

This fixes segmentation faults due to stack-overflow caused by too deep
recursion.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 946ecd19ea)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Metadata streams have priv_data set to NULL.

Reviewed-by: Josh de Kock <josh@itanimul.li>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit fdb8c455b6)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 3c0328d58d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

They are not valid and can cause problems/crashes for API users.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a86ebbf7f6)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

This is similar to commit 9ac61e73d0.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 0edd569466)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

This prevents a division by zero in read_packet.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 25012c5644)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Otherwise put_bits can be called with a value that doesn't fit in the
sample_len, causing an assertion failure.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit cdb5479c9d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

This fixes a heap-buffer-overflow detected by AddressSanitizer.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 360bc0d90a)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

It can read less than the requested amount, in which case buf contains
uninitialized data, causing problems like segmentation faults later on.

Also make sure that image->size is positive, so that it can't match a
negative error code.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 89eb398c7f)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

This problem was introduced in commit
4b90dcb849.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit c82b8ef0e4)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

The implicit checks via v_data_size and a_data_size don't work in the case
'(hdr_size > 7) && !ctx->alpha_info'.

This fixes segmentation faults due to invalid reads.

This problem was introduced in commit
547c2f002a.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1e33035ee7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

It causes a cb_depth of 32, leading to assertion failures in get_bits.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 226d35c845)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit d54c95a143)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 467eece1be)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1bbb18fe82)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

The code assumes that s->streams[0] is valid.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ff100c9dd9)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

The API does not allow returning AVERROR codes.

It triggers an assert in av_parser_parse2.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5249706e9d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Metadata streams have priv_data set to NULL.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 0efb610611)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

This fixes heap-use-after-free detected by AddressSanitizer.

Reviewed-by: Luca Barbato <lu_zero@gentoo.org>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit bb6a7b6f75)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

If it fails, buffers aren't allocated, causing NULL pointer dereferencing.

Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit db79dedb1a)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

This was broken by the following Libav commit:
4c387c7 ppc: dsputil: do unaligned block accesses correctly

The following tests fail due to this:
fate-checkasm
fate-vsynth1-dnxhd-2k-hr-hq fate-vsynth1-dnxhd-edge1-hr
fate-vsynth1-dnxhd-edge2-hr fate-vsynth1-dnxhd-edge3-hr
fate-vsynth1-dnxhd-hr-sq-mov fate-vsynth1-dnxhd-hr-hq-mov
fate-vsynth2-dnxhd-2k-hr-hq fate-vsynth2-dnxhd-edge1-hr
fate-vsynth2-dnxhd-edge2-hr fate-vsynth2-dnxhd-edge3-hr
fate-vsynth2-dnxhd-hr-sq-mov fate-vsynth2-dnxhd-hr-hq-mov
fate-vsynth3-dnxhd-2k-hr-hq fate-vsynth3-dnxhd-edge1-hr
fate-vsynth3-dnxhd-edge2-hr fate-vsynth3-dnxhd-edge3-hr
fate-vsynth3-dnxhd-hr-sq-mov fate-vsynth3-dnxhd-hr-hq-mov

Fixes trac ticket #5508.

Reviewed-by: Carl Eugen Hoyos <ceffmpeg@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 3932ccc472)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

This fixes out-of-bounds reads by the bitstream reader.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 60178e78f2)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

The number of channels is used as divisor in decode_frame, so it must
not be zero to avoid SIGFPE crashes.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5540d6c134)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

This fixes out-of-bounds reads.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 14e4e26559)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Otherwise another frame gets referenced into picture, triggering an assert
(from commit 13aae8) in av_frame_ref.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a92f8edf0c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

This fixes asserts (from commit 13aae8) in av_frame_ref and
av_frame_move_ref.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1966ea012f)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>