Fixes: Out of array access
Fixes: 20659/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5658548592967680
Fixes: 20659/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5723561177382912

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a035fd88ae)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: negation of -2147483648 cannot be represented in type 'int32_t' (aka 'int'); cast to an unsigned type to negate this value to itself
Fixes: signed integer overflow: -2147483648 - 1 cannot be represented in type 'int32_t' (aka 'int')
Fixes: 20492/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5764066459254784

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7a92147f87)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: out of array read
Fixes: 20796/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5111364702175232.fuzz

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b4a33387cb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Reviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bc41a29a5a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

IFF-PBM-HAM6 can read out of array without this overallocation
Fixes: Out of array read
Fixes: 19752/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5675331403120640

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8652f4e7a1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

video is allocated before parameters like bpp are read.

Fixes: out of array access
Fixes: 19084/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5718556033679360
Fixes: 19465/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5759908398235648

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f1b97f62f8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: Timeout (107sec - 75ms>
Fixes: 18812/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-6295585225441280

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 185f441ba2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: index 9 out of bounds for type 'const uint64_t [8][256]'
Fixes: 18409/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5767030560522240
Fixes: 18720/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5651995784642560

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a1f8b36cc4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: Timeout (31sec -> 41ms)
Fixes: 18380/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5645210121404416

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 32b3c8ce7d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: undefined memcpy() use
Fixes: 16302/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5678750575886336

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dfa5d1a366)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

The initialization of the uint32_t plane32_lut matrix uses left shifts
of the form 1 << plane; plane can be as big as 31 which means that this
is undefined behaviour as 1 will be simply an int. So make it unsigned
to avoid this.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f12e662a3d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This checks the ham value much stricter and avoids hitting cases which cannot be reached
with data from the libavformat demuxer.

Fixes: out of array access
Fixes: 15320/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5080476840099840
Fixes: 15423/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5630765833912320

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f76d7352e0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: out of array access
Fixes: 15381/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5668057826983936

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0f9789c8e3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: 15372/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_IFF_ILBM_fuzzer-5708881759567872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 92e8db532c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

It appears it is wrong.

Fixes CID1361959

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

This should avoid long loops
related to CID1361958

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes Ticket5585

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Signed-off-by: Diego Biurrun <diego@biurrun.de>

It was merged with the iff_ilbm decoder in commit
929a24efff.

Define AV_CODEC_ID_IFF_BYTERUN1 as AV_CODEC_ID_IFF_ILBM for API
compatibility.

Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

Express bitfields more simply.

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

Fixes out of array access

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes use of uninitialized memory
Fixes: msan_uninit-mem_7fa0dea15eae_8988_test.iff
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes use of uninitialized memory
Fixes: msan_uninit-mem_7fe205e395ed_8988_test.iff
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Define positive return values as non errors and leave further meaning undefined
This allows future extensions to use these values

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Coccinelle profile used:

  @@
  expression r, ctx, f, loglevel, str, flags;
  @@

  -if ((r = ff_get_buffer(ctx, f, flags)) < 0) {
  -    av_log(ctx, loglevel, str);
  -    return r;
  -}
  +if ((r = ff_get_buffer(ctx, f, flags)) < 0)
  +    return r;

  @@
  expression r, ctx, f, loglevel, str;
  @@

  -if ((r = ff_reget_buffer(ctx, f)) < 0) {
  -    av_log(ctx, loglevel, str);
  -    return r;
  -}
  +if ((r = ff_reget_buffer(ctx, f)) < 0)
  +    return r;

  @@
  expression r, ctx, f, loglevel, str, flags;
  @@

  -if ((r = ff_thread_get_buffer(ctx, f, flags)) < 0) {
  -    av_log(ctx, loglevel, str);
  -    return r;
  -}
  +if ((r = ff_thread_get_buffer(ctx, f, flags)) < 0)
  +    return r;

...along with some manual patches for the remaining ones.