Fixes: Timeout
Fixes: 6297/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_NUV_fuzzer-4882404863901696

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 939440ad1a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: Timeout
Fixes: 6297/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_NUV_fuzzer-4882404863901696

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8ee3265dbe)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Diego Biurrun <diego@biurrun.de>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

Express bitfields more simply.

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

There is no point in populating NuvContext with another DSPContext.

Also split static and dynamic initialization bits to avoid running the
static initialization parts over and over.

Fixes use of uninitialized memory
Fixes msan_uninit-mem_7f4a141261de_7405_nirvana.nuv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

CC: libav-stable@libav.org

The decompressed buffer can be used after codec_reinit, so it must be
preserved.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

And properly update the buf_size with the correct size.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Update the fate reference since the last broken frame is not decoded
anymore.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Coccinelle profile used:

  @@
  expression r, ctx, f, loglevel, str, flags;
  @@

  -if ((r = ff_get_buffer(ctx, f, flags)) < 0) {
  -    av_log(ctx, loglevel, str);
  -    return r;
  -}
  +if ((r = ff_get_buffer(ctx, f, flags)) < 0)
  +    return r;

  @@
  expression r, ctx, f, loglevel, str;
  @@

  -if ((r = ff_reget_buffer(ctx, f)) < 0) {
  -    av_log(ctx, loglevel, str);
  -    return r;
  -}
  +if ((r = ff_reget_buffer(ctx, f)) < 0)
  +    return r;

  @@
  expression r, ctx, f, loglevel, str, flags;
  @@

  -if ((r = ff_thread_get_buffer(ctx, f, flags)) < 0) {
  -    av_log(ctx, loglevel, str);
  -    return r;
  -}
  +if ((r = ff_thread_get_buffer(ctx, f, flags)) < 0)
  +    return r;

...along with some manual patches for the remaining ones.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes assertion failure

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

It's got_frame, not data size

Signed-off-by: Martin Storsjö <martin@martin.st>

CC: libav-stable@libav.org

The check is bogus since the nuv frameheader is already skipped
and the (decompressed) RTjpeg header is checked.

This reverts commit f6afacdb3b.

CC: libav-stable@libav.org

The previous validity check seems to work only for some
(presumably older) files, in current versions the first bytes
now contain the data size.

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Also break some long lines, remove codec function placeholder comments
and add spaces in sample/pixel format lists.

Signed-off-by: Martin Storsjö <martin@martin.st>

Signed-off-by: Martin Storsjö <martin@martin.st>

Signed-off-by: Martin Storsjö <martin@martin.st>

Since it contains dimensions parsing an invalid one has rather
annoying effects.

Signed-off-by: Anton Khirnov <anton@khirnov.net>

Signed-off-by: Anton Khirnov <anton@khirnov.net>

This fixes visible corruption in the incomplete last frame
of the FATE sample.

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>

There were multiple issues, for example might we have to re-run
the decompression when the size of the buffer increased,
we should always use a decompression buffer large enough for
the header (so we do not get stuck when the size is too small).

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>

Since it contains dimensions parsing an invalid one has rather
annoying effects.

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>

It's more readable and less prone to breakage.

Signed-off-by: Diego Biurrun <diego@biurrun.de>

Signed-off-by: Diego Biurrun <diego@biurrun.de>