This fixes segmentation faults due to stack-overflow caused by too deep
recursion.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Sean McGovern <gseanmcg@gmail.com>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Bug-Id: 1098
Cc: libav-stable@libav.org
Signed-off-by: Sean McGovern <gseanmcg@gmail.com>

Return sensible error values and forward error codes.

Fixes out of array access
Fixes: ce19e41f0ef1e52a23edc488faecdb58/asan_heap-oob_2504e97_4202_ffa0df1baed14022b9bfd4f8ac23d0cb.smk

Bug-Id: CVE-2015-8365
CC: libav-stable@libav.org

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4a9af07a49)
Signed-off-by: Diego Biurrun <diego@biurrun.de>

Do not rely on indirectly including it from bitstream.h.

Signed-off-by: Diego Biurrun <diego@biurrun.de>

This avoids the danger that get_bits.h might get indirectly #included before
BITSTREAM_READER_LE is defined.

Also sort headers into canonical order where appropriate.

Express bitfields more simply.

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

Signed-off-by: Martin Storsjö <martin@martin.st>

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>

Also try to free local allocations on errors.

Signed-off-by: Martin Storsjö <martin@martin.st>

Also prevent a memory leak.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Smacker audio uses overflows instead of clipping.

This reverts commit 375ca0aca8.

It's got_frame, not data size

It will be useful in the upcoming transition to refcounted AVFrames.

Also reorder some other #include when applicable.

This removes some code apparently left over from vlc reader
debugging.

Signed-off-by: Mans Rullgard <mans@mansr.com>

Also break some long lines, remove codec function placeholder comments
and add spaces in sample/pixel format lists.

Signed-off-by: Martin Storsjö <martin@martin.st>

Fixes Bug #265

Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

Also remove one pointless zero initialization in rangecoder.c.

Signed-off-by: Martin Storsjö <martin@martin.st>

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

CC: libav-stable@libav.org

The prefix is a historic remnant that probably meant "alternative".
Now that the A32 bitstream reader has been dropped it makes no sense anymore.

Deprecate avcodec_decode_audio3().
Implement audio support in avcodec_default_get_buffer().
Implement the new audio decoder API in all audio decoders.

This matches the output sample format and the range of the output values.

stereo & 16bit is untested due to lack of samples

Signed-off-by: Martin Storsjö <martin@martin.st>

Signed-off-by: Martin Storsjö <martin@martin.st>

Signed-off-by: Martin Storsjö <martin@martin.st>

It's more readable and less prone to breakage.

Signed-off-by: Diego Biurrun <diego@biurrun.de>

The release_buffer was cleaning the provided frame, thus causing the
successive call to avctx->reget_buffer() to allocate a new frame.  In
case the returned frame was not the same one previously returned but a
new one with different data, it resulted in artifacts.

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>