This makes switching to encode2() simpler, because it allows us to know
exactly how large should the output buffer be before we start writing
into it.

Red zone usage is not allowed in the Win64 ABI.

Update api so it will compile again.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

Signed-off-by: Anton Khirnov <anton@khirnov.net>

Signed-off-by: Anton Khirnov <anton@khirnov.net>

This allows the following commit to extrapolate better dts for the first
frame. Pts difference between the first two frames is reused as the
difference between pts and dts of the first frame.

Apple ProRes Format Specifications mentions target data size for every frame,
so make sure frame meets it. This also allows encoder to demand much smaller
packet sizes for output.

The parser uses VLC tables initialized in vc1_common_init(), therefore
we should call this function on parser init also.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Return 0 means "please return the same data again", i.e. it causes an
infinite loop. Instead, return an error.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Return 0 indicates "please return the same data again", i.e. it causes
an infinite loop. Instead, return that we consumed the buffer if we
finished decoding succesfully, or return an error if an error occurred.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Martin Storsjö <martin@martin.st>

Fixes: CVE-2011-3951

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

If bufsize < headersize, init_get_bits() will be called with a negative
number, causing it to fail and any subsequent call to get_bits() will
crash because it reads from a NULL pointer.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

On EOF, get_bits() will continuously return 0, causing an infinite
loop.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Martin Storsjö <martin@martin.st>

Signed-off-by: Martin Storsjö <martin@martin.st>

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

Width is padded for ham encodings too.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

The unused code being removed is for encoding only and therefore is not needed
by the decoder.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

WMApro actually support 13-bits block sizes (potentially even up to 14),
and thus we should support that also. If we get block sizes beyond what
the decoder can handle (14 is possible depending on s->decode_flags),
error out instead of crashing.

Fixes invalid free() if any of the buffers are not allocated due to either
not decoding a header or an error prior to allocating all buffers.

Fixes CVE-2012-0858
CC: libav-stable@libav.org

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

Add a check to avoid writing past the end of the channel_unit.components[]
array.

Bug Found by: cosminamironesei
Fixes CVE-2012-0853
CC: libav-stable@libav.org

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

This makes the check that avoids overwrite of the samples array actually
work properly.

fixes CVE-2012-0848
CC: libav-stable@libav.org

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Update FATE reference to account for now non-existent palette packet.
This also fixes the FATE test if frame data is not initialized in
get_buffer(), so update comment in avconv accordingly.

By replacing memcpy with an unrolled loop using the alignment knowledge
it has, some speedup can be obtained.

Before (gcc 4.6.1): ~400 cycles
After: ~370 cycles

Overall, around 2% speed increase when decoding a 2400s mp3 to f32le.

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Martin Storsjö <martin@martin.st>

Signed-off-by: Martin Storsjö <martin@martin.st>