Prevents crashes with the fuzzed samples from bugs 88, 89 and 125 after
"golomb: avoid infinite loop on all-zero input".

Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

Factors all context dynamic memory handling to its own functions.
Fixes bug 220.

Also adds an error message.

It is used as bitfield with 16 entries.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes: Tikcet969

Thanks-to: ITU for the freely available spec.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

The wrong variable was passed into decode_ham_plane32()
Fixes: Ticket922

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Recent versions of gcc (4.4+) no longer give false positive warnings.

Also give a more suitable name to the MMI-optimized IDCT;
it is not PS2-specific, as the name currently suggests.

This fixes compilation with -Werror=missing-prototypes.

%ifdef HAVE_AVX must now be %if HAVE_AVX.

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>

The cleanup is only done now when
a picture is returned (assuming that it has to be done when its returned)
a error is returned (assuming that there will be no further progress on the frame)
the codec is not h264 (this is still needed due to some deadlocks in realvideo)

This fixes a decoding regression with 00017.MTS

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This order is more logic and might prevent a race.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This fixes some deadlock without the generic cleanup code.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Some MMX-only CPUs do not have support for CMOV.
All SSE/MMX2 CPUs should be fine, thus no check was
added to those functions.
See also https://sourceforge.net/tracker/?func=detail&aid=3358347&group_id=205275&atid=992986

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>

Fixes ticket #973.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Makes the next patch smaller.

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>

Simplifies use of verbatim mode.

Otherwise it is not needed because it matches the frame size as encoded in
the extradata.

This avoids an indirection and will simplify implementation of encode2()

Gives a better estimate of buffer requirements and a better decision of
whether or not to use verbatim mode.

It is already set in avcodec_alloc_frame()

encoded ALAC does not have a fixed number of bits per sample

Prevents a division by zero later on.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

In bps == 3 case output is decoded directly.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This partially reverts commit 58dabf7bf2.
It is no longer necessary to use unaligned mov.
The swapped mov argument fix remains though.

The DC coefficient should be included, too.
This probably was missed because DC quantizer is always
even for MPEG-1/2 but this function is also used for MPEG-4.

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>