Also add an additional sanity check to the alt_quant table.
Fixes invalid reads with corrupted files.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org

Also improve some error messages.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org

Fixes invalid reads for corrupted files.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org

They can be different if the last keyframe failed to decode correctly.
Fixes possible invalid reads in such a case.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org

This way, the special IDCT permutations are no longer needed. This
is similar to how H264 does it, and removes the dsputil dependency
imposed by the scantable code.

Also remove the unused type == 0 cases from the plain C version
of the idct.

Signed-off-by: Martin Storsjö <martin@martin.st>

While this change isn't bitexact, the IDCTs weren't bitexact to
start with either.

This simplifies decoupling vp3 from dsputil.

Signed-off-by: Martin Storsjö <martin@martin.st>

Signed-off-by: Martin Storsjö <martin@martin.st>

Signed-off-by: Martin Storsjö <martin@martin.st>

Signed-off-by: Martin Storsjö <martin@martin.st>

It is only used for error resilience. This allows building the
h264 decoder without dsputil, if error resilience is disabled.

Signed-off-by: Martin Storsjö <martin@martin.st>

The non-intra-pcm branch in hl_decode_mb (simple, 8bpp) goes from 700
to 672 cycles, and the complete loop of decode_mb_cabac and hl_decode_mb
(in the decode_slice loop) goes from 1759 to 1733 cycles on the clip
tested (cathedral), i.e. almost 30 cycles per mb faster.

Signed-off-by: Martin Storsjö <martin@martin.st>

Signed-off-by: Martin Storsjö <martin@martin.st>

Put a copy of the 8bit functions only in dsputil, where they are
used for some other things (e.g. mpeg4qpel, mspel, cavsqpel).

Signed-off-by: Martin Storsjö <martin@martin.st>

The pointers that get assigned ff_cropTbl were made const in
9e0f14f1, but other variables that transitively are assigned
based on these variables were missed.

Signed-off-by: Martin Storsjö <martin@martin.st>

These are widely used throughout libavcodec, nothing dsputil-specific.

Change ff_cropTbl to a statically initialized table, to avoid
initializing it with a function call.

Signed-off-by: Martin Storsjö <martin@martin.st>

There's no point in these pointers not being const.

Signed-off-by: Martin Storsjö <martin@martin.st>

This makes the vp3 decoder less dependent on dsputil, and will aid
in making it (eventually) dsputil-independent.

Signed-off-by: Martin Storsjö <martin@martin.st>

This way, they can be shared between mpeg4qpel and h264qpel without
requiring either one to be compiled unconditionally.

Signed-off-by: Martin Storsjö <martin@martin.st>

Signed-off-by: Martin Storsjö <martin@martin.st>

In the non-bitexact mode, vp3 currently decodes to the same
frame crcs as before 28f9ab702 (and the output visually looks
correct).

Signed-off-by: Martin Storsjö <martin@martin.st>

Timing on Arrandale:
        C   SSE
Win32:  57   44
Win64:  47   38
Unrolling and not storing mask both save some cycles.

Signed-off-by: Diego Biurrun <diego@biurrun.de>

There is no arch-optimized version of the H.261 loop filter and there
likely will never be, so the dsputil overhead does not give any benefit.

Signed-off-by: Diego Biurrun <diego@biurrun.de>

It can be 0 or -1 for invalid files, which may result in invalid memory
access.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

This can happen when the number of skipped lines is not consistent with
the number of coded lines.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Peter Meerwald <p.meerwald@bct-electronic.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>

CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

Signed-off-by: Anton Khirnov <anton@khirnov.net>

This fixes standalone compilation of the msmpeg4v2, msmpeg4v3
and wmv2 encoders, that previously failed to link due to the
decoder codepaths requiring error_resilience.

Signed-off-by: Martin Storsjö <martin@martin.st>