Number of planes is not always equal to the number of components even
for formats marked with PIX_FMT_PLANAR -- e.g. NV12 has three components
in two planes.
The total frame size is a combination of the 12 bits in the sequence
header and 2 more bits in the the sequence extension. While the
specification explicitly forbids the dimensions from the sequence header
from being 0 (thus ruling out multiples of 4096), such videos
apparrently exist in the wild so we should attempt to decode them.
Based on a patch by Michael Niedermayer <michaelni@gmx.at>
Fixes Bug 416.
To avoid allocating ridiculous amounts of memory for corrupted files,
read the input in chunks limited to filesize or an arbitrary large
amount when that is not known (chosen to be 50M).
The check `src > dst' in the form `&c->out[-back] > c->out' invokes
pointer overflow, which is undefined behavior in C.
Remove the check. Also replace `&c->out[-back] < c->out_start' with
a safe form `c->out - c->out_start < back' to avoid overflow.
CC: libav-stable@libav.org
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Simplify `p->buf > p->buf + p->buf_size - 4' as `p->buf_size < 4'.
Avoid a possible out-of-bounds pointer, which is undefined behavior
in C.
CC: libav-stable@libav.org
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
When `off' is 0, `0x537F6103 << 32' in the following expression invokes
undefined behavior, the result of which is not necessarily 0.
(0x537F6103 >> (off * 8)) | (0x537F6103 << (32 - (off * 8)))
Avoid oversized shifting.
CC: libav-stable@libav.org
Signed-off-by: Xi Wang <xi.wang@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
This will help in supporting old versions, e.g. version 3.93 uses the same
range coder but different predictor and version 3.82 uses different range
coder and predictor. Also this should not make decoding newer versions slower
by introducing additional checks on versions.
The tool nowadays supports more than one track per file,
this makes reading the code slightly less confusing.
Signed-off-by: Martin Storsjö <martin@martin.st>
AVCodecContext release_buffer() shall be NULL for audio codecs using
get_buffer. The backward compatibility code hence have to check before
calling it.