Signed-off-by: Martin Storsjö <martin@martin.st>

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>

Also try to free local allocations on errors.

Signed-off-by: Martin Storsjö <martin@martin.st>

CC: libav-stable@libav.org

Signed-off-by: Diego Biurrun <diego@biurrun.de>

Signed-off-by: Diego Biurrun <diego@biurrun.de>

This makes sure that linesize * start_y doesn't overflow, so that
emulated_edge_mc can get back the original value if needed.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>

The code tries to decode a number of channels at the
offset given by the ff_alac_channel_layout_offsets table.
Even if the number of channels decoded so far doesn't
exceed the total number of channels, we need to check that
we actually can decode that number of channels at this offset
as well.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>

Otherwise buffer size calculations in allocate_buffers could
overflow later, making the code think a large enough buffer
actually was allocated.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>

Fixes invalid reads.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Otherwise picmemset can get called with negative y, resulting in an
invalid write.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Fixes invalid reads with corrupted files.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Martin Storsjö <martin@martin.st>

Remove the header decoding for PCM audio from mpeg.c and the
20/24bit parts from pcm.c and merge them into a new decoder in
pcm-dvd.c.

The decoder has added support for samples that span multiple
packets and modified 20/24bit group decoding. Both is needed to
decode samples that have been generated with DVD-Lab Pro 2. The
decoding of 16bit PCM and two channel 24bit is identical to
before. No other samples are known to verify the correctness of
the encoding this software does.
The complete list of tested formats is
48kHz/16bit/2-8 channels
48kHz/24bit/2-5 channels
96kHz/16bit/2-4 channels
96kHz/24bit/2 channels

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

The original idea was to collect PCM codecs that could appear in various
MPEG streams in this file. Discussion in IRC lead to the conclusion that
one codec per file would be better and stop the need for #ifdefs.

Also give consistent names to init functions.

Only consume an AVPacket when all the samples have been read.

When the rate of samples output is limited (by the default value
of max_samples), consuming the first packet immediately will cause
timing problems:

- The first packet with PTS 0 will output 4608 samples and be
consumed entirely
- The second packet with PTS 64 will output the remaining samples
(typically, a lot, that's why max_samples exist) until the decoded
samples of the first packet have been exhausted, at which point the
samples of the second packet will be decoded and output when
av_decode_frame is called with the next packet).

That means there's a PTS jump since the first packet is 'decoded'
immediately, which can be seen with avplay or mplayer: the timing
jumps immediately to 6.2s (which is the size of a packet).

Sample: http://streams.videolan.org/issues/6348/Goldwave-MAClib.ape
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

The parsing function return AVERROR and AAC_AC3_PARSE_ERROR values,
make sure they are not misunderstood.

This both allows factoring out size check for both MetaSound and TwinVQ-VQF
decoders and fixes the situation when there are several MetaSound frames
stuffed together (that happens in 8kHz @ 8kbps MetaSound in ASF for example).

This ensures that decode_cabac_residual_internal actually does get inlined,
which it otherwise does not, even though it is marked as always_inline.

The functions are used by all codecs that enable the h264chroma component
and the file is already compiled conditional on h264chroma being enabled.

Most of our VP56 optimizations are VP6-only and will stay that way.
So avoid compiling them for VP5-only builds.