Found-by: jamrial
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5782e0ba8c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: runtime error: signed integer overflow: 4096 - -2147483648 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 89325417e7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Paul B Mahol <onemda@gmail.com>

This silences missing EOC marker when decoding j2k files with Psot set to 0.

Signed-off-by: Paul B Mahol <onemda@gmail.com>

Fixes out of array read
Fixes: j2k-poc.bin

Found-by: Lucas Leong <wmliang.tw@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

The profiles are a property of the codec, so it makes sense to export
them through AVCodecDescriptors, not just the codec implementations.

Fixes: undefined shift
Fixes: c409ef86f892335a0a164b5871174d5a/asan_heap-oob_1dff564_2159_162b7234616deab02b544410455eb07b.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: 03e0abe721b1174856d41a1eb5d6a896/signal_sigabrt_7ffff6ae7cc9_3813_e71bf3541abed3ccba031cd5ba0269a4.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes potential integer overflows
Fixes: 03e0abe721b1174856d41a1eb5d6a896/signal_sigabrt_7ffff6ae7cc9_3813_e71bf3541abed3ccba031cd5ba0269a4.avi

This fix is choosen to be simple to backport, better solution
for master is planed

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: 0231a17345734228011c6f35a64e4594/asan_heap-oob_1d92a72_3218_1213809a9e3affec77e4c191fdfdc0a9.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes out of array access
Fixes: b877a6b788a25c70e8b1d014f8628549/asan_heap-oob_1da2c3f_2324_5a1b329b0b3c4bb6b1d775660ac56717.r3d

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: Ticket4878

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes: CID1322305 and CID1322304

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Move the packet-specific parsers in separate functions.

That loop is completely stand-alone.

That loop is completely stand-alone.

Matches the openjpeg behaviour.

"Fixes" CID1322361

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Fixes CID1322304, CID1322305

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Express bitfields more simply.

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

Convert doxygen to multiline and express bitfields more simply.

Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

not just at the end

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes: Ticket4677

Solution based on information from
https://code.google.com/p/openjpeg/issues/detail?id=80

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes Ticket2586

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes Ticket4657

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Thats what the spec says should be done
aka iterate in the wrong space and use special cases to patch that up.
It sometimes can result in different order

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

The order in j2k is not the simple and logic one

Fixes Ticket4670

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

These will be needed in subsequent commits

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This also reduces the amount of memory needed
Fixes Ticket4672

The new code seems slightly faster as well, probably due to better cache usage

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes infinite loop
Fixes Ticket4663

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes Ticket4649

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Like the 5/3 case this is needed to avoid overflows and similarly for 16bpp
output pure 32bit operations are insufficient if high quality is wanted

Note, this code-path is only used in bitexact mode, so this should not
affect the speed of any real use-case

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This avoids a multiplication and 64bits

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This fixes overflows, using fewer bits would impact high bit depth quality

Fixes Ticket4654

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

The condition is impossible now.
the TPsot type is changed to unsigned int to prevent gcc from producing warnings

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

This is the maximum that the syntax of jpeg2000 is capable to store

Fixes Ticket4639

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Fixes Ticket4635

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>