Browse Source

softfloat: handle INT_MIN correctly in av_int2sf

Otherwise v=INT_MIN doesn't get normalized and thus triggers av_assert2
in other functions.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
tags/n3.0
Andreas Cadhalpun 9 years ago
parent
commit
9ac61e73d0
1 changed files with 6 additions and 1 deletions
  1. +6
    -1
      libavutil/softfloat.h

+ 6
- 1
libavutil/softfloat.h View File

@@ -153,7 +153,12 @@ static inline av_const SoftFloat av_sub_sf(SoftFloat a, SoftFloat b){
* @returns a SoftFloat with value v * 2^frac_bits
*/
static inline av_const SoftFloat av_int2sf(int v, int frac_bits){
return av_normalize_sf((SoftFloat){v, ONE_BITS + 1 - frac_bits});
int exp_offset = 0;
if(v == INT_MIN){
exp_offset = 1;
v>>=1;
}
return av_normalize_sf(av_normalize1_sf((SoftFloat){v, ONE_BITS + 1 - frac_bits + exp_offset}));
}

/**


Loading…
Cancel
Save