Browse Source

dnxhdenc: fixed signed multiplication overflow

The low 32 bits of a multiplication are the same for signed
and unsigned operands.  Casting to unsigned before multiplying
is thus equivalent while avoiding signed overflow, which is
undefined by the C99 standard.

Signed-off-by: Mans Rullgard <mans@mansr.com>
tags/n0.9
Mans Rullgard 14 years ago
parent
commit
d341d5fd2c
1 changed files with 1 additions and 1 deletions
  1. +1
    -1
      libavcodec/dnxhdenc.c

+ 1
- 1
libavcodec/dnxhdenc.c View File

@@ -623,7 +623,7 @@ static int dnxhd_mb_var_thread(AVCodecContext *avctx, void *arg, int jobnr, int
for (mb_x = 0; mb_x < ctx->m.mb_width; ++mb_x, pix += 16) {
unsigned mb = mb_y * ctx->m.mb_width + mb_x;
int sum = ctx->m.dsp.pix_sum(pix, ctx->m.linesize);
int varc = (ctx->m.dsp.pix_norm1(pix, ctx->m.linesize) - (((unsigned)(sum*sum))>>8)+128)>>8;
int varc = (ctx->m.dsp.pix_norm1(pix, ctx->m.linesize) - (((unsigned)sum*sum)>>8)+128)>>8;
ctx->mb_cmp[mb].value = varc;
ctx->mb_cmp[mb].mb = mb;
}


Loading…
Cancel
Save