This can be used to add region of interest side data to video frames.

it is expected that there will be more files to support native mode,
so put all the dnn codes under libavfilter/dnn

The main change of this patch is to move the file location, see below:
modified:   libavfilter/Makefile
new file:   libavfilter/dnn/Makefile
renamed:    libavfilter/dnn_backend_native.c -> libavfilter/dnn/dnn_backend_native.c
renamed:    libavfilter/dnn_backend_native.h -> libavfilter/dnn/dnn_backend_native.h
renamed:    libavfilter/dnn_backend_tf.c -> libavfilter/dnn/dnn_backend_tf.c
renamed:    libavfilter/dnn_backend_tf.h -> libavfilter/dnn/dnn_backend_tf.h
renamed:    libavfilter/dnn_interface.c -> libavfilter/dnn/dnn_interface.c

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>

This avoids automatic conversions to rgb if scene change score is used in the expression.

Below is the tested results for the new added pixel format without autoscale to rgb24:
1. AV_PIX_FMT_YUVJ420P
time ./ffprobe -of compact=p=0 -show_entries frame=pkt_pts:frame_tags -bitexact -f lavfi \
"sws_flags=+accurate_rnd+bitexact;movie=../fate-suite/svq3/Vertical400kbit.sorenson3.mov,select=gt(scene\,.25)"

master:
pkt_pts=1620|tag:lavfi.scene_score=1.000000
pkt_pts=4140|tag:lavfi.scene_score=0.875036
pkt_pts=5800|tag:lavfi.scene_score=1.000000
pkt_pts=6720|tag:lavfi.scene_score=0.461625
pkt_pts=8160|tag:lavfi.scene_score=1.000000
pkt_pts=9760|tag:lavfi.scene_score=1.000000
pkt_pts=14080|tag:lavfi.scene_score=0.838916
pkt_pts=15700|tag:lavfi.scene_score=1.000000
pkt_pts=18500|tag:lavfi.scene_score=0.474948
pkt_pts=20040|tag:lavfi.scene_score=0.379700
pkt_pts=21760|tag:lavfi.scene_score=1.000000
./ffprobe -of compact=p=0 -show_entries frame=pkt_pts:frame_tags -bitexact -f  0.71s user 0.01s system 99% cpu 0.721 total

patch applied:
pkt_pts=1620|tag:lavfi.scene_score=1.000000
pkt_pts=4140|tag:lavfi.scene_score=0.668643
pkt_pts=5800|tag:lavfi.scene_score=0.996721
pkt_pts=6720|tag:lavfi.scene_score=0.357390
pkt_pts=8160|tag:lavfi.scene_score=0.886268
pkt_pts=9760|tag:lavfi.scene_score=0.926219
pkt_pts=14080|tag:lavfi.scene_score=0.650033
pkt_pts=15700|tag:lavfi.scene_score=1.000000
pkt_pts=18500|tag:lavfi.scene_score=0.316402
pkt_pts=20040|tag:lavfi.scene_score=0.269509
pkt_pts=21760|tag:lavfi.scene_score=1.000000
./ffprobe -of compact=p=0 -show_entries frame=pkt_pts:frame_tags -bitexact -f  0.19s user 0.01s system 81% cpu 0.240 total

2. AV_PIX_FMT_YUV420P
time ./ffprobe -of compact=p=0 -show_entries frame=pkt_pts:frame_tags -bitexact -f lavfi \
"sws_flags=+accurate_rnd+bitexact;movie=../Passengers_Breakfast_1080-sdr.mkv,select=gt(scene\,.2)"
master:
pkt_pts=3587|tag:lavfi.scene_score=0.462364
pkt_pts=4838|tag:lavfi.scene_score=0.419519
pkt_pts=6548|tag:lavfi.scene_score=0.397027
pkt_pts=9968|tag:lavfi.scene_score=0.419245
pkt_pts=12471|tag:lavfi.scene_score=0.413084
pkt_pts=16225|tag:lavfi.scene_score=0.506370
pkt_pts=19645|tag:lavfi.scene_score=0.507538
pkt_pts=22314|tag:lavfi.scene_score=0.504319
pkt_pts=24817|tag:lavfi.scene_score=0.417544
pkt_pts=25651|tag:lavfi.scene_score=0.413916
pkt_pts=26652|tag:lavfi.scene_score=0.487707
18.58s user 0.07s system 99% cpu 18.663 total

patch applied:
pkt_pts=3587|tag:lavfi.scene_score=0.272173
pkt_pts=4838|tag:lavfi.scene_score=0.247841
pkt_pts=6548|tag:lavfi.scene_score=0.233134
pkt_pts=9968|tag:lavfi.scene_score=0.247253
pkt_pts=12471|tag:lavfi.scene_score=0.244129
pkt_pts=16225|tag:lavfi.scene_score=0.302531
pkt_pts=19645|tag:lavfi.scene_score=0.303560
pkt_pts=22314|tag:lavfi.scene_score=0.301861
pkt_pts=24817|tag:lavfi.scene_score=0.249331
pkt_pts=25651|tag:lavfi.scene_score=0.247096
pkt_pts=26652|tag:lavfi.scene_score=0.287728
10.90s user 0.06s system 99% cpu 10.967 total

3. AV_PIX_FMT_YUV422P

time ./ffprobe -of compact=p=0 -show_entries frame=pkt_pts:frame_tags -bitexact -f lavfi \
"sws_flags=+accurate_rnd+bitexact;movie=../Passengers_Breakfast_1080-sdr.mkv,format=yuv422p,select=gt(scene\,.2)"
master:

patched applied:
pkt_pts=3587|tag:lavfi.scene_score=0.224017
pkt_pts=4838|tag:lavfi.scene_score=0.204225
pkt_pts=9968|tag:lavfi.scene_score=0.204636
pkt_pts=12471|tag:lavfi.scene_score=0.202772
pkt_pts=16225|tag:lavfi.scene_score=0.248765
pkt_pts=19645|tag:lavfi.scene_score=0.250144
pkt_pts=22314|tag:lavfi.scene_score=0.248802
pkt_pts=24817|tag:lavfi.scene_score=0.208362
pkt_pts=25651|tag:lavfi.scene_score=0.205777
pkt_pts=26652|tag:lavfi.scene_score=0.230742

4.  AV_PIX_FMT_YUV420P10
time ./ffprobe -of compact=p=0 -show_entries frame=pkt_pts:frame_tags -bitexact -f lavfi \
"sws_flags=+accurate_rnd+bitexact;movie=../Passengers_Breakfast_4k.mkv,select=gt(scene\,.2)"

master:
pkt_pts=3587|tag:lavfi.scene_score=0.269890
pkt_pts=4838|tag:lavfi.scene_score=0.248957
pkt_pts=6548|tag:lavfi.scene_score=0.234619
pkt_pts=9969|tag:lavfi.scene_score=0.224912
pkt_pts=12471|tag:lavfi.scene_score=0.225158
pkt_pts=16225|tag:lavfi.scene_score=0.289809
pkt_pts=19645|tag:lavfi.scene_score=0.285013
pkt_pts=22314|tag:lavfi.scene_score=0.280295
pkt_pts=24817|tag:lavfi.scene_score=0.206486
pkt_pts=25651|tag:lavfi.scene_score=0.208556
pkt_pts=26652|tag:lavfi.scene_score=0.249577
./ffprobe -of compact=p=0 -show_entries frame=pkt_pts:frame_tags -bitexact -f  76.03s user 0.22s system 99% cpu 1:16.27 total

patch applied
pkt_pts=3587|tag:lavfi.scene_score=0.269890
pkt_pts=4838|tag:lavfi.scene_score=0.248957
pkt_pts=6548|tag:lavfi.scene_score=0.234619
pkt_pts=9969|tag:lavfi.scene_score=0.224912
pkt_pts=12471|tag:lavfi.scene_score=0.225158
pkt_pts=16225|tag:lavfi.scene_score=0.289809
pkt_pts=19645|tag:lavfi.scene_score=0.285013
pkt_pts=22314|tag:lavfi.scene_score=0.280295
pkt_pts=24817|tag:lavfi.scene_score=0.206486
pkt_pts=25651|tag:lavfi.scene_score=0.208556
pkt_pts=26652|tag:lavfi.scene_score=0.249577
./ffprobe -of compact=p=0 -show_entries frame=pkt_pts:frame_tags -bitexact -f  50.27s user 0.20s system 99% cpu 50.476 total

5. AV_PIX_FMT_RGBA, AV_PIX_FMT_ABGR, AV_PIX_FMT_BGRA, AV_PIX_FMT_GRAY8
are tested by format= with the fate sample: Vertical400kbit.sorenson3.mov like below:
time ./ffprobe -of compact=p=0 -show_entries frame=pkt_pts:frame_tags -bitexact -f lavfi \
"sws_flags=+accurate_rnd+bitexact;movie=../fate-suite/svq3/Vertical400kbit.sorenson3.mov,format=rgba,select=gt(scene\,.25)"

patch applied:
pkt_pts=1620|tag:lavfi.scene_score=1.000000
pkt_pts=4140|tag:lavfi.scene_score=0.656277
pkt_pts=5800|tag:lavfi.scene_score=1.000000
pkt_pts=6720|tag:lavfi.scene_score=0.346218
pkt_pts=8160|tag:lavfi.scene_score=0.987686
pkt_pts=9760|tag:lavfi.scene_score=1.000000
pkt_pts=14080|tag:lavfi.scene_score=0.629187
pkt_pts=15700|tag:lavfi.scene_score=1.000000
pkt_pts=18500|tag:lavfi.scene_score=0.356211
pkt_pts=20040|tag:lavfi.scene_score=0.284775
pkt_pts=21760|tag:lavfi.scene_score=1.000000

6. AV_PIX_FMT_YUVJ422P
time ./ffprobe -of compact=p=0 -show_entries frame=pkt_pts:frame_tags -bitexact -f lavfi \
"sws_flags=+accurate_rnd+bitexact;movie=../fate-suite/svq3/Vertical400kbit.sorenson3.mov,format=yuvj422p,select=gt(scene\,.25)"

patch applied:
pkt_pts=1620|tag:lavfi.scene_score=0.838281
pkt_pts=4140|tag:lavfi.scene_score=0.541382
pkt_pts=5800|tag:lavfi.scene_score=0.780588
pkt_pts=6720|tag:lavfi.scene_score=0.298274
pkt_pts=8160|tag:lavfi.scene_score=0.699106
pkt_pts=9760|tag:lavfi.scene_score=0.730136
pkt_pts=14080|tag:lavfi.scene_score=0.537742
pkt_pts=15700|tag:lavfi.scene_score=0.811946
pkt_pts=18500|tag:lavfi.scene_score=0.263382
pkt_pts=21760|tag:lavfi.scene_score=0.880773

Reviewed-by: Marton Balint <cus@passwd.hu>
Signed-off-by: Limin Wang <lance.lmwang@gmail.com>

This patch does not make other pixel formats usable yet to make sure the test
result is the same with rgb32 format.

Reviewed-by: Marton Balint <cus@passwd.hu>
Signed-off-by: Limin Wang <lance.lmwang@gmail.com>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

support regions of interest sidedata

Signed-off-by: Jun Zhao <barryjzhao@tencent.com>

Fixes non-monotonous timestamps.

Fixes non-monotonous timestamps.

Fixes non-monotonous timestamps.

Fixes non-monotonous timestamps.

Previously it was used only for displaying frequency response.

The previous solution was very bad.

Next step after this one will be adding support for more window sizes.

Changes to vf_drawtext.c written by
Calvin Walton <calvin.walton@kepstin.ca>

Changes to filters.texi written by
greg Luce <electron.rotoscope@gmail.com>
with lots of help from Moritz Barsnick and Gyan

Fixes #7947.

Add sub-options for in_color_matrix/out_color_matrix

Signed-off-by: Jun Zhao <barryjzhao@tencent.com>

The linearize function (usually refered to EOTF) is the inverse of
delinearize function (usually referred to OETF). Demarcation point of
EOTF should be beta*delta, but the actual value used now in the source
code is beta.

For ITU Rec.709, they are 0.081 (0.018*4.5) and 0.018 respectively
(beta = 0.018 and delta = 4.5), and they correspond to pixel value 5
and 21 for an 8-bit image. Linearized result of pixel within that range
(5-21) will be different, but this commit will make linearize function
of the filter more accurate in the mathematical sense.

Signed-off-by: Yonglin Luo <vincenluo@tencent.com>
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>

Fixes non-monotonous timestamps.

Fixes non-monotonous timestamps.

Fixes non-monotonous timestamps.

Fixes non-monotonous timestamps.

Fixes non-monotonous timestamps.

low_power mode will use a fixed HW engine (SFC), thus can offload EU usage.
high quality mode will take EU usage (AVS sampler).

Performance and EU usage (Render usage) comparsion on Intel(R) Xeon(R) CPU E3-1225 v5 @ 3.30GHz:

High quality mode : ffmpeg -hwaccel qsv -c:v h264_qsv -i bbb_sunflower_1080p_30fps_normal_2000frames.h264 \
-vf scale_qsv=w=1280:h=736:mode=hq -f null -
fps=389
RENDER usage: 28.10 (provided by MSDK metrics_monitor)

Low Power mode: ffmpeg -hwaccel qsv -c:v h264_qsv -i ~/bbb_sunflower_1080p_30fps_normal_2000frames.h264 \
-vf scale_qsv=w=1280:h=736:mode=low_power -f null -
fps=343
RENDER usage: 0.00

Low power mode (SFC) may be disabled if not supported by
MSDK/Driver/HW, and replaced by AVS mode interanlly.

Signed-off-by: Zhong Li <zhong.li@intel.com>

Redundant condition: '!A || B' is equivalent to '!A || (A && B)' but
more clearly.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>

1. Currently output format is hard-coded as NV12, thus means
   CSC is always done for not NV12 input such as P010.
   Follow original input format as default output.
2. Add an option to specify output format.

Signed-off-by: Zhong Li <zhong.li@intel.com>

Fixes compilation on x86_32

Signed-off-by: James Almer <jamrial@gmail.com>

The horizontal pass get ~2x performance with the patch
under single thread.

Tested overall performance using the command(avx2 enabled):
./ffmpeg -i 1080p.mp4 -vf gblur -f null /dev/null
./ffmpeg -i 1080p.mp4 -vf gblur=threads=1 -f null /dev/null
For single thread, the fps improves from 43 to 60, about 40%.
For multi-thread, the fps improves from 110 to 130, about 20%.

Signed-off-by: Ruiling Song <ruiling.song@intel.com>

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

Signed-off-by: Zhong Li <zhong.li@intel.com>

extra_hw_frames is needed for some cases.
(eg: qsv decoding + qsv vpp + qsv look_ahead encoding, transcoding
 failed if no extra_hw_frames supported:
 ffmpeg -hwaccel qsv -c:v h264_qsv -i bbb_sunflower_1080p_30fps_normal_2000frames.mp4 \
-vf vpp_qsv=w=1280:h=720:extra_hw_frames=100 -v verbose  -c:v h264_qsv \
-look_ahead 1 -look_ahead_depth 100 out1.mp4
)

Signed-off-by: Zhong Li <zhong.li@intel.com>