Choose what types of reference frames will be used based on what types
are available, and make the intra-only mode explicit (GOP size one,
which must be used for MJPEG).

This was added in libva 2.1.0 (VAAPI 1.1.0).  Use AVCodecContext.qmax,
matching the existing behaviour for qmin, and clean up the defaults so
that we only pass min/max when explicitly set.

Query which modes are supported and select between VBR and CBR based
on that - this removes all of the codec-specific rate control mode
selection code.

The only common option here is low_power - it was previously supported
for H.264 only, that specific option is removed.

Previously there was one fixed choice for each codec (e.g. H.265 -> Main
profile), and using anything else then required an explicit option from
the user.  This changes to selecting the profile based on the input format
and the set of profiles actually supported by the driver (e.g. P010 input
will choose Main 10 profile for H.265 if the driver supports it).

The entrypoint and render target format are also chosen dynamically in the
same way, removing those explicit selections from the per-codec code.

The codec-specific context now contains both the common context and the
codec-specific options directly.

This will make it easier to support options in common between different
encoders.  It also cleans up some of the field naming.

We may check priv->sei_needed only when writing SEI messages.

Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>

This makes it easier for users of the CBS API to get alloc/free right -
all subelements use the buffer API so that it's clear how to free them.
It also allows eliding some redundant copies: the packet -> fragment copy
disappears after this change if the input packet is refcounted, and more
codec-specific cases are now possible (but not included in this patch).

... instead of making callers allocate it themselves.  This is more
consistent with other APIs in libav.

Explicitly identify decoder/encoder wrappers with a common name. This
saves API users from guessing by the name suffix. For example, they
don't have to guess that "h264_qsv" is the h264 QSV implementation, and
instead they can just check the AVCodec .codec and .wrapper_name fields.

Explicitly mark AVCodec entries that are hardware decoders or most
likely hardware decoders with new AV_CODEC_CAPs. The purpose is allowing
API users listing hardware decoders in a more generic way. The proposed
AVCodecHWConfig does not provide this information fully, because it's
concerned with decoder configuration, not information about the fact
whether the hardware is used or not.

AV_CODEC_CAP_HYBRID exists specifically for QSV, which can have software
implementations in case the hardware is not capable.

Based on a patch by Philip Langdale <philipl@overt.org>.

Merges Libav commit 47687a2f8a.

Explicitly identify decoder/encoder wrappers with a common name. This
saves API users from guessing by the name suffix. For example, they
don't have to guess that "h264_qsv" is the h264 QSV implementation, and
instead they can just check the AVCodec .codec and .wrapper_name fields.

Explicitly mark AVCodec entries that are hardware decoders or most
likely hardware decoders with new AV_CODEC_CAPs. The purpose is allowing
API users listing hardware decoders in a more generic way. The proposed
AVCodecHWConfig does not provide this information fully, because it's
concerned with decoder configuration, not information about the fact
whether the hardware is used or not.

AV_CODEC_CAP_HYBRID exists specifically for QSV, which can have software
implementations in case the hardware is not capable.

Based on a patch by Philip Langdale <philipl@overt.org>.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>

This should refer to the existing SPS structure, not the VAAPI sequence
parameter buffer (which is not yet initialised).

From ffmpeg commit f31478ba14.

Signed-off-by: Mark Thompson <sw@jkqxz.net>

SEI headers should be inserted as generic raw data (the old specific
type has been deprecated in libva2).

With pre-2.0 Intel drivers in CBR mode, if an explicit SEI message with
the old (now deprecated) type is not included, the driver generates and
inserts some timing SEI which is almost certainly invlaid.  Before
7a4fac5e91 we always inserted our own SEI
so this would not be visible, but since then it has been possible to
disable that.  We would also like to avoid using the deprecated type,
and using the new type, while working in old drivers, does not suppress
the spurious message like the old type does.

Therefore, suppress the bad SEI insertion by providing a zero-length
buffer with the old type, which the driver can insert harmlessly.

This has been deprecated in libva2 because hardware does not and will not
support it.  Therefore never consider it for decode, and for encode assume
the user meant constrained baseline profile instead.

This should refer to the existing SPS structure, not the VAAPI sequence
parameter buffer (which is not yet initialised).

Signed-off-by: Jun Zhao <jun.zhao@intel.com>
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
Signed-off-by: Mark Thompson <sw@jkqxz.net>

SEI headers should be inserted as generic raw data (the old specific
type has been deprecated in libva2).

Fixes CID 1419829.

This should be ticks, not time_scale steps - it was wrong for all
framerates not a multiple of 1/2.

(cherry picked from commit 30645174e3)

Included by default with non-IDR intra frames.

(cherry picked from commit a49ee60d5f)

Adds a new private option to enable them (off by default).

(cherry picked from commit 820a4483af)

(cherry picked from commit 7a4fac5e91)

This has been deprecated in libva2 because hardware does not and will not
support it.  Therefore never consider it for decode, and for encode assume
the user meant constrained baseline profile instead.

This avoids confusion with equivalent H.265 SEI values when both are
being used at the same time.

(cherry picked from commit 6ea220cbee)

This should be ticks, not time_scale steps - it was wrong for all
framerates not a multiple of 1/2.

Follow libx264 style to support "coder" option, and set it to
cabac by default.

Signed-off-by: Yi A Wang <yi.a.wang@intel.com>
Signed-off-by: Jun Zhao <jun.zhao@intel.com>
Reviewed-by: Steven Liu <lingjiujianke@gmail.com>
Signed-off-by: Mark Thompson <sw@jkqxz.net>

Use AVCodecContext.compression_level rather than a private option,
replacing the H.264-specific quality option (which stays only for
compatibility).

This now works with the H.265 encoder in the i965 driver, as well as
the existing cases with the H.264 encoder.

(cherry picked from commit 19388a7200)

Included by default with non-IDR intra frames.

Adds a new private option to enable them (off by default).

Use AVCodecContext.compression_level rather than a private option,
replacing the H.264-specific quality option (which stays only for
compatibility).

This now works with the H.265 encoder in the i965 driver, as well as
the existing cases with the H.264 encoder.

The non-H.26[45] codecs already use this form.  Since we don't
currently generate I frames for codecs which support them separately
to IDR, the p_per_i variable is set to infinity by default so that it
doesn't interfere with any other calculation.  (All the code for I
frames still exists, and it works for H.264 if set manually.)

(cherry picked from commit 6af014f402)

This avoids confusion with equivalent H.265 SEI values when both are
being used at the same time.

The non-H.26[45] codecs already use this form.  Since we don't
currently generate I frames for codecs which support them separately
to IDR, the p_per_i variable is set to infinity by default so that it
doesn't interfere with any other calculation.  (All the code for I
frames still exists, and it works for H.264 if set manually.)

Default to using VBR when a target bitrate is set, unless the max rate
is also set and matches the target.  Changes to the Intel driver mean
that min_qp is also respected in this case, so set a codec default to
unset the value rather than using the current default inherited from
the MPEG-4 part 2 encoder.

(cherry picked from commit eddfb57210)

Before this change, it was possible to overflow pic_order_cnt_lsb and
generate a stream with invalid POC numbering.  This makes sure that
the field is large enough that a single IDR B* P sequence uses fewer
than half the available POC lsb values.

(cherry picked from commit 89725a8512)

Default to using VBR when a target bitrate is set, unless the max rate
is also set and matches the target.  Changes to the Intel driver mean
that min_qp is also respected in this case, so set a codec default to
unset the value rather than using the current default inherited from
the MPEG-4 part 2 encoder.

Before this change, it was possible to overflow pic_order_cnt_lsb and
generate a stream with invalid POC numbering.  This makes sure that
the field is large enough that a single IDR B* P sequence uses fewer
than half the available POC lsb values.

Disable B frames when using baseline/constrained baseline profile,
following H.264 spec Annex A.2.1.

Signed-off-by: Jun Zhao <jun.zhao@intel.com>
Signed-off-by: Yi A Wang <yi.a.wang@intel.com>
Signed-off-by: Mark Thompson <sw@jkqxz.net>

In H.264 section 8.2.1, we have that "The bitstream shall not contain
data that result in Min(TopFieldOrderCnt, BottomFieldOrderCnt) not
equal to 0 for a coded IDR frame".  This fixes the encoder to always
conform to this - previously the POC values formed an unbroken
sequence, not resetting to zero on IDR frames.

Signed-off-by: Mark Thompson <sw@jkqxz.net>

Same as e0df56f25d.  This was accidentally
reintroduced while merging c8241e730f.

when meeting IDR frame, vaapi_encode_h264 poc number don't reset, now fix
this issue based on h264 spec. Some decoder don't care this case, but this
fix will enhance the encoder action. Before this fix, poc number is
negative in some case.

Reviewed-by: Jun Zhao <jun.zhao@intel.com>
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
Signed-off-by: Mark Thompson <sw@jkqxz.net>

(cherry picked from commit ec17ab381e)

(cherry picked from commit 17a0f9481c)

(cherry picked from commit 956a54129d)