当前位置：首页 > news >正文

libaom 源码分析：码率控制超分辨率逻辑

news 2024/11/27 15:17:50

超分辨率

超分编码参数：在 aom_encoder.h 文件中结构体aom_codec_enc_cfg 中配置相关参数；

/*!\brief Frame super-resolution scaling mode.** Similar to spatial resampling, frame super-resolution integrates* upscaling after the encode/decode process. Taking control of upscaling and* using restoration filters should allow it to outperform normal resizing.*/aom_superres_mode rc_superres_mode;/*!\brief Frame super-resolution denominator.** The denominator for superres to use. If fixed it will only change if the* cumulative scale change over resizing and superres is greater than 1/2;* this forces superres to reduce scaling.** Valid denominators are 8 to 16.** Used only by AOM_SUPERRES_FIXED.*/unsigned int rc_superres_denominator;/*!\brief Keyframe super-resolution denominator.** The denominator for superres to use. If fixed it will only change if the* cumulative scale change over resizing and superres is greater than 1/2;* this forces superres to reduce scaling.** Valid denominators are 8 - 16 for now.*/unsigned int rc_superres_kf_denominator;/*!\brief Frame super-resolution q threshold.** The q level threshold after which superres is used.* Valid values are 1 to 63.** Used only by AOM_SUPERRES_QTHRESH*/unsigned int rc_superres_qthresh;/*!\brief Keyframe super-resolution q threshold.** The q level threshold after which superres is used for key frames.* Valid values are 1 to 63.** Used only by AOM_SUPERRES_QTHRESH*/unsigned int rc_superres_kf_qthresh;

超分控制参数：对应着超分编码参数，在 encoder.h 文件中结构体 SuperResCfg 控制超分相关编码参数；

superres_qthresh：这个字段指定在使用 AOM_SUPERRES_QTHRESH 模式时，应用于参考帧（inter frames）的量化器索引（qindex）阈值。这个阈值控制何时在参考帧上应用超分辨率。量化器索引控制质量与压缩之间的权衡，较低的
qindex 通常表示更高的质量和更大的比特率。
superres_kf_qthresh：与 superres_qthresh 类似，但用于关键帧（I帧）。该值控制超分辨率在关键帧上应用的阈值。关键帧对于视频播放至关重要，因为它们通常作为参考帧供后续的参考帧使用。
superres_scale_denominator：该字段定义了一个分数的分母，表示参考帧上采样前后超块宽度的比例。这个分数的分子是常量
SCALE_NUMERATOR。该字段决定了视频在进行超分辨率处理时，超块（superblock）维度的上采样程度。
superres_kf_scale_denominator：与 superres_scale_denominator 类似，但用于关键帧。它指定关键帧在上采样时的超块比例。
superres_mode：该字段表示编码器使用的超分辨率模式。aom_superres_mode 是一个枚举类型，定义了不同的超分辨率策略，例如不同的算法或缩放技术。
enable_superres：一个布尔标志，指示是否为整个视频序列启用超分辨率。如果该值为 true，则编码器将按照配置应用超分辨率；如果为 false，则不应用超分辨率。

/*!* \brief Encoder config related to frame super-resolution.*/
typedef struct {/*!* Indicates the qindex based threshold to be used when AOM_SUPERRES_QTHRESH* mode is used for inter frames.*/int superres_qthresh;/*!* Indicates the qindex based threshold to be used when AOM_SUPERRES_QTHRESH* mode is used for key frames.*/int superres_kf_qthresh;/*!* Indicates the denominator of the fraction that specifies the ratio between* the superblock width before and after upscaling for inter frames. The* numerator of this fraction is equal to the constant SCALE_NUMERATOR.*/uint8_t superres_scale_denominator;/*!* Indicates the denominator of the fraction that specifies the ratio between* the superblock width before and after upscaling for key frames. The* numerator of this fraction is equal to the constant SCALE_NUMERATOR.*/uint8_t superres_kf_scale_denominator;/*!* Indicates the Super-resolution mode to be used by the encoder.*/aom_superres_mode superres_mode;/*!* Flag to indicate if super-resolution should be enabled for the sequence.*/bool enable_superres;
} SuperResCfg;

超分辨率模式：通过 aom_superres_mode 枚举控制；

AOM_SUPERRES_NONE: 该模式禁用了所有帧的超分辨率处理。即，不应用任何超分辨率技术，编码的帧保持原始分辨率。
AOM_SUPERRES_FIXED: 所有帧都按照指定的固定比例进行编码并应用超分辨率。此模式下，超分辨率的比例是固定的，所有帧都会被缩放到相同的分辨率。
AOM_SUPERRES_RANDOM: 所有帧都按照随机比例进行编码并应用超分辨率。在此模式下，每一帧的超分辨率比例是随机的，意味着每一帧的分辨率可能不同。
AOM_SUPERRES_QTHRESH: 每一帧的超分辨率比例根据该帧的量化器索引（q index）来确定。这个模式是基于编码时的质量指标来动态调整超分辨率的比例，q index 控制着编码质量与压缩之间的权衡。
AOM_SUPERRES_AUTO: 在此模式下，编码器会自动选择是否使用超分辨率，并在使用超分辨率时自动选择超分辨率的比例。编码器根据每一帧的特性决定是否应用超分辨率，并且在应用时，自动决定合适的缩放比例。

/*!\brief Frame super-resolution mode. */
typedef enum {/**< Frame super-resolution is disabled for all frames. */AOM_SUPERRES_NONE,/**< All frames are coded at the specified scale and super-resolved. */AOM_SUPERRES_FIXED,/**< All frames are coded at a random scale and super-resolved. */AOM_SUPERRES_RANDOM,/**< Super-resolution scale for each frame is determined based on the q indexof that frame. */AOM_SUPERRES_QTHRESH,/**< Full-resolution or super-resolution and the scale (in case ofsuper-resolution) are automatically selected for each frame. */AOM_SUPERRES_AUTO,
} aom_superres_mode;

控制逻辑图：
set_encoder_config 函数中关于超分的配置，其中disable_superres函数用来设置未开启超分时的参数设置。

// Set Super-resolution mode configuration.if (extra_cfg->lossless || cfg->large_scale_tile) {disable_superres(superres_cfg);} else {superres_cfg->superres_mode = cfg->rc_superres_mode;superres_cfg->superres_scale_denominator =(uint8_t)cfg->rc_superres_denominator;superres_cfg->superres_kf_scale_denominator =(uint8_t)cfg->rc_superres_kf_denominator;superres_cfg->superres_qthresh =av1_quantizer_to_qindex(cfg->rc_superres_qthresh);superres_cfg->superres_kf_qthresh =av1_quantizer_to_qindex(cfg->rc_superres_kf_qthresh);if (superres_cfg->superres_mode == AOM_SUPERRES_FIXED &&superres_cfg->superres_scale_denominator == SCALE_NUMERATOR &&superres_cfg->superres_kf_scale_denominator == SCALE_NUMERATOR) {disable_superres(superres_cfg);