Buyer Toolkit — Evaluate, Compare, Decide

How Cires21 is turning video benchmarking into a discipline worth trusting

Why Measuring Video Is Hard

Video encoding is one of those domains where complexity hides behind familiar metrics. A bitrate number seems straightforward. A VMAF score looks like it settles the quality question. But both are abstractions layered on top of a system with dozens of interacting variables, and each abstraction strips away context that matters.

Consider what happens during a live encoding session. The encoder must make frame-by-frame decisions under real-time constraints, adapting to scene changes, motion intensity, and content complexity. It must do this while managing thermal limits, power budgets, and latency requirements that shift depending on the delivery target. A benchmark that captures one of these dimensions while ignoring the others is not incomplete. It is misleading.

The Measurement Complexity Stack

Exhibit 1: Each layer of the measurement stack introduces variables that simpler benchmarks tend to ignore.

 

Cires21’s C21 Live Encoder, which supports codecs ranging from H.264 and HEVC to AV1, VP9, and MPEG-2 across protocols including HLS, DASH, CMAF, SRT, and WebRTC, has to perform under all of these conditions simultaneously. That breadth of real-world deployment experience is precisely what led the company to question whether the industry’s measurement conventions were keeping pace with the technology they were meant to evaluate.

The Limits of Headline Metrics

The most common benchmarking pitfall is what might be called the “single-number fallacy.” An encoder achieves a VMAF score of 95 at a given bitrate. Another achieves 93. The first must be better. But this reasoning collapses under scrutiny when you consider what was left unmeasured.

Was the content representative of actual production workloads, or was it selected to flatter one architecture? Were the encoders configured with equivalent optimization effort, or did one receive weeks of tuning while the other ran with defaults? Was power consumption tracked per stream, or was the total system draw buried in a footnote? Did the quality metric capture tail performance (the worst 5% of frames that viewers actually notice), or did a generous average mask inconsistency?

KEY INSIGHT
A benchmark that answers only the question it was designed to ask may still mislead the person using it to make a different decision entirely. The gap between “what was measured” and “what the operator needs to know” is where most bad infrastructure investments originate.

This is not a theoretical concern. Cires21’s engineering team, led by CTO David Gonzalez, has documented specific cases where standard industry benchmarks produced rankings that reversed under real-world operating conditions. An encoder that won a controlled lab comparison consumed three times the power per stream in continuous 24/7 operation. A codec configuration that excelled on test clips produced unacceptable quality variance on broadcast content with frequent scene changes.

Efficiency as a System Property

One of the most important conceptual shifts in Cires21’s approach is treating efficiency not as a feature of individual components but as a property of the entire system. A chip that encodes streams at low wattage is only efficient if it maintains quality, handles the required density, and operates within the thermal envelope of a production rack. Efficiency is the relationship between all of these variables, not a number that lives on a datasheet.

This perspective becomes especially important when evaluating purpose-built silicon like VPUs (Video Processing Units) against general-purpose GPUs. The two architectures make fundamentally different engineering trade-offs, and comparing them requires a framework that accounts for those differences rather than ignoring them.

Cires21 confronted this directly when it began integrating NETINT Quadra T1U VPUs into the C21 Live Encoder. The encoder had evolved through three generations of acceleration: a CPU era where software encoding dominated, a GPU era that brought density improvements at higher power costs, and beginning in 2024, a VPU era that promised a different set of trade-offs entirely. Each transition demanded new measurement approaches because the previous generation’s benchmarks could not fairly evaluate the new architecture

Building a Better Benchmark

Rather than accepting industry convention, Cires21 developed a structured methodology for designing what it calls “fair benchmarks for heterogeneous video compute.” The framework centers on four principles that sound obvious in theory but are rarely followed in practice.

Designing Fair Benchmarks for Heterogeneous Video Compute

Exhibit 2: Cires21’s four-pillar methodology for designing benchmarks that hold up under operational scrutiny.

First, objective definition. Before any test runs, the team establishes precisely what question the benchmark is designed to answer. “Which encoder is better?” is not a valid objective. “Which architecture delivers the lowest cost per stream at broadcast quality under 24/7 operation?” is. The specificity forces everyone involved to confront the trade-offs before the numbers are generated rather than after.

Second, normalization and controls. Every comparison must use equivalent codec settings, equivalent optimization effort, and equivalent operational conditions. When Cires21 benchmarked the NETINT Quadra T1U against the NVIDIA RTX 4000 Ada, both platforms received the same tuning methodology applied by engineers with comparable expertise on each architecture. The VisualON Optimizer was integrated to ensure perceptual quality optimization was consistent across platforms.

Third, content selection. Test clips must represent the diversity of actual production workloads: high-motion sports content, talking-head interviews, graphics-heavy broadcasts, and low-light footage. Using only friendly content (static scenes, low complexity) flatters every encoder and tells the operator nothing about behavior under stress.

Fourth, metric selection. Cires21 insists on percentile-based quality reporting rather than averages. A mean VMAF of 95 can hide a long tail where 5% of frames fall below 85, which is exactly the kind of artifact that viewers notice and complain about. The 5th percentile VMAF score is often a more honest indicator of viewer experience than the mean.

What the Numbers Revealed

When Cires21 applied this methodology to its VPU evaluation, the results were instructive. The joint whitepaper with NETINT, titled “More Streams, Less Power: Energy Trade-offs,” benchmarked the Quadra T1U against the NVIDIA RTX 4000 Ada across H.264, HEVC, and AV1 encoding workloads.

Cires 21 + NETINT Whitepaper: VPU vs GPU Benchmark results

Exhibit 3: Benchmark results from the Cires21-NETINT whitepaper across energy, operations, and quality dimensions

The headline finding was striking: VPUs delivered 4.7 times greater energy efficiency, consuming between 0.4 and 0.7 watts per stream compared to significantly higher consumption on the GPU platform. But the methodology’s value showed in the supporting data. Quality was not sacrificed for efficiency. VMAF scores remained comparable across both architectures and all three codecs. The benchmarks included percentile distributions, not just averages, confirming that tail quality was maintained.

At the operational level, the numbers translated into concrete deployment advantages: a 75% reduction in rack space, a 50% decrease in power consumption, and a fourfold increase in encoding density. These are the metrics that infrastructure planners actually use to model total cost of ownership, and Cires21 published them alongside the raw performance data so operators could map the results to their own environments.

THE BIGGER PICTURE
Efficiency benchmarks only earn trust when they are accompanied by the full measurement context: content diversity, quality percentiles, power methodology, and documentation sufficient for an independent team to reproduce the results. Cires21’s published methodology meets this standard.

From Benchmarks to Deployment Decisions

The real test of any measurement framework is whether it survives contact with production. Cires21’s experience deploying VPU-based encoding across its customer base suggests that the benchmarks held. The C21 Live Encoder now operates in environments ranging from national broadcasting (RTVE, TV3, EITB) to sports streaming (Dorna Sports, Real Madrid) to telecommunications infrastructure (Telefonica, Cellnex Telecom), and the transition from GPU to VPU acceleration has proceeded without the quality regressions that poorly characterized technology transitions typically produce.

This matters because the video industry is entering a period of significant infrastructure renewal. The migration to AV1, the growth of live-to-VOD workflows through tools like Cires21’s C21 Live Editor, the expansion of AI-driven content processing through platforms like MediaCopilot, and the continued push toward lower latency via WebRTC (WHIP/WHEP) all place increasing demands on encoding infrastructure. Operators making five-year capital investments need benchmarks they can trust, not benchmarks that were designed to close a sale.

Common Pitfalls and How to Avoid Them

Cires21’s methodology documentation identifies several measurement traps that are worth highlighting because they recur across the industry.

The optimization parity problem: When comparing two architectures, it is common for the incumbent to receive extensive tuning while the challenger runs with minimal optimization. This produces results that reflect the effort invested in tuning, not the inherent capability of the hardware. Cires21’s protocol requires that each platform receive equivalent engineering attention, documented and auditable.

The friendly-content trap: Benchmarks that use only low-complexity test sequences overstate the performance of every encoder and obscure the differences that matter most. Production content is unpredictable, and measurement must reflect that unpredictability. Cires21 mandates a content mix that includes challenging material like sports with fast camera motion, concerts with strobe lighting, and news tickers with sharp text overlays.

The averages-vs-percentiles blindspot: A mean VMAF of 95 across a session can coexist with individual frames scoring below 80. Viewers do not experience averages. They experience the worst frames. Any quality claim that reports only mean scores without 5th or 1st percentile data is incomplete, and Cires21’s framework treats it as such.

The power-measurement gap: Many benchmarks omit power data entirely or report it at the wall level without isolating the encoding workload. Cires21’s methodology requires per-stream power measurement under sustained load, because power consumption often increases non-linearly as density rises and thermal management becomes a factor.

Measurement as a Competitive Advantage

There is an emerging argument that the organizations with the best measurement practices will make the best infrastructure decisions, and that this advantage compounds over time. Operators who invest in rigorous evaluation avoid expensive technology bets that underperform in production. They build vendor relationships grounded in transparency rather than marketing claims. They create internal knowledge that makes each subsequent decision faster and better informed.

Cires21’s trajectory illustrates this pattern. By publishing its benchmarking methodology rather than treating it as proprietary, the company has positioned itself as a trusted evaluation partner, not just a technology vendor. The video industry does not lack for performance data. What it lacks is performance data that operators can trust, contextualize, and reproduce. Cires21’s contribution is not simply another set of benchmark numbers. It is a framework for deciding which numbers deserve to be believed so operators can get the technical results they need.

This article is part of a series examining how purpose-built video processing is reshaping streaming infrastructure. Cires21 (cires21.com) is a video technology company founded in 2008 in Madrid, Spain, providing encoding, delivery, and AI-powered solutions for broadcast and streaming operators across Europe and the Americas.

 

ACCESS NOW:  ASIC-Based Transcoding
for High-volume Use Cases
Including social media, broadcast, interactive platforms, and service providers

ACCESS NOW

You have three basic approaches to transcoding – CPU-only, GPU, and ASICs. Kenneth outlines the pros and cons of each with extensive throughput, CAPEX, and OPEX examples.

If you’ve ever benchmarked software codecs, you know the quality/throughput tradeoff; simply stated, the higher the quality, the lower the throughput. In contrast, for many first-generation hardware encoders, throughput was prioritized, but the quality was fixed; you got what you got.

Most next-gen hardware encoders offer presets or other switches to optimize quality at a cost to throughput that can be even more striking than with software. In comparing specifications for encoders, remember the quality/throughput tradeoff. And when you see quality stats, think, “hmm, at what throughput?” Or, if you see throughput stats, ask, “at what quality?”

Whenever you test a hardware encoder, you should start by identifying the configuration options that most impact quality and throughput and then test across a range of configurations to get a sense of the performance/quality tradeoff. If you plug in pricing and power consumption figures, you can also easily compute the cost per stream and watts per stream. This is the CAPEX and OPEX side of the equation.

Then you can choose the “operating point” that delivers the optimum blend of quality and throughput for your applications. When comparing multiple encoders, you should perform the same analysis for each to enable a complete apples-to-apples comparison.

Recently, I benchmarked the performance of NETINT’s Quadra Video Processing Unit (VPU) against the NVIDIA T4. In this post, I’ll review the testing and the Quadra results to give you a feel for the hardware evaluation process. In a future post, I’ll review the NVIDIA findings and compare the two.

Benchmarking Quadra

Briefly, Quadra is NETINT’s newest ASIC-based transcoder, called a VPU, because it has onboard decoding, scaling, encoding, and overlay, plus an 18 TOPS AI engine. The VPU can create encoded bitstreams in H.264, HEVC, and AV1.

Quadra has two major configuration options that impact quality, lookahead buffer, and rate-distortion optimization.

Briefly, the lookahead buffer allows the encoder to look at frames ahead of the frame being encoded, so it knows what’s coming and can make more intelligent decisions. This improves encoding quality, particularly at/around scene changes, and it can improve bitrate efficiency. But, lookahead adds latency equal to the lookahead duration, and it can decrease throughput.

Table 1 shows the impact of a 40-frame lookahead buffer when encoding to the H.264 format. The top-line harmonic mean VMAF score is 2.3 points lower, which is borderline significant. But the low-frame differential of almost 16 points could predict transient problems that might be apparent to some viewers. But in addition to injecting 1.3 seconds of latency into the process, you see that the lookahead cuts the throughput by 33%, from 36 1080p streams to 24.

Table 1. Quality and performance impact
of a 40-frame lookahead on Quadra H.264 encoding.

Rate distortion optimization (RDO) functions like most presets and adjusts several parameters that impact both quality and throughout, with higher values increasing quality and reducing throughput. With H.264 output, Quadra offers one level of RDO, while with HEVC, there are three levels, 1, 2, and 3.

Table 2 shows the range of H.264 options tested during the recent benchmarking. LA is lookahead, and I tested three values, 40, 20, and 0. I also tested with RDO on and off. To provide some perspective of quality, the x264 Quality Equivalent shows x264 quality encoded using the same parameters using the presets shown.

At the highest quality setting, Quadra’s output quality slightly exceeded that of x264 using the slow preset, and the unit produced 16 1080p streams. You see that dropping the lookahead from 40 to 20 with RDO disabled had little impact on quality or throughput but cut latency by 0.66 sec, making that choice easy for latency-sensitive events.

Table 2. H.264 options and results.

At the lowest possible quality setting, Quadra’s quality dropped to slightly better than veryfast quality, which is often the x264 preset used for live applications to ensure at least nominal throughput with CPU-only transcoding.

At this quality level, the VPU outputs 36 1080p streams. By adding the cost per stream and watts per stream data, you will get a true feel for the comparative CAPEX and OPEX costs produced by all settings combinations.

Table 3 shows the same data for HEVC transcoding using the same lookahead options and RDO at 1, 2, 3, and 0 (disabled). At the highest quality levels, the output quality nearly matched the x265 encoder using the slow setting but only produced four streams. At the other end of the spectrum, output quality nearly matched x265 using the very fast preset, but the Quadra produced 40 1080p 30 streams, four more than using the H.264 format.

Table 3. HEVC options and results.

There are several new hardware encoders coming, and their launches will be accompanied by aggressive claims about quality and throughput. My recommendation is not to assume that the same settings were used for both. In short, you better do your own testing. Trust but verify comes to mind.

When you perform your own testing, remember the methodology explained above:

1. Identify the most critical quality-related options for your specific application. All producers have different priorities, whether it’s bitrate efficiency, absolute quality, ultra-low latency, density, power consumption, or cost per stream. You need to know what your critical constraints are in order to arrive at the best solution analysis.
2. Test across a range of configurations from high quality/low throughput to low quality/high throughput. Increasingly, even for quality-driven use cases, imperceptible quality tradeoffs might be necessary to meet an operational cost or energy efficiency target. You should choose the operating point that delivers the optimum blend for your application.
3. Compute quality, cost per stream, and watts per stream at the operating point to compare against other technologies. Remember to factor in the CAPEX of the additional servers required to run a software encoding service. Our customers report a reduction in the number of machines needed by 90% or more and this can translate to tens of millions or even hundreds of millions of dollars in savings or reclaimed CPUs that can be used in other parts of the operation.

In the next post, we’ll share quality results from the NVIDIA T4 GPU and compare them to Quadra.

ROI encoding is particularly relevant to cloud gaming, where viewers prefer fast-moving action, high resolutions, and high frame rates, but also want to play at low bitrates on wireless or cellular networks with ultra-low latency. These factors make cloud gaming a challenging compression environment.

If you encode with FFmpeg on AWS, you probably know that you have three CPU options: AMD, Graviton, and Intel. Which delivers the most bang for the buck?

For those in a hurry, it’s Graviton for x264 and AMD for x265, often by a significant margin. But the devil is always in the details, and if you want to learn how we tested and how big a difference your CPU selection makes, you can follow the narrative or hopscotch through the fancy charts below. We conclude with a look at the optimal core count for those encoding with AMD CPUs.

Testing the AWS CPUs

Let me start by saying that this was my first foray into CPU testing on AWS, and while it appears straightforward, some unconsidered complexity may have skewed the results. If you see any errors or other factors worth considering, please drop me a note at jan.ozer@netint.com.

Second, your source clip and command string may produce different results than those shown below. If you’re spending big to encode with FFmpeg on AWS, don’t consider my results the final word; instead, consider them as evidence that your CPU choice really does matter and as motivation to perform your own tests.

Those caveats aside, let’s dig into the testing.

Codecs/Configurations/Command Strings

I tested three test cases.

• 8-bit 1080p30 with x264
• 8-bit 1080p30 with x265
• 10-bit 4K60p with x265

I present the command strings at the bottom of this article. Note that I used the veryslow preset for x264, slower for x265 at 1080p30, and slow for the 4K60 HEVC encodes. Why such demanding presets? Because based upon a total cost of distribution (encoding and bandwidth), the optimal economic decision when view counts will exceed 10,000 views is to use a high-quality preset.

Based upon a total distribution cost (encoding and bandwidth), the optimal economic decision when view counts exceed 10,000 views is to use a high-quality preset.

Remember, presets don’t determine quality; your quality expectations do. Most compressionists target a VMAF score of between 93-95 VMAF points for the top rung of their encoding ladders. Using the veryslow preset, you might achieve that at, say, 3 Mbps. Using ultrafast, you might need a bit rate of as much as 5 Mbps to achieve the same quality. Ultrafast might cut your encoding time/cost by 90%, but you only pay that once, while you pay bandwidth costs for each video view. Even at a cost per GB of $0.02, it takes less than 10,000 views for the veryslow preset to break even based on lower bandwidth costs.

Instances and Pricing

I tested using the 8-core instances and on-demand pricing shown in Table 1. I tested all systems running Ubuntu version 22.04. Note that the cost delta between Intel and AMD is ten percent, a number I’ll refer to below.

Table 1: Instances and on-demand pricing tested.

Encoding Procedure

As you’ll see in the charts below, I started encoding a single FFmpeg instance and kept adding simultaneous encodes until the cost per stream began to increase, indicating that spinning up another instance was more cost effective than adding additional encodes to the same system.

FFmpeg Versions

Here’s where things get a bit complicated. My premise was that I would produce the optimal results using FFmpeg versions compiled specifically for each CPU tested. I downloaded builds for Graviton, AMD, and Intel from https://johnvansickle.com/ffmpeg/ and happily contributed via PayPal. However, I was also in touch with MulticoreWare, who requested that I test with an advanced version of their x265 codec that was optimized for Graviton.

Figure 1. I tested with CPU-specific versions of FFmpeg 6.0 from https://johnvansickle.com/ffmpeg/.

Before testing, I compared the performance of the stock version of FFmpeg (Version 4.4) with the CPU-specific versions from Vansickle on the AMD and Intel platforms and for x264 on Graviton. In all cases, the Vansickle version produced the same or better throughput with identical quality.

Note that in other tests on different AMD instances with core counts ranging from 2 – 32, the Vansickle version was not always the best performer. So, if you try the Vansickle versions or your own CPU-specific compiled versions, you should verify that it outperforms the native version in all relevant use cases.

Note that the MulticoreWare version of FFmpeg performed much better on the Graviton system than the generic version of 4.4 or the Vansickle version, though still far behind Intel and particularly AMD. As you’ll see clearly below, if you’re running x265 on a Graviton system using high quality presets, you’re missing a great opportunity to shave your costs.

For the record, I tried upgrading the stock version of FFmpeg on the Ubuntu system to version 6.0 but ran into multiple issues that ultimately corrupted the system and forced me to start back at ground zero. Unfortunately, Ubuntu operation and maintenance are not a core-strengths of mine, but since I ran all tests using Version 6.0, whether supplied by Vansickle or MulticoreWare, the results should be representative.

Table 2 shows the different versions of FFmpeg that I ran on the three systems for the three test cases.

Table 2. The FFmpeg versions deployed on the three systems for the three test cases.

Results

Here are the results for the three test cases.

1080p x264

Figure 2 shows the cost per hour to produce a 1080p30 stream using FFmpeg and the x264 codec. One of the more interesting testing results was that the combination of FFmpeg and Ubuntu handled multiple instances of FFmpeg with minimal overhead, particularly on the Graviton CPU. You see this with the cost per hour for Graviton remaining consistent through twelve instances, while it increased slightly for Intel after 10 instances and AMD after 12.

In all cases, you see the cost per instance drop significantly when moving from single to multiple simultaneous encodes. If you’re performing a single 1080p x264 encode on an 8-core system, you’re probably wasting money.

On the other hand, once each CPU hits the lowest cost per hour, it’s time to consider adding another instance. The cost per stream will remain the same, but your encoding speed will double. So, if you’re encoding on a Graviton system, your encoding time will double if you perform twelve simultaneous encodes as opposed to six, but your cost per hour will be almost exactly the same. If you spin up another 8-core system and encode six simultaneous encodes on the two systems, your cost will be almost identical, but your throughput will double.

Figure 2. Cost per hour to produce a single 1080p stream using the x264 codec and FFmpeg. Graviton is clearly the most cost-effective.

1080p x265

What a difference a codec makes. Where Graviton was the clear leader for x264, it’s the clear laggard for x265. Again, I produced the Graviton results shown in Figure 3 using a version of FFmpeg supplied by x265 developer MulticoreWare; the results would have been much worse with either the Vansickle version or the stock version. As you may know, Graviton is an Arm-based CPU that uses a different instruction set than Intel or AMD CPUs. While the x264 codec was Arm-friendly, the x265 codec was decidedly the reverse, at least using the high-quality presets that I used in my tests.

Interestingly, for both Intel and AMD, we realized the lowest cost per stream at relatively low simultaneous stream counts, two for Intel and two and three for AMD. If your testing confirms this, you should consider adding instances once you achieve this threshold rather than adding additional encodes to existing instances.

Figure 3. Cost per hour to produce a single 1080p stream using the x265 codec and FFmpeg.

Comparing the lowest cost Intel ($6.60) to the lowest cost AMD ($5.49), shows a cost delta of about 17%. As shown in Table 1, 10% of this relates to pricing, leaving about a 7% performance delta.

For the record, note that an Amazon engineer ran similar tests here and found that Graviton was faster for both x264 and x265. Note, however, that the author used the ultrafast preset, while I used higher quality presets for the stated reasons. Have a look and draw your own conclusions.

4K60 x265

In 4K60p testing, the Graviton was clearly overwhelmed from both a cost and performance aspect, unable to complete even three simultaneous encodes. The overall cost delta between Intel and AMD narrowed slightly, dropping to 13.7% overall, with 10% relating to pricing. The actual throughput delta between the two in these tests is 3.7%.

Figure 4. Cost per hour to produce a single 4K60p stream using the x265 codec and FFmpeg.

This 4K60 test stressed memory usage much more so than the 1080p tests, limiting successful simultaneous transcodes to two for Graviton and four for AMD and Intel. Interestingly, in these tests, AMD produced the lowest cost per stream while running a single encode, and Intel did so at 2. With these challenging encodes; you may want to spin up new machines after only one or two encodes rather than attempting more simultaneous encodes. Or, perhaps, try a machine with more cores. Hold that thought until the last section.

For reference, Table 3 summarizes the lowest cost per hour for the three test cases.

Table 3. Cost per hour for the three test cases on the three tested CPUs.

Which leads us to the last section.

What’s the Optimal Number of Cores for FFmpeg?

AWS offers multiple core counts in all three CPU flavors: what’s the optimal core count? To evaluate this, I ran tests on multiple AMD CPUs for all three test cases and present the results below.

Let’s talk about expectations first. AWS charges linearly for the machine cores, so an 8-core system costs twice as much as a 4-core system and a quarter of a 32-core system. Given the results presented above, where FFmpeg/Ubuntu proved highly efficient when processing multiple instances, I expected a similar cost per hour for all CPUs. The results were close.

With x264, 2-core and 8-core systems were slightly more affordable than 16-core, though a 32-core system finally caught up at 32 simultaneous transcodes. If you’re going to run a 32-core system for 1080p30/x264 encodes, you need to be running quite a few simultaneous encodes to achieve the optimal cost per stream.

Figure 5. x264 encoding cost for the CPU core counts shown.

With x265 encoding at 1080p, the results were closer to what I expected, though again, the 2-core and 8-core systems were slightly more affordable. Unlike x264, the 32-core system became slightly more expensive as the number of simultaneous encodes increased, making eight simultaneous streams the most affordable.

Figure 6. x265 encoding cost for 1080p30 encodes and the CPU core counts shown.

When encoding 4K videos, the phrase “go big or go home” comes to mind. Here, 32-cores delivered the lowest cost, though only by a fraction, and only at four simultaneous encodes. After that, the cost per hour increases slightly through eight encodes and then starts a more serious climb.

Figure 7. x265 encoding cost for 4K60 encodes and the CPU core counts shown.

As you can see, all these results are highly codec and source material specific. The most important takeaway from this article should not be that Graviton is best for x264 and AMD best for x265. It should be that real differences exist between the performance of the CPUs, and these differences may translate to significant cost differentials. If you’re spending even a few thousand dollars a month on AWS for FFmpeg encoding, it makes sense to run tests like these to identify the most cost-effective CPU and core-count.

Test Strings

1080p30 x264:

ffmpeg -y -i Orchestra.mp4 -c:v libx264 -profile:v high -preset veryslow -g 60 -keyint_min 60 -sc_threshold 0 -b:v 4200k -pass 1 -f mp4 /dev/null

ffmpeg -y -i Orchestra.mp4 -c:v libx264 -preset veryslow -g 60 -keyint_min 60 -sc_threshold 0 -b:v 4200k -maxrate 8400k -bufsize 8400k -pass 2 orchestra_x264_output.mp4

1080p30 x265:

ffmpeg -y -i Football_short.mp4 -c:v libx265 -preset slower -x265-params keyint=60:min-keyint=60:scenecut=0:bitrate=3500:pass=1 -f mp4 /dev/null

ffmpeg -y -i Football_short.mp4 -c:v libx265 -preset slower -x265-params keyint=60:min-keyint=60:scenecut=0:bitrate=3500:vbv-maxrate=7000:vbv-bufsize=7000:pass=2 Football_x265_HD_output.mp4

4K60 x265:

ffmpeg -y -i Football_4K60.mp4 -c:v libx265 -preset slow -x265-params keyint=120:min-keyint=120:scenecut=0:bitrate=12500K:pass=1 -f mp4 /dev/null

ffmpeg -y -i Football_4K60.mp4 -c:v libx265 -preset slow -x265-params keyint=120:min-keyint=120:scenecut=0:bitrate=12500K:vbv-maxrate=25000K:vbv-bufsize=25000K:pass=2 Football_4K_output.mp4

https://youtu.be/BOZZuiemMAU

HARD QUESTIONS ON HOT TOPICS: AMD, Graviton, and Intel
– three CPU options to encode with FFmpeg on AWS
Watch the full conversation on YouTube: https://youtu.be/BOZZuiemMAU

From binge-watching our favorite shows to attending virtual conferences, the demand for high-quality video delivery continues to grow exponentially. As content providers strive to meet these expectations, they face a constant challenge: delivering top-notch video quality while managing bandwidth costs effectively. Enter Capped Constant Rate Factor (CRF) Encoding is a sophisticated technique revolutionizing how we approach video streaming optimization.

The Quest for Efficient Video Delivery

In the world of video streaming, efficiency and quality are paramount. As audiences demand better viewing experiences, optimizing video delivery without unnecessarily increasing costs is essential. Unlike traditional constant bitrate methods, capped CRF provides more flexibility, making it ideal for live streaming scenarios where video quality and bandwidth savings must be balanced.

This essay will explore the intricacies of capped CRF encoding and why the implementation in the NETINT Quadra VPU family has profound implications. By the conclusion, you’ll gain a thorough understanding of why this technique is transforming real-time streaming and how to leverage it effectively in various use cases.

Section 1: The Evolution of Video Encoding

To fully appreciate the significance of Capped CRF, we need to understand its predecessors and the limitations they sought to overcome. Let’s explore the journey from traditional encoding methods to today’s more sophisticated approaches.

1.1 Constant Bitrate (CBR) Encoding: The Legacy Approach

Constant Bitrate (CBR) encoding has long been a staple in streaming applications due to its predictability and simplicity. However, its inherent inefficiencies have become increasingly apparent as video content complexity has grown.

Technical Deep Dive: CBR Mechanics

CBR maintains a fixed bitrate throughout the video by adjusting the Quantization Parameter (QP) frame-by-frame. It utilizes a strict rate control mechanism to meet the target bitrate for each Group of Pictures (GOP). While this approach guarantees a consistent data rate, it comes with significant drawbacks:

1. Bandwidth Inefficiency: Simple or static scenes consume as much data as complex ones, leading to unnecessary data transmission. Imagine watching a movie where a static shot of a landscape uses the same amount of data as an action-packed car chase. That’s CBR in action, and it’s not the most efficient bandwidth use.
2. Quality Inconsistencies: Complex scenes may suffer from quality degradation as the fixed bitrate cannot encode all details adequately. This results in a fluctuation of quality that’s inversely proportional to scene complexity.

Research by Katsavounidis et al. (2015) demonstrated that CBR can result in up to 40% wasted bandwidth in certain content types, particularly those with varying scene complexities [1]. This inefficiency becomes particularly problematic in large-scale video streaming services, where even small bandwidth savings can translate to significant cost reductions.

1.2 Variable Bitrate (VBR) Encoding: A Step Forward

Variable Bitrate (VBR) encoding emerged as an improvement over CBR, allowing for bitrate fluctuations based on scene complexity. This approach aimed to address the inefficiencies of CBR by allocating more bits to complex scenes and fewer to simpler ones.

Technical Insights: VBR Operation

VBR employs a looser rate control mechanism, often using a two-pass encoding process to allocate bits more efficiently. It aims for consistent quality across scenes but can lead to significant bitrate fluctuations. While VBR represented a substantial improvement over CBR, it also introduced new challenges:

1. Unpredictable Bandwidth Usage: VBR can lead to significant bitrate spikes during complex scenes without upper bounds. This unpredictability can cause issues in bandwidth-constrained environments or when estimating storage requirements.
2. Potential Buffering Issues: These spikes can cause playback interruptions in bandwidth-restricted environments, leading to a poor user experience.

The MPEG-DASH standard (ISO/IEC 23009-1:2019) acknowledges these challenges and provides guidelines for implementing adaptive bitrate streaming to mitigate some VBR-related issues [2]. However, even with these improvements, VBR alone couldn’t fully solve the quality-vs-efficiency dilemma in video streaming.

Section 2: Understanding Capped CRF Encoding

As the limitations of both CBR and VBR became apparent, researchers and engineers in the video compression field sought a more balanced approach. This quest led to the development of Constant Rate Factor (CRF) encoding and its refined version, Capped CRF.

2.1 The Foundations of CRF

Constant Rate Factor (CRF) is a rate control mode that aims to achieve a consistent perceptual quality across the entire video. It’s based on the x264 video encoding library and has been widely adopted in various encoding standards.

Technical Deep Dive: CRF Mechanics

CRF operates on a scale from 0 (lossless) to 51 (lowest quality), with lower values indicating higher quality and larger file sizes. The CRF algorithm dynamically adjusts each frame’s quantization parameter (QP) based on its complexity. This approach allows for:

1. Quality-Driven Encoding: Unlike CBR or VBR, which prioritize bitrate, CRF focuses on maintaining a consistent quality level.
2. Content-Aware Bit Allocation: Complex scenes receive more bits, while simpler scenes use fewer, resulting in more efficient bandwidth use.
3. Perceptual Optimization: The CRF algorithm is designed to allocate bits to align with human visual perception, potentially improving subjective quality.

However, standard CRF also has limitations. In scenes with extreme complexity, it could use an excessive amount of data, leading to unpredictable bandwidth usage. This is where Capped CRF comes into play.

2.2 The Innovation of Capped CRF

Capped CRF builds upon the CRF concept by introducing an upper limit on the bitrate. This approach combines the quality-focused nature of CRF with the predictability of CBR.

Key Technical Aspects of Capped CRF

1. Dynamic Bitrate Allocation: Adjusts bitrate according to scene complexity, similar to standard CRF.
2. Bitrate Ceiling: Imposes a maximum bitrate to prevent excessive bandwidth usage during complex scenes.
3. Quality-Driven Encoding: Prioritizes consistent perceptual quality while respecting bitrate constraints.
4. Adaptive Quantization: Utilizes advanced algorithms to adjust quantization parameters within the bitrate cap constraint.

Research by Netflix’s video algorithms team has shown that Capped CRF can achieve up to 30% bitrate savings compared to CBR while maintaining equivalent perceptual quality [3]. This significant improvement demonstrates the potential of Capped CRF to revolutionize video streaming efficiency.

Section 3: Technical Comparison of Encoding Methods

To fully appreciate the impact of Capped CRF, it’s essential to compare it against other encoding methods from a technical standpoint. Let’s dive deep into the mechanics of each approach:

3.1 Constant Bitrate (CBR)

• Encoding Process: Maintains a fixed bitrate throughout the video by adjusting the Quantization Parameter (QP) on a frame-by-frame basis.
• Rate Control: Utilizes a strict rate control mechanism to ensure the target bitrate is met for each Group of Pictures (GOP).
• Quality Variation: Quality fluctuates inversely with scene complexity, as the fixed bitrate constrains the available bits per frame.
• Bandwidth Usage: Predictable but inefficient, often wasting bandwidth on simple scenes.
• Application: Commonly used in live streaming scenarios where predictable bandwidth usage is crucial.

3.2 Variable Bitrate (VBR)

• Encoding Process: Allows bitrate to vary based on scene complexity, typically within a specified range.
• Rate Control: Employs a looser rate control mechanism, often using a two-pass encoding process to allocate bits more efficiently.
• Quality Variation: Aims for more consistent quality across scenes but can lead to significant bitrate fluctuations.
• Bandwidth Usage: More efficient than CBR but can result in unpredictable spikes.
• Application: Often used for on-demand content where overall file size is a concern.

3.3 Constant Rate Factor (CRF)

• Encoding Process: Adjusts bitrate dynamically to maintain a consistent quality level across the entire video.
• Rate Control: Uses a psychovisual model to determine the optimal number of bits for each frame based on its complexity.
• Quality Variation: Maintains relatively consistent quality but can result in large variations in bitrate.
• Bandwidth Usage: Highly efficient but potentially unpredictable.
• Application: Ideal for archival purposes or when quality is the primary concern.

3.4 Capped CRF

• Encoding Process: Combines CRF’s quality-focused approach with a maximum bitrate limit.
• Rate Control: Utilizes CRF’s psychovisual model but introduces a bitrate cap to prevent excessive bandwidth usage.
• Quality Variation: Strives for consistent quality while ensuring bitrate stays within predefined limits.
• Bandwidth Usage: Efficient and predictable, offering the best of both CRF and CBR.
• Application: Well-suited for adaptive streaming scenarios and live events where both quality and bandwidth predictability are important.

The ITU-T H.264 and H.265 standards provide the framework for implementing these various rate control methods, with Capped CRF being an extension of the rate control mechanisms described in these standards [4][5].

1080p RESULTS

Table 1 shows 1080p results, divided between easy-to-encode and hard-to-encode content. The encoded CBR clips are at 4.5 Mbps, and the same cap was applied for capped CRF encoding.

Table 1. 1080p results using Quadra VPU and capped CRF encoding.

In CBR mode, Quadra VPUs do not reach the target rate as accurately as when using capped CRF mode. This won’t degrade viewer quality of experience since the VMAF scores exceed 95, so this missing on the low side saves excess bandwidth with no visual quality detriment.

In this comparison, bitrate savings is minimized, particularly at CRF 19 and 21, as the capped CRF clips in the hard-to-encode content have a higher bitrate than the CBR counterparts (4,419 and 4,092 to 3,889). Not surprisingly, CRF 19 and 21 deliver little bandwidth savings and a slightly higher quality than CBR.

At CRF 23, things get interesting, with an overall bandwidth savings of 16.1% and a negligible quality delta from CBR. With a VMAF score of around 95, CRF 23 might be the target for engineers delivering premium content. Engineers targeting slightly lower quality can choose CRF 27, achieve a bitrate savings of 43%, and an efficient 2.4 Mbps bit rate for hard-to-encode footage. At CRF 27, Quadra VPUs encoded the hard-to-encode Football clip at 3,999 kbps with an impressive VMAF score of 93.39.

As with H.264 and HEVC, AV1 capped CRF reduces throughput. Specifically, a single Quadra VPU installed in a 32-core workstation outputs 23 simultaneous CBR streams using CBR encoding. This dropped to eighteen for capped CRF, a reduction of 22%.

4K RESULTS

Many engineers encoding with AV1 deliver UHD content, so we ran similar tests with the Quadra and 4K30 8-bit content with a CBR target and bitrate cap of 16 Mbps. Using four clips, including a 4K version of the high-motion Football clip to much less dynamic content like Netflix’s Meridian clip and Blender Foundation’s Sintel

Table 2. 4K results for the Quadra VPU and capped CRF encoding.

In CBR mode, the Quadra VPU hit the bitrate target much more accurately at 4K than 1080p, so even at CRF 19, the VPU delivered a 13% bitrate savings with a VMAF score of 96.23. Again, CRF 23 delivered a VMAF score of close to 95, with 45% savings over CBR. Impressively, at CRF 23, Quadra delivered an overall VMAF score of 94.87 for these 4K clips at 7.78 Mbps, with the Football clip weighing in at 14.3 Mbps.

Of course, these savings directly relate to the cap and CBR target. It’s certainly fair to argue that 16 Mbps is excessive for 4K AV1-encoded content, though Apple recommends 16.8 for 8-bit 4K content with HEVC here.

The point is, when you encode with CBR, you’re limiting quality to control bandwidth costs. With capped CRF, you can set the cap higher than your CBR target, knowing that all content contains easy-to-encode regions that will balance out the impact of the higher cap and deliver similar or lower bandwidth costs. With these comparative settings, capped CRF provides higher quality video during hard-to-encode regions than CBR, similar quality during all other scenes, and improved quality of experience at the same cost or lower than CBR.

Section 4: Implementing Capped CRF for Maximum Efficiency

Integrating Capped CRF encoding into a video streaming workflow requires careful planning and execution. Here’s a detailed, step-by-step guide to implementing Capped CRF for maximum efficiency:

4.1. Content Analysis

Understanding the characteristics of your video content is crucial before encoding. This analysis will inform your encoding decisions and help optimize the Capped CRF settings.

• Tools: Utilize software like FFprobe to analyze video complexity, motion, and texture characteristics.
• Process: a) Run scene detection algorithms to identify transitions and complexity variations within the content. b) Generate complexity maps for each video, highlighting areas of high motion or detail. c) Analyze the distribution of scene complexities to inform your CRF and cap settings.

4.2. CRF Level Selection

Choosing the right CRF value is critical for balancing quality and efficiency.

• Starting Point: Begin with a CRF value of 18-23 for high-quality streaming.
• Optimization: a) Use objective quality metrics like VMAF (Video Multimethod Assessment Fusion) or SSIM (Structural Similarity Index) to fine-tune the CRF value for your specific content. b) Conduct A/B testing with a diverse set of video samples to validate your CRF selecti

4.3. Bitrate Cap Determination

Setting an appropriate bitrate cap ensures efficient bandwidth usage without compromising quality on complex scenes.

• Analysis: Encode a representative set of your content using uncapped CRF to understand the bitrate distribution.
• Cap Selection: a) Consider using percentile-based capping (e.g., 95th percentile of uncapped CRF bitrate) as a starting point. b) Adjust the cap based on your specific quality requirements and bandwidth constraints.

4.4. Encoding Configuration

With your CRF level and bitrate cap determined, it’s time to configure your encoder.

● For x264/x265 Encoders:

-crf 23 -maxrate 5M -bufsize 10M

• Adjust the maxrate and bufsize values based on your determined cap and desired rate control strictness.
• For AV1 Encoding: Refer to the AV1 Bitstream & Decoding Process Specification for specific configuration options related to rate control in AV1 [9].

4.5. Quality Assurance

Implement a robust quality assurance process to ensure your Capped CRF encodes meet your quality standards.

• Automated Assessment: a) Use tools like FFMPEG with libvmaf to calculate objective quality metrics for your encodes. b) Implement automated checks to flag any encodes that fall below your quality thresholds.
• Subjective Testing: a) Conduct regular visual inspections of your encodes, particularly for challenging scenes. b) Perform A/B testing with human viewers to validate that objective metrics align with subjective quality perceptions.

4.6. Adaptive Streaming Integration

To maximize the benefits of Capped CRF, integrate it into an adaptive streaming setup.

• Rendition Generation: a) Create multiple Capped CRF renditions at different quality levels and bitrate caps. b) Ensure a smooth quality ladder between renditions to facilitate seamless adaptive streaming.
• Manifest Creation: a) Generate DASH or HLS manifests that include your Capped CRF renditions. b) Consider using Per-Title Encoding techniques to further optimize your adaptive streaming setup [10].

4.7. Performance Monitoring

Implement comprehensive monitoring to track the performance of your Capped CRF encodes in real-world conditions.

• Server-Side Metrics: a) Track bandwidth usage, encoding time, and storage requirements for your Capped CRF encode. b) Monitor CDN performance and delivery efficiency.
• Client-Side Metrics: a) Collect Quality of Experience (QoE) metrics such as startup time, rebuffering ratio, and average bitrate. b) Analyze user engagement and viewing patterns about your Capped CRF implementation.

By following these steps and continually refining your approach based on performance data, you can maximize the efficiency of Capped CRF encoding in your video streaming pipeline.

Section 5: Common Pitfalls and How to Avoid Them

While Capped CRF offers significant advantages, its implementation can be complex. Here are some common pitfalls and strategies to avoid them:

5.1 Suboptimal Cap Selection

One of the most prevalent issues is setting an inappropriate bitrate cap. This can negate the benefits of CRF encoding.

Technical Impact:

• Too Low: Insufficient bitrate for complex scenes, leading to visible artifacts and quality degradation.
• Too High: Inefficient bandwidth use, potentially negating the benefits of using Capped CRF.

Solution: Implement a data-driven approach to cap selection. Utilize objective quality metrics such as VMAF (Video Multimethod Assessment Fusion) or SSIM (Structural Similarity Index) to assess the quality impact of different cap levels. The Netflix VMAF model, which correlates closely with subjective quality assessments, can be particularly useful [6].

5.2 Neglecting Network Dynamics

Failing to account for varying network conditions can lead to suboptimal streaming experiences, even with Capped CRF.

Technical Impact: Static bitrate caps may not adapt well to fluctuating network conditions, potentially causing buffering or quality drops.

Solution: Integrate Capped CRF with Adaptive Bitrate (ABR) streaming technologies. The DASH-IF Implementation Guidelines provide comprehensive recommendations for implementing ABR with various encoding methods [7]. Consider implementing per-chunk encoding, where the CRF and cap are adjusted for each segment based on content complexity and network conditions.

5.3 Over-reliance on Default Settings

Using preset CRF values and caps without content-specific optimization can result in suboptimal performance.

Technical Impact: Generic settings may not adequately address the unique characteristics of different content types, leading to inefficient encoding.

Solution: Implement content-aware encoding (CAE) strategies. Research by Katsavounidis and Guo (2018) demonstrates how machine learning models can be used to predict optimal encoding parameters based on content features [8]. Develop a content classification system that can automatically select appropriate CRF and cap settings based on video characteristics.

5.4 Ignoring Encoder-Specific Optimizations

Different encoders (x264, x265, AV1, etc.) have unique characteristics and optimizations that can affect Capped CRF performance.

Technical Impact: Linking encoder-specific features can result in suboptimal quality or efficiency.

Solution: Familiarize yourself with your chosen encoder’s specific features and optimizations. For example:

• For x264/x265, explore options like adaptive quantization modes, psychovisual optimizations, and rate control lookahead.
• For AV1, investigate AV1-specific tools like super-resolution, film grain synthesis, and compound prediction.

Regularly test and benchmark different encoder settings to find the optimal configuration for your Capped CRF implementation.

5.5 Overlooking Audio Encoding

While focusing on video optimization, it’s easy to overlook the impact of audio encoding on overall streaming performance.

Technical Impact: Inefficient audio encoding can negate some of the bandwidth savings achieved through Capped CRF video encoding.

Solution: Implement adaptive audio encoding alongside your Capped CRF video strategy. Consider using modern audio codecs like Opus or AAC-ELD for efficient, high-quality audio delivery. Adjust audio bitrates dynamically based on content type (e.g., higher bitrates for music, lower for speech) and in response to network conditions.

Section 6: Future Trends and Innovations in Video Encoding

As we look to the future, several emerging trends and innovations are likely to shape the landscape of video encoding, potentially enhancing or complementing Capped CRF techniques:

6.1 AI-Driven Encoding Optimization

Machine learning and artificial intelligence are increasingly being applied to video encoding processes, offering the potential for even more efficient and intelligent bitrate allocation.

Potential Developments:

• AI-powered scene analysis for more accurate complexity assessment
• Dynamic adjustment of CRF levels and bitrate caps based on real-time learning from user engagement data
• Predictive encoding that anticipates upcoming scene changes for smoother-quality transitions

6.2 Next-Generation Codecs

The development of new video codecs like AV1, VVC (Versatile Video Coding), and EVC (Essential Video Coding) promises even greater compression efficiency.

Implications for Capped CRF:

• Potential for lower CRF values and bitrate caps while maintaining quality
• Need for new optimization strategies tailored to the characteristics of these codecs
• Possibility of codec-specific variations on the Capped CRF concept

6.3 Per-Scene and Per-Object Encoding

Advancements in video analysis may enable more granular approaches to bitrate allocation.

Future Possibilities:

• Applying different CRF levels to different objects or regions within a frame
• Dynamically adjusting encoding parameters based on the semantic importance of scene elements
• Integrating with video understanding technologies for content-aware encoding at an unprecedented level of detail

6.4 Edge Computing for Real-Time Encoding

The growth of edge computing infrastructure could enable more sophisticated real-time encoding strategies.

Potential Applications:

• Distributed encoding systems that can apply Capped CRF more effectively in live streaming scenarios
• Real-time content analysis and encoding parameter adjustment at the network edge
• Personalized encoding profiles based on individual viewer preferences and device capabilities

The Future of Video Streaming Optimization

Capped CRF encoding represents a significant advancement in the quest for efficient, high-quality video delivery. By combining the quality-focused approach of Constant Rate Factor encoding with the predictability of bitrate caps, this technique offers a powerful solution to many of the challenges modern streaming platforms face.

As we’ve explored in this comprehensive essay, implementing Capped CRF requires a deep understanding of video encoding principles, careful configuration, and ongoing optimization. However, the potential benefits,including bandwidth savings, consistent quality, and improved viewer experiences,make it a compelling option for content providers of all sizes.

Looking ahead, the field of video encoding continues to evolve rapidly. Emerging technologies like AI-driven optimization, next-generation codecs, and more granular encoding approaches promise to refine our ability to deliver high-quality video efficiently further. With its flexible and quality-focused approach, Capped CRF is well-positioned to integrate with and benefit from these advancements.

The key to success for streaming providers will be staying informed about these developments and continuously refining their encoding strategies. By leveraging techniques like Capped CRF and embracing emerging innovations, they can ensure they deliver the best viewing experience while optimizing resource utilization.

As we move into an era of increasingly immersive and data-intensive video experiences, the principles behind Capped CRF – balancing quality, efficiency, and adaptability – will undoubtedly remain crucial. Whether you’re a small content creator or a large streaming platform, understanding and implementing advanced encoding techniques like Capped CRF can give you a significant edge in the competitive and ever-evolving world of digital video.

References

[1] Katsavounidis, I., et al. (2015). “A Video Complexity Verifier for Dynamic Range Video.” IEEE International Conference on Image Processing (ICIP).

[2] ISO/IEC 23009-1:2019. “Information technology , Dynamic adaptive streaming over HTTP (DASH) , Part 1: Media presentation description and segment formats.”

[3] Netflix Technology Blog. (2017). “Per-Title Encode Optimization.” [Online]

[4] ITU-T Recommendation H.264 (2019). “Advanced video coding for generic audiovisual services.”

[5] ITU-T Recommendation H.265 (2019). “High efficiency video coding.”

[6] Li, Z., et al. (2016). “Toward a practical perceptual video quality metric.” Netflix Tech Blog.

[7] DASH Industry Forum. (2020). “DASH-IF Implementation Guidelines.”

[8] Katsavounidis, I., and Guo, L. (2018). “Video codec comparison using the dynamic optimizer framework.” Proc. SPIE 10752, Applications of Digital Image Processing XLI.

[9] Alliance for Open Media. (2018). “AV1 Bitstream & Decoding Process Specification.”

[10] Toni, L., et al. (2016). “Per-Title Encode Optimization.” Netflix Engineering Blog.

[11] Zadtootaghaj, S., et al. (2020). “Content-Aware Video Encoding for Cloud Gaming.” ACM Multimedia Systems Conference.

[12] Kua, J., et al. (2017). “A Survey of Rate Adaptation Techniques for Dynamic Adaptive Streaming Over HTTP.” IEEE Communications Surveys & Tutorials.

[13] Li, Z., et al. (2018). “VMAF: The Journey Continues.” Netflix Technology Blog.

ACCESS NOW: ASIC-Based Transcoding
for High-volume Use Cases
Including social media, broadcast, interactive platforms, and service providers

ACCESS NOW

NETINT recently added capped CRF to the rate control mechanism across our Video Processing Unit (VPU) product lines. With the wide adoption of content-adaptive encoding techniques (CAE), constant rate factor (CRF) encoding with a bit rate cap gained popularity as a lightweight form of CAE to reduce the bitrate of easy-to-encode sequences, saving delivery bandwidth with constant video quality. It’s a mode that we expect many of our customers to use, and this document will explain what it is, how it works, and how to get the most use from the feature.

In addition to working with H.264, HEVC, and AV1 on the Quadra VPU line, capped CRF works with H.264 and HEVC on the T408 and T432 video transcoders. This document details how to encode with capped CRF using the H.264 and HEVC codecs on Quadra VPUs, though most application scenarios apply to all codecs across the NETINT VPU lines.

What is Capped CRF and How Does it Work?

Capped CRF is a bitrate control technique that combines constant rate factor (CRF) encoding with a bit rate cap. Multiple codecs and software encoders support it, including x264 and x265 within FFmpeg. In contrast to CBR and VBR encoding, which encode to a specified target bitrate (and ignore output quality), CRF encodes to a specified quality level and ignores the bitrate.

CRF values range from 0-51, with lower numbers delivering higher quality at higher bitrates (less savings) and higher CRF values delivering lower quality levels at lower bitrates (more bitrate savings). Many encoding engineers will utilize values spanning 21 to 23. Which is right for you? As you will read below, your desired quality and bitrate savings balance determines the best value for your use case.

For example, with the x264 codec, if you transcode to CRF 23, the encoder typically outputs a file with a VMAF quality of 93-95. If that file is a 4K60 soccer match, the bitrate might be 30 Mbps. If it’s a 1080p talking head, it might be 1.2 Mbps. Because CRF delivers a known quality level, it’s ideal for creating archival copies of videos. However, since there’s no bitrate control, in most instances, CRF alone is unusable for streaming delivery.

When you combine CRF with a bit rate cap, you get the best of both worlds, a bit rate reduction with consistent quality for easy-to-encode clips and similar to CBR quality and bitrate or more complex clips.

Here’s how capped CRF could be used with the Quadra VPU:

ffmpeg -i input crf=23:vbvBufferSize=1000:bitrate=6000000 output

The relevant elements are:

• CRF=23 – sets the quality target at around 95 VMAF
• vbvBufferSize=1000 – sets the VBV buffer to one second (1000 ms)
• bitrate=6000000 – caps the bitrate at 6 Mbps.

These commands would produce a file that targets close to 95 VMAF quality but, in all cases, peaks at around 6 Mbps.

For a simple-to-encode talking head clip, Quadra produced a file with an average bitrate of 1,274 kbps and a VMAF score of 95.14. Figure 1 shows this output in a program called Bitrate Viewer. Since the entire file is under the 6 Mbps cap, the CRF value controls the bitrate throughout.

Encoding this clip with Quadra using CBR at 6 Mbps produced a file with a bit rate of 5.4 Mbps and a VMAF score of 97.50. Multiple studies have found that VMAF scores above 95 are not perceptible by viewers, so the extra 2.26 VMAF score doesn’t improve the viewer’s quality of experience (QoE). In this case, capped CRF reduces your bandwidth cost by 76% without impacting QoE.

Figure 1. Capped CRF encoding a simple-to-encode video in Bitrate Viewer.

You see this in Figure 2, showing the capped CRF frame with a VMAF score of 94.73 on the left and the CBR frame with a VMAF score of 97.2 on the right. The video on the right has a bit rate over 4 Mbps larger than the video on the left, but the viewer wouldn’t notice the difference.

Figure 2. Frames from the talkinghead clip. Capped CRF at 1.23 Mbps on the left,
CBR at 5.4 Mbps on the right. No viewer would notice the difference.

Figure 3 shows capped CRF operation with a hard-to-encode American football clip. The average bitrate is 5900 kbps, and the VMAF score is 94.5. You see that the bitrate for most of the file is pushing against the 6 Mbps cap, which means that the cap is the controlling element. In the two regions where there are slight dips, the CRF setting controls the quality.

Figure 3. Capped CRF encoding a hard-to-encode video in Bitrate Viewer.

In contrast, the CBR encode of the football clip produced a bit rate of 6,013 kbps and a VMAF score of 94.73. Netflix has stated that most viewers won’t notice a VMAF differential under 6 points, so a viewer would not perceive the .25 VMAF delta between the CBR and capped CRF file. In this case, capped CRF reduced delivery bandwidth by about 2% without impacting QoE.

Of course, as shown in Figure 2, the two-minute segment tested was almost all high motion. The typical sports broadcast contains many lower-motion sequences, including some commercials, cutting to the broadcasters, or during timeouts and penalty calls. In most cases, you would expect many more dips like those shown in Figure 2 and more substantial savings.

So, the benefits of capped CRF are as follows:

• You can use a single ladder for all your content, automatically saving bitrate on easy-to-encode clips and delivering the equivalent QoE on hard-to-encode clips.
• Even if you modify your ladder by type of content, you should save bandwidth on easy-to-encode regions within all broadcasts without impacting QoE.
• Provides the benefit of CAE without the added integration complexity or extra technology licensing cost. Capped CRF is free across all NETINT VPU and video transcoder products.

Producing Capped CRF

Using the NETINT Quadra VPU series, the following commands for H.264 capped CRF will optimize video quality and deliver a file or stream with a fully compliant VBV buffer. As noted previously, this command string with the appropriate modifications to codec value will work across the entire NETINT product line. For example, to output HEVC, change -c:v h264_ni_quadra_enc to -c:v h265_ni_quadra_enc.

Here’s the command string.

ffmpeg -y -i input.mp4 -y -c:v h264_ni_quadra_enc -xcoder-params “gopPresetIdx=5:RcEnable=0:crf=23:intraPeriod=120:lookAheadDepth=10:cuLevelRCEnable=1:v
bvBufferSize=1000:bitrate=6000000:tolCtbRcInter=0:tolCtbRcIntra=0:zeroCopyMode=0″ output.mp4

Here’s a brief explanation of the encoding-related switches.

• -c:v h264_ni_quadra_enc -xcoder-params – Selects Quadra’s H.264 codec and identifies the codec commands identified below.
• gopPresetIdx=5 – this chooses the Group of Pictures (GOP) pattern, or the mixture of B-frame and P-frames within each GOP. You should be able to adjust this without impacting capped CRF performance.
• RcEnable=0 – this disables rate control. You must use this setting to enable capped CRF.

• crf=23 – this chooses the CRF value. You must include a CRF value within your command string to enable capped CRF.

• intraPeriod=120 – This sets the GOP size to four seconds which we used for all tests. You can adjust this setting to your normal target without impacting CRF operation.
• lookAheadDepth=10 – This sets the lookahead to 10 frames. You can adjust this setting to your normal target without impacting CRF operation.
• cuLevelRCEnable=1 – this enables coding unit-level rate control. Do not adjust this setting without verifying output quality and VBV compliance.

• vbvBufferSize=1000 – This sets the VBV buffer size. You must set this to trigger capped CRF operation.

• bitrate=6000000 – This sets the bitrate. You must set this to trigger capped CRF operation. You can adjust this setting to your target without impacting CRF operation.
• tolCtbRcInter=0 – This defines the tolerance of CU-level rate control for P-frames and B-frames. Do not adjust this setting without verifying output quality and VBV compliance.

• tolCtbRcIntra=0 – This sets the tolerance of CU level rate control for I-frames. Do not adjust this setting without verifying output quality and VBV compliance.

• zeroCopyMode=0 – this enables or disables the libxcoder zero copy feature. Do not adjust this setting without verifying output quality and VBV compliance.

You can access additional information about these controls in the Quadra Integration and Programming Guide.

Choosing the CRF Value and Bitrate Cap – H.264

Deploying capped CRF involves two significant decisions, choosing the CRF value and setting the bitrate cap. Choosing the CRF value is the most critical decision, so let’s begin there.

Table 1 shows the bitrate and VMAF quality of ten files encoded with the H.264 codec using the CRF values shown with a 6 Mbps cap and using CBR encoding with a 6 Mbps cap. The table presents the easy-to-encode files on top, showing clip-specific results and the average value for the category. The Delta from CBR shows the bitrate and VMAF differential from the CBR score. Then the table does the same for hard-to-encode clips, showing clip-specific results and the average value for the category. The bottom two rows present the overall average bitrate and VMAF values and the overall savings and quality differential from CBR.

Table 1. CBR and capped CRF bitrates and VMAF scores for H.264 encoded clips.

As mentioned, with CRF, lower values produce higher quality. In the table, CRF 19 produces the highest quality (and lowest bitrate savings), and CRF 27 delivers the lowest quality (and highest bitrate savings). What’s the right CRF value? The one that delivers the target VMAF score for your typical clips for your target audience.

For the test clips shown, CRF 19 produces an average quality of well over 95; as mentioned above, VMAF scores beyond 95 aren’t perceivable by the average viewer, so the extra bandwidth needed to deliver these files is wasted. Premium services should choose CRF values between 21-23 to achieve the top rung quality of around 95 VMAF scores. These deliver more significant bandwidth savings than CRF 19 while preserving the desired quality level. In contrast, commodity services should experiment with higher values like 25-27 to deliver slightly lower VMAF scores while achieving more significant bandwidth savings.

What bitrate cap should you select? CRF sets quality, while the bitrate cap sets the budget. In most cases, you should consider using your existing cap. As we’ve seen, with easy-to-encode clips, capped CRF should deliver about the same quality of experience with the potential for bitrate savings. For hard-to-encode clips, capped CRF should deliver the same QoE with the potential for some bitrate savings on easy-to-encode sections of your broadcast.

Note that identifying the optimal CRF value will vary according to the complexity of your video files, as well as frame rate, resolution, and bitrate cap. If you plan to implement capped CRF with Quadra or any encoder, you should run similar tests on your standard test clips using your encoding parameters and draw your own conclusions.

Now let’s examine capped CRF and HEVC.

Choosing the CRF Value and Bitrate Cap – HEVC

Table 2 shows the results of HEVC encodes using CBR at 4.5 Mbps and the specified CRF values with a cap of 4.5 Mbps. With these test clips and encoding parameters, Quadra’s CRF values produce nearly the same result, with CRF values 21-23 appropriate for premium services and 25 – 27 good settings for UGC content.

Table 2. CBR and capped CRF bitrates and VMAF scores for HEVC encoded clips.

Again, the cap is yours to set; we arbitrarily reduced the H.264 bitrate cap of 6 Mbps by 25% to determine the 4.5 Mbps cap for HEVC.

Capped CRF Performance

Note that as currently tested, capped CRF comes with a modest performance hit, as shown in Table 3. Specifically, in CBR mode, Quadra output twenty 1080p30 H.264-encoded streams. This dropped to sixteen using capped CRF, a reduction of 20%.

For HEVC, throughput dropped from twenty-three to eighteen 1080p30 streams, a reduction of about 22%. We performed all tests using CRF 21, with a 6 Mbps cap for H.264 and 4.5 Mbps for HEVC. Note that these are early days in the CRF implementation, and it may be that this performance delta is reduced or even eliminated over time.

Table 3. 1080p30 outputs produced using the techniques shown.

We installed the Quadra in a workstation powered by a 3.6 GHz AMD Ryzen 5 5600X 6-Core Processor running Ubuntu 18.04.6 LTS with 16 GB of RAM. As you can see in the table, we also tested output for the x264 codec in FFmpeg using the medium and veryfast presets, producing two and five 1080p30 outputs, respectively. For x265, we tested using the medium and ultrafast presets and the workstation produced one and three 1080p30 streams.

Even at the reduced throughput, Quadra’s CRF output dwarfs the CPU-only output. When you consider that the NETINT Quadra Video Server packs ten Quadra VPUs into a single 1RU form factor, you get a sense of how VPUs offer unparalleled density and the industry’s lowest cost per stream and power consumption per stream.

Bandwidth is one of the most significant costs for all live-streaming productions. In many applications, capped CRF with the NETINT Quadra delivers a real opportunity to reduce bandwidth cost with no perceived impact on viewer quality of experience.