Server Recommission — Beating the Shortage

How VPUs Revolutionize Video Encoding Cost & Performance

Viewers expect high-quality content delivered seamlessly, whether they’re watching on-demand movies or live sports events. However, achieving this level of performance can be financially and operationally challenging. That’s where Video Processing Units (VPUs) revolutionize the game. This article is tailored for video streaming engineers looking to optimize workflows, cut costs, and elevate user experiences.

Let’s explore how VPUs can redefine your video processing infrastructure, key insights on leveraging this technology, and actionable strategies to integrate it effectively.

Breaking the Cost Barrier: Why VPUs are Game-Changing

Delivering massive volumes of video content globally can overwhelm traditional infrastructure. High operating costs and reliance on public cloud solutions often make scalability unsustainable. VPUs provide a solution that dramatically reducing capital (CapEx) and operational expenditures (OpEx).

VPUs significantly increase throughput, allowing you to achieve up to 10 times the processing power compared to CPUs while reducing server requirements by 80%.

VPUs significantly increase throughput, allowing you to achieve up to 10 times the processing power compared to CPUs. They reduce server requirements by 80%, which saves on hardware investments and energy costs. Moreover, lower energy consumption contributes to greener operations. For instance, API.video reduced its transcoding costs to just $0.0002 per minute by integrating VPUs. This move enabled them to offer customers free transcoding services, which enhanced their competitive positioning. Start small by integrating VPUs into existing workflows to measure cost and efficiency gains before a full-scale rollout.

Hardware Acceleration vs. Traditional Software: Why VPUs Win

Traditional CPU-based video processing struggles to keep up with growing demands. VPUs offload compute-intensive tasks, enabling CPUs to focus on dynamic packaging, real-time subtitling, or other essential operations.

VPUs enhance performance by offering 20 times greater throughput. They simplify architecture by allowing software transcoding frameworks like FFmpeg or GStreamer to run on the same hardware. Additionally, VPUs streamline operations that would otherwise require significant computational resources. However, avoid underestimating integration efforts. Ensure compatibility with your existing infrastructure, including codecs and formats. Also, avoid overloading CPUs by reserving tasks better suited for VPUs, thus freeing CPU resources.

Flexible Deployment Options to Meet Every Need

Whether you’re operating a live streaming platform, a video-on-demand service, or a high-volume encoding system, VPUs offer flexibility in deployment. Add-in cards (AICs) are ideal for retrofitting existing servers. Standalone video servers come pre-configured for high-density encoding and decoding. For dedicated tasks, ASICs provide high-performance options. Products like NETINT Quadra VPUs support ultra-low latency, 8K HDR, and high-efficiency AV1 encoding. Quadra Video Servers can encode 320 live streams at 1080p30, supporting formats like HEVC, H.264, and AV1. Evaluate your workload’s scalability requirements and choose a deployment model that aligns with your growth plans.

Meeting Modern Encoding Demands with Advanced Codecs

As video resolutions and quality expectations rise, so does the demand for efficient codecs. VPUs support modern formats while maintaining high compression efficiency and low latency. They support H.264, HEVC, and AV1 codecs, which efficiently handle high-definition and ultra-high-definition formats. VPUs seamlessly encode and decode ultra-high-resolution content like 8K HDR. They also integrate AI capabilities for overlays, graphics, and real-time subtitling. However, neglecting codec compatibility can create issues. Ensure your chosen VPU supports all required formats. Additionally, overlooking future-proofing can lead to costly upgrades later. Opt for VPUs that handle emerging standards like AV1 to remain ahead of the curve.

Simplifying Workflow Management with Bitstreams Edge

Managing high-volume video streams can be complex. Bitstreams Edge, a VPU control interface, simplifies this process, making it easier for engineers to monitor, configure, and optimize their workflows. Bitstreams Edge streamlines live stream setup with intuitive templates and features a comprehensive dashboard to monitor system health, error alerts, and resource loads. It utilizes AI-powered subtitling through OpenAI Whisper for real-time voice-to-text captions. The platform supports protocols like SRT, RTMP, DASH, HLS, multicast, and UDP. Use Bitstreams Edge’s reporting logs to identify bottlenecks and optimize resource allocation effectively.

Real-World Success Stories: Proven Impact

Companies like API.video have reduced costs to $0.0002 per minute, enabling free transcoding services. Mayflower Live Streaming transitioned from CPUs to NETINT VPUs, saving $8.6 million in the first year and cutting encoding OPEX by over 90%. Zapping, a streaming service in Latin America, halved its hardware footprint while improving live sports streaming quality with VPUs.

The Path Forward: Revolutionizing Video Streaming Workflows

VPUs offer a transformative opportunity to optimize your video streaming infrastructure. By integrating this technology, you can achieve significant cost savings, enhance performance and scalability, simplify workflow management with tools like Bitstreams Edge, and future-proof your operations with advanced codec support and AI features.

To get started, identify key pain points in your video processing workflows to assess your needs. Evaluate solutions and choose the right VPU model based on your scalability and codec requirements. Begin with a pilot deployment to measure performance gains and use management tools like Bitstreams Edge to streamline operations. The evolution of video streaming demands innovative solutions. VPUs not only meet these challenges but position your platform for sustained growth.

The time to act is now – start leveraging the power of VPUs to stay ahead in the ever-competitive video streaming landscape.

The time to act is now – start leveraging the power of VPUs to stay ahead in the ever-competitive video streaming landscape.

Schedule a meeting to learn how NETINT VPUs can enhance live streaming with energy-efficient, scalable solutions.

ACCESS NOW: ASIC-Based Transcoding
for High-volume Use Cases
Including social media, broadcast, interactive platforms, and service providers

ACCESS NOW

Efficient silicon is necessary but not sufficient. What it takes to make video processing deployable at data center scale.

The Abstraction Breaks Down

At a modest scale, video infrastructure decisions can remain largely abstract. Engineering teams optimize for throughput or quality and assume the underlying systems will accommodate whatever they build. When concurrency is limited and workloads are intermittent, inefficiency shows up primarily as a higher bill.

At production scale, that assumption stops holding. Video workloads are sustained, power-intensive, and concurrency-driven. As deployments grow, teams hit constraints that no amount of software optimization can solve: power budgets per rack, thermal dissipation limits, floor space availability, and the sheer operational costs of maintaining large server fleets.

In these environments, inefficiency does not simply raise costs. It determines whether a given workload can physically exist within real data center constraints. The question shifts from how fast, to how deployable?

THE PHYSICAL CONSTRAINT STACK

Hard limits that cannot be abstracted away at scale

At modest scale, there are budget items. At production scale, they become hard deployment constraints 

Exhibit 1: The Physical Constraint Stack. At scale, these limits compound and become non-negotiable.

Why General-Purpose Servers Struggle

Historically, video processing has relied on general-purpose compute, with GPUs added when acceleration was needed. That model provides flexibility, but it introduces challenges when applied to dense, sustained video workloads.

CPU- and GPU-based encoding often exhibits fluctuating power draw depending on workload mix and tuning. High-performance accelerators can concentrate heat in ways that complicate airflow and cooling design. Fewer streams per server means more rack space, more cabling, and more operational overhead. And capacity planning becomes probabilistic rather than deterministic, forcing infrastructure teams to overprovision power, cooling, and space to accommodate worst-case scenarios.

The result is a planning model built on estimates and margins rather than known, repeatable behavior. Every variable that fluctuates adds a layer of uncertainty to facility design, procurement, and operations.

KEY INSIGHT
At production scale, the distinction between a variable workload and a predictable one is not a matter of preference. It is the difference between overprovisioning by 30-40% and sizing to actual demand.

VPUs Change the Planning Model

Within the VPU Ecosystem, Video Processing Units shift the planning model from theoretical performance to predictable physical behavior. From an infrastructure perspective, three changes matter most.

Deterministic power profiles. Dedicated video silicon delivers stable, bounded power consumption per stream. This simplifies power budgeting at the server and rack level. Teams can plan around known energy draw and compute capacity rather than peak estimates.

Higher density per footprint. More streams per server means fewer systems are needed to deliver a given level of video encoding capability. Rack count drops. Cabling simplifies. Operational overhead shrinks.

Thermal predictability. Consistent thermal output improves cooling efficiency and reduces the need for conservative design margins. Airflow planning becomes straightforward rather than defensive.

PLANNING MODEL COMPARISON

Exhibit 2: How VPU-accelerated infrastructure changes planning assumptions across five key dimensions.

Together, these characteristics let video infrastructure teams plan capacity based on concrete physical constraints rather than peak-performance estimates. With VPUs, the planning model shifts from probabilistic to deterministic.

From Silicon to System: Where Server Design Earns Its Keep

Efficient silicon alone is not sufficient. To be deployable at scale, acceleration must be integrated into systems designed for reliability, serviceability, and lifecycle management.

Server design for sustained video workloads involves considerations that go well beyond raw performance: chassis airflow and component placement, power supply sizing and redundancy, PCIe layout and expansion flexibility, firmware stability and platform validation, and long-term serviceability across multi-year deployments.

This is the kind of engineering that Dell Technologies has refined across decades of enterprise server design. The PowerEdge platform, for example, offers PCIe Gen5 expansion across a range of form factors. The R760 supports up to eight PCIe slots with a mix of Gen4 and Gen5 risers, giving infrastructure teams the flexibility to pair NVMe VPU accelerators alongside networking and storage cards without slot contention. For denser configurations, the XE series delivers higher accelerator counts in validated thermal envelopes. The underlying principle is the same: provide enough expansion headroom so that adding purpose-built acceleration does not require rearchitecting the server.

Thermal management is equally critical. Dell’s Multi-Vector Cooling (MVC) framework combines hardware innovations with AI-driven adaptive fan control. The latest generation uses a fuzzy-logic closed-loop controller that adjusts fan speed based on real-time thermal sensor input and power telemetry, optimizing for both cooling performance and acoustic efficiency. For video workloads, where sustained thermal output is the norm rather than the exception, this kind of adaptive control keeps components within safe operating ranges without the energy waste of running fans at full speed continuously.

In this context, VPUs are not treated as experimental accelerators bolted onto general-purpose hardware. They are production components within validated server platforms, designed to operate reliably under sustained load for years at a time.

This distinction matters. Experimental hardware often works in the lab but creates problems at scale: inconsistent firmware behavior, unvalidated thermal profiles, unclear support paths. Production-grade integration eliminates these risks before they reach the data center floor.

Density Changes Operational Economics

When stream density increases, the operational math changes at every level. Fewer servers mean fewer rack positions. Fewer racks mean less power, less cooling, and less floor space. Fewer systems to manage means fewer failure points, less cabling, and simpler monitoring.

This is not a marginal improvement. When you move from dozens of streams per server to hundreds, you are not optimizing an existing model. You are changing the category of infrastructure required. Deployments that once filled rows of racks can fit in a fraction of the space, with proportional reductions in operational complexity.

Dell’s iDRAC management platform amplifies this advantage. Each PowerEdge server includes an integrated remote access controller capable of streaming real-time telemetry, covering power consumption, thermal sensors, fan speeds, and component health, via the Redfish API. For video deployments, where dozens of servers may run identical sustained workloads, this telemetry makes it straightforward to monitor fleet health from a single pane. When a VPU’s thermal signature shifts or a power supply nears its rated capacity, operations teams can catch the issue before it becomes a service disruption.

For organizations operating multiple facilities or expanding into new regions, this density advantage compounds. Each new deployment starts from a smaller, simpler, more predictable baseline.

Planning for Lifecycle, Not Just Launch

Video infrastructure is rarely static. Platforms evolve, codecs change, and demand grows over time. Deployable systems must account for multi-year support horizons, incremental capacity expansion, hardware refresh cycles, and operational continuity during upgrades.

INFRASTRUCTURE LIFECYCLE

Where pedrictable workload behavior reduces friction

Exhibit 3: The Infrastructure Lifecycle. Predictable workloads make each phase iterative rather than disruptive.

Predictable, efficient workloads reduce the friction at every stage of this lifecycle. When capacity scales linearly and power behavior is stable, infrastructure planning becomes iterative rather than disruptive. A hardware refresh is an upgrade, not a redesign. A capacity expansion is a known quantity, not a new engineering project.

Dell’s OpenManage Enterprise suite is designed to automate precisely these lifecycle tasks across large fleets. It can manage up to 8,000 PowerEdge devices from a single console, handling firmware updates, configuration compliance, and hardware health monitoring through policy-based automation. For video operators who may be deploying dozens or hundreds of VPU-equipped servers, the ability to push firmware updates, validate configurations, and track hardware health across an entire fleet without touching individual machines is a material reduction in operational burden. Combined with Ansible integration for infrastructure-as-code workflows, the management layer scales alongside the hardware

THE BIGGER PICTURE
When servers are designed around predictable, efficient video processing, higher-level architectural decisions become easier. Compute can be placed closer to users when latency matters. Capacity can scale without rearchitecting facilities. Operational teams can support video growth without proportional increases in complexity.

Infrastructure Is the Foundation

Efficiency in video processing is often discussed in terms of cost or performance. At infrastructure scale, it becomes a question of feasibility. Can this workload physically exist in this facility? Can it be supported over time? Can it scale without rebuilding the environment around it?

From silicon to rack, deployable video infrastructure depends on predictable behavior, density, and supportable system design. VPUs change what is possible at the hardware level. Server platforms translate that efficiency into systems that can be deployed, operated, and evolved over time. When the silicon is purpose-built, like the VPU is for video, and the server platform is engineered for sustained, real-world operation, what once required rows of general-purpose machines can fit in a fraction of the space, with a fraction of the operational overhead. This is why the world’s largest video platforms and social networks all use custom silicon to encode and process video.

Within the VPU Ecosystem, infrastructure is not an afterthought. It is the foundation that makes efficiency real.

VPU Ecosystem Series  This article is part of a series examining how purpose-built video silicon is reshaping infrastructure, economics, and architecture across the video delivery chain. Each installment explores a different partner’s perspective within the ecosystem.

ACCESS NOW:  ASIC-Based Transcoding
for High-volume Use Cases
Including social media, broadcast, interactive platforms, and service providers

ACCESS NOW

StreamVX and NETINT Technologies partner to create ultra-low power/high-density video encoding solutions for the Data center.

Soaring energy prices in Europe in 2022 and new restrictions on the construction of hyperscale data centers have forced streaming video platforms to look for ways to reduce their energy consumption costs while simultaneously expanding their encoding capacity.

Collaborating to meet the needs of streaming video platforms in current volatile market conditions

StreamVX, a leading European supplier of streaming video solutions, and NETINT Technologies, a pioneer in ASIC (Application Specific Integrated Circuit) encoding technology, are collaborating to create the industry’s lowest-power, highest-density video encoding solutions that will meet the needs of streaming video platforms in current volatile market conditions.

Energy prices in Europe are projected to soar and remain high for the foreseeable future. As a result, governments are implementing energy conservation policies. These new government policies are having a real and immediate effect on the streaming video industry. Some EU (European Union) member countries have already restricted the construction of new hyper-scale data centers, citing energy and physical-space shortfalls.

These restrictions severely limit hyper-scale video platforms’ ability to increase encoding capacity, which is needed to deploy new streaming services. Pushback on new hyper-scale data centers is not constrained to Europe, with new data center builds in Asia and North America also facing challenges.

With difficulty in deploying new hyper-scale data centers, “rack-locked” video platforms are looking for solutions that enable them to increase their encoding density in their existing server footprint while minimizing energy consumption.

Data center operators face a 40% increase in costs due to the looming energy crisis.

Data center operators face a 40% increase in costs due to the looming energy crisis.
Global events in 2022 are driving uncertainty in the data center industry. With a looming energy crisis potentially driving cost increases of up to 40%, streaming video platforms need to act decisively to mitigate massive OPEX increases.
A great strategy is to migrate from software encoding to ASIC-based encoding, which can reduce OPEX by as much as 40x, enabling streaming video platforms to remain profitable amid unstable energy costs.

Next-generation streaming solutions

StreamVX creates next-generation streaming solutions that virtualize the functionality of the infrastructure used by broadcasters, MSOs, and Telcos. VxOS, Stream VX’s advanced platform, enables the virtualization of various processing functions, which are packaged as vxAPPs enabling optimized workflows for applications, including live streaming, OTT/SVOD, and digital ad insertion. The flexibility of StreamVX’s solutions allows them to run on-premises, in the AWS (Amazon Web Services) cloud, or in any other public or private cloud.

StreamVX’s most advanced encoding solutions, powered by NETINT VPUs (Video Processing Units), offer best-in-class, low-power, low-latency encoding performance. NETINT VPUs utilize ASIC-based video processing to real-time encode AV1, HEVC, and H.264 at up to 8x 4Kp60 or 32x 1080p60 streams per VPU. A StreamVX encoding server fully populated with 10x NETINT VPUs offers up to 40x greater encoding capacity than existing software encoding solutions.

My personal mission is to reduce carbon footprints via dense technology

Our top expertise areas are reducing infrastructure and minimizing costs. I am an environmentalist whose personal mission is to reduce carbon footprints via dense technology. NETINT provides these types of solutions – said Szymon Karbowski, CEO of StreamVX

Minimize host CPU utilization

By offloading complex encode/decode processing to its onboard ASIC, NETINT VPUs minimize host CPU utilization. The result significantly improved real-time transcoding density compared to CPU/software-based encoding.

NETINT VPUs utilize as little as 7 watts per module, enabling StreamVX encoding solutions to offer, in addition to a 40x increase in encoding density, a 40x reduction in power consumption compared to existing CPU/software-based encoding.

StreamVX and NETINT are delivering a scalable encoding platform that meets the critical requirements of streaming video platform operators, giving them the means to expand their encoding capacity while simultaneously lowering their energy costs and enabling them to remain profitable under the current unpredictable global business environment.

The demand for high-density video decoding solutions has skyrocketed, particularly in security, surveillance, and advanced video analytics use cases. This comprehensive analysis examines three cutting-edge approaches to meet these demanding requirements: software decoders running on CPUs, GPU-accelerated decoders, and the innovative NETINT Quadra VPU hardware decoders. Each solution offers unique advantages but differs significantly in its ability to process multiple concurrent video streams in real time, a critical factor for modern security and analytics applications.

While CPU-based solutions provide flexibility, they need help with linear cost scaling and high power consumption when handling large volumes of video streams. GPU decoders offer improved performance and seamless AI integration. Still, the decoder can be a system bottleneck, with just 15% of the die devoted to video IP (decoding and encoding).

In contrast, NETINT Quadra VPUs are the most cost-effective and energy-efficient option for large-scale deployments. These specialized processors deliver superior performance per watt, lower operational costs, and excellent scalability, making them ideal for handling the high volume of streams typical in multi-camera setups.

When paired with GPUs for AI processing, VPUs create a powerful combination capable of efficiently tackling complex tasks such as object recognition, threat detection, and anomaly identification across surveillance networks. This synergy enhances the speed and accuracy of video analysis. It promises a new era of intelligent, responsive security systems that can process vast amounts of visual data with unprecedented efficiency and insight.

Software Decoding on CPUs

While software decoders running on x86 or Arm CPUs offer flexibility, they come with significant cost and energy drawbacks. The linear cost scaling and high power consumption make them less efficient for large-scale video streaming operations. However, their ease of management, adaptability, and ability to handle various tasks beyond video decoding make them a common option. Let’s look deeper into CPUs.

Scalability vs. Cost

Software decoders running on CPUs in a cloud environment offer significant flexibility and scalability, allowing companies to adjust resources based on demand. This flexibility ensures that systems can handle peak loads, such as during major live events, by provisioning additional CPU instances to accommodate viewer spikes. However, this scalability comes at a cost.

The primary issue is linear cost scaling,an increase in the number of video streams requires a proportional increase in CPU instances, resulting in a direct and linear rise in costs. For instance, decoding 100 streams costs $X, and decoding 200 streams will cost approximately $2X, assuming no volume discounts from the cloud provider. This lack of economies of scale makes CPU-based software decoders less cost-effective for large-scale video streaming services.

A study comparing CPU-based transcoding to ASIC-based solutions found that CPU-based setups require more servers and consume more power, leading to higher operational expenses over time. Specifically, supporting a large number of video channels with CPUs can be more than 20 times less efficient than using specialized hardware such as ASICs.

Energy Consumption

As general-purpose processors, CPUs consume more power per decoded stream than dedicated hardware like GPUs or VPUs (Video Processing Units). This high power consumption translates to increased operational costs in terms of energy bills and the cooling infrastructure required to maintain optimal operating conditions. CPUs running software decoders typically draw more power and necessitate more cooling than hardware-accelerated solutions, making them less energy-efficient. Higher power consumption also means higher cooling requirements, further inflating operational costs. Data centers must invest in robust cooling systems to manage the heat generated by numerous active CPU instances, adding to the total cost of ownership (TCO).

Cloud Infrastructure

One of the key advantages of using CPUs in a cloud environment is the simplified management of resources. Cloud infrastructure allows for easy provisioning and de-provisioning of CPU instances, automated scaling, and centralized management of resources. This ease of management can reduce the administrative overhead and complexity of maintaining a large-scale video streaming infrastructure. However, despite these management benefits, the cost per instance in a cloud environment only benefits from significant economies of scale when using CPUs for video decoding. Cloud providers charge based on usage, and since the demand for video decoding scales linearly, so do the costs. This linear cost scaling makes CPU-based solutions less attractive for large-scale operations compared to specialized hardware solutions that can offer better cost efficiency at scale.

Technical Benefits

CPUs are highly versatile and can handle various tasks beyond video decoding, making them suitable for environments with varied workloads. These include data processing, running algorithms, and managing databases, making CPUs a valuable component in multi-functional environments.

CPUs’ versatility means they can adapt to different video codecs and formats without requiring specialized hardware. This adaptability is particularly useful in dynamic environments where the requirements for video processing can change frequently. One advantage of software decoders on CPUs is the ease of updating and maintaining the software. Updates can be deployed quickly and efficiently across the cloud infrastructure, ensuring the decoders remain compatible with the latest video codecs and standards.

GPU Decoding

Operational and Cost Analysis

Initial Investment: GPUs require a substantial initial investment compared to CPUs. High-performance GPUs, such as the NVIDIA Tesla series, can cost several thousand dollars per unit. The performance gains in concurrent stream processing justify this higher upfront cost. GPUs are designed to handle parallel tasks efficiently, making them suitable for applications that require processing multiple video streams simultaneously.

Operational Costs: GPU instances are significantly more expensive in a cloud environment than CPU instances. AWS charges considerably more for its p3 instances (which include NVIDIA Tesla V100 GPUs) than for standard compute-optimized instances. This cost delta impacts operational budgets, especially for services requiring continuous, high-volume video decoding.

However, the improved performance per dollar for concurrent stream processing helps offset some of these costs. GPUs can decode multiple video streams simultaneously, leading to better utilization rates and overall cost efficiency in high-demand scenarios.

GPUs are inherently designed for parallel processing, leading to better power efficiency than CPUs for high-volume video decoding tasks. The architecture of GPUs, with thousands of smaller cores, allows for the simultaneous processing of many threads. This is particularly beneficial for video decoding, where each frame can be processed in parallel, reducing the time and energy required for decoding.

Studies have shown that GPUs can be more than twice as power-efficient as CPUs for specific workloads, including video decoding. This increased efficiency reduces operational costs and minimizes the environmental impact of data centers by lowering the power consumption and cooling requirements.

Technical Benefits

Concurrent Stream Processing: GPUs excel at handling multiple video streams simultaneously, making them ideal for real-time applications. Their architecture, designed for parallelism, allows for efficient processing of tasks such as video decoding, where multiple frames or streams must be processed concurrently. This capability is crucial for live streaming services and applications requiring real-time video analytics.

Latency and Throughput: GPUs’ high throughput and low latency make them suitable for tasks requiring immediate processing and response, such as AI-based video analytics. This performance advantage is critical in live sports streaming or interactive video applications, where delays can significantly impact user experience.

Architecture for AI Workloads: GPU’ architecture is particularly well-suited for AI workloads. With their ability to handle large-scale parallel computations, GPUs can accelerate the processing of complex AI models used in video analytics. Tasks such as object recognition, threat detection, and anomaly detection benefit from GPUs’ parallel processing power, resulting in faster and more accurate analyses.

Lower Latency and Higher Throughput: GPUs provide lower latency and higher throughput for AI workloads than CPUs. This is because GPUs can simultaneously process multiple elements of an AI model, reducing the time required for tasks like neural network inference. For instance, in video surveillance, GPUs can quickly analyze video feeds to detect potential threats or identify objects, providing real-time insights and enhancing security measures.

GPUs offer a compelling solution for video decoding. While the initial investment in GPU hardware is higher and operational costs more than those of CPUs, the improved performance makes GPUs attractive for some workloads. Here’s an example cloud gaming architecture where the GPU utilizing P2P DMA is able to transfer frames with zero latency to the VPU.

NETINT Quadra VPU Hardware Decoders

Operational and Cost Analysis

Cost Efficiency: Compared to CPUs and GPUs, the cost per decoded stream with NETINT VPUs is substantially lower – for most applications, the difference in OPEX and CAPEX is 80% lower compared to CPU architectures. NETINT’s specialized hardware is designed specifically for video decoding, leading to more efficient resource utilization. The NETINT Quadra Video Server, integrated with VPUs, can transcode video streams at a fraction of the cost per stream compared to traditional CPU or GPU-based solutions.

Optimized Power Consumption: NETINT VPUs are engineered to optimize power consumption for video decoding tasks. The Quadra T1U draws 27 watts while encoding and decoding 32 1080p30 streams. Unlike general-purpose CPUs and GPUs, VPUs are designed to perform video processing tasks with the highest quality and efficiency. This specialization leads to lower power usage per stream.

Reduced Cooling Requirements: VPUs’ reduced power consumption also means less heat generation, lowering cooling requirements. Data centers can save on cooling infrastructure and energy costs, decreasing operational expenses. Studies and practical deployments have shown that VPU-based systems can reduce power consumption by over 20x compared to CPU-based systems under similar workloads.

Cost Savings at Scale: As the demand for video streams increases, the cost per stream using VPUs decreases. This is due to the efficient use of resources and the ability to handle multiple streams simultaneously without a proportional increase in power or cooling requirements. The host CPU operates at just 20 to 35% in standard operation. This scalability ensures that VPUs remain cost-effective even for very large deployments.

Technical Benefits

High Performance with Low Latency: NETINT VPUs are specialized hardware designed explicitly for video decoding. This specialization allows them to offer high performance with low latency, critical for real-time streaming and high-volume applications. The VPUs can process video streams quickly and efficiently, ensuring smooth playback and minimal delay.

Real-time Streaming: VPUs’ low latency is a significant advantage for applications like live sports, news broadcasting, and interactive video services. Real-time processing capabilities ensure that viewers experience minimal lag, enhancing the overall user experience.

Dedicated Hardware

Consistent Performance: As specialized hardware, NETINT VPUs deliver consistent performance that general-purpose CPUs and GPUs cannot match. This consistency is crucial for applications where reliability and predictability are essential. Unlike CPUs and GPUs, which may experience performance variability depending on the workload, VPUs maintain stable and high performance under varying conditions.

Avoiding Variability: VPUs’ dedicated nature means they are not subject to the same performance fluctuations that can affect general-purpose processors running software. This stability ensures that video streams are processed reliably without the risk of dropped frames or latency spikes, which is critical for maintaining high-quality video delivery.

Use Case Analysis

CPUs

Flexibility vs. Performance: CPUs are inherently versatile and can handle various computational tasks. This flexibility makes them valuable in environments where workloads vary significantly. However, when it comes to AI-driven video analytics, CPUs need help to meet the intensive processing demands, especially at scale. The computational requirements for tasks such as object recognition, threat detection, and anomaly detection are substantial, and CPUs often need more parallel processing capabilities to handle these efficiently.

Prohibitive Costs: The cost of using CPUs for AI-driven video analytics becomes prohibitive at scale. Each additional video stream necessitates more CPU power, leading to linear cost scaling. As the number of streams increases, so does the need for additional CPU instances, which directly translates to higher operational costs. This linear scalability without economies of scale makes CPUs a less viable option for large-scale AI video analytics.

GPUs

Performance Optimization: GPUs are designed for parallel processing, making them well-suited for AI workloads. Their architecture, with thousands of small cores, allows for the efficient handling of multiple video streams and real-time analytics tasks. For example, tasks like neural network inference, common in AI-driven video analytics, can be processed much faster on GPUs than on CPUs.

Operational Costs: Despite their performance advantages, GPUs’ operational costs remain high. Cloud providers charge premium rates for GPU instances, which can significantly impact the budget of organizations relying on high-volume AI analytics. The performance gains justify the cost, but these expenses can add up for sustained operations, especially when dealing with continuous, large-scale video streams.

NETINT VPUs

Offloading Decoding Tasks: NETINT VPUs offer a specialized solution for video decoding, allowing GPUs to focus solely on AI processing. By offloading the decoding task to VPUs, the system can achieve better overall performance and efficiency. VPUs handle the video stream decoding efficiently, freeing up GPU resources for intensive AI tasks such as real-time analytics and inferencing.

Cost Savings and Performance Enhancements: This hybrid approach of using VPUs for decoding and GPUs for AI processing can yield significant cost savings and performance enhancements. VPUs are designed to be highly efficient at decoding, reducing power consumption and operational costs. Combined with GPUs for AI processing, this setup provides a scalable and cost-effective solution for AI-driven video analytics, particularly in high-volume scenarios.

High-Volume Decoding for Transcoded Video Streams

CPUs

Efficiency and Cost: CPUs become less efficient and more expensive as the volume of decoded streams increases. The linear cost scaling associated with CPU-based decoding means that each additional stream incurs a proportional increase in computing cost. As the demand for video streams grows, this approach quickly becomes cost inefficient. The higher power consumption of CPUs also contributes to increased operational costs, making them a less desirable option for high-volume decoding.

GPUs

Performance in High-Volume Decoding: GPUs offer better performance for high-volume decoding than CPUs. Their ability to handle parallel processing makes them more suitable for scenarios where multiple video streams must be decoded simultaneously. However, the high operational costs associated with GPU instances in cloud environments remain a significant drawback. While GPUs provide the necessary performance, the financial burden can be substantial, particularly for continuous, large-scale operations. AI workloads that require decoding many video streams typically run out of video IP capacity before AI inference capability.

NETINT VPUs

Cost-Effective and Efficient Solution: NETINT VPUs excel in high-volume decoding scenarios, providing a cost-effective and efficient solution. VPUs are specialized hardware designed specifically for video processing tasks, offering superior performance and energy efficiency compared to general-purpose CPUs and GPUs. The cost per decoded stream with VPUs is significantly lower by up to 20x. Frequently, for every ten CPU-based machines in a data center, nine may be removed.

Scalability and Performance: VPUs’ specialized nature ensures high performance and scalability. They can handle a large number of video streams simultaneously without a proportional increase in power consumption or operational costs. This scalability is crucial for large-scale deployments, where maintaining high performance and cost efficiency is essential. Organizations can achieve significant operational savings by leveraging VPUs for high-volume decoding while ensuring reliable and high-quality video delivery.

Another advantage of NETINT VPUs is their ability to reduce storage costs by as much as 80%.

Conclusion

The use of NETINT VPUs for video decoding offers a range of benefits over traditional CPU and GPU solutions, particularly in high-volume and AI-driven video analytics scenarios. While CPUs provide flexibility, they need help with performance and cost efficiency at scale. GPUs deliver better performance but at a higher operational cost and frequently with limited availability.

NETINT VPUs, on the other hand, offer a specialized, efficient, and cost-effective solution for optimizing video decoding and enhancing overall system performance. By integrating VPUs into their infrastructure, organizations can achieve substantial cost savings, improved energy efficiency, and superior scalability, making VPUs an ideal choice for modern video streaming and analytics applications.

For more detailed insights, the article “Economics of Transcoding” analyzes the cost benefits associated with hardware-accelerated transcoding solutions.

ACCESS NOW: ASIC-Based Transcoding
for High-volume Use Cases
Including social media, broadcast, interactive platforms, and service providers

ACCESS NOW

Cloud computing offers an exciting vision where immersive user experiences are decoupled from the compute resources necessary to deliver those experiences. This decoupling gives service providers a special opportunity to deliver a broad range of interactive services using a wide variety of devices from smartphones to connected televisions to game consoles to computers.

However, the explosive growth of these new video services comes at a cost. In this article, you’ll learn about the adverse impact on the environment as software-based encoding infrastructures struggle to meet this new demand. In addition to the environmental impact, software-based encoding solutions do not scale economically. We’ll show how new technologies like ASIC-based video transcoders and advanced video codecs are reducing TCO and reducing carbon emissions impacts making new services like cloud gaming, virtual desktop, real-time video conferencing, and AR/VR viable.

Service providers offering interactive video services also face additional technical challenges including demanding latency requirements. Let’s look at how cloud architectures must evolve to adapt to these new operational requirements.

The Cloud – Reimagined

Placing compute resources in the Cloud enables a new generation of interactive video services to operate with the responsiveness of a local application. New interactive services shift unprecedented stress onto network computing resources, requiring service providers to look to more efficient video encoding technologies so that they can scale their operations with better cost and energy efficiency.

New cloud architectures that combine hardware encoders that are hosted on commodity x86 and Arm-based servers promise to resolve user experience gaps while changing the economics of high-scale video for the better. These architectures distribute the video encoding and compute functions closer to the user, while reducing end-to-end latency to the range of 100-200ms, decreasing backhaul traffic, and enabling new forms of data/sensor/display processing.

Consumer applications like interactive social video, cloud, and mobile game streaming, virtual and augmented reality enable users to engage anywhere, at any time, and on any device using an application service in the cloud running remotely. The primary way to reduce latency and improve the responsiveness and viewer experience is to position compute resources closer to the user. This means taking a decentralized approach where some of the architectural computing blocks are located outside of the data center (cloud) to the network edge or alternate location.

As we move these compute resources closer to the user, achieving the economy of scale needed for hyperscale data centers becomes challenging. Fortunately, these costs can be managed by placing application servers and video encoding systems together in regional/metropolitan data centers so that video and application control traffic need only to hop from the regional data center to the local base station or the network service provider central office. In so doing, system latency is significantly reduced, and video quality is increased by employing purpose-built video encoding hardware powered by ASICs that are hosted on x86 and Arm-based servers.

Advanced Video CODECs are driving the need for ASICs.

Video technology is instrumental in nearly every app or product in the market and this means balancing bitrate, video quality, framerate, and resolution will forever drive the development of next-generation codec technologies that can deliver a higher compression advantage. As new video codec standards are developed, they come with a promise of halving the bitrate from the current/previous standard. Consider that H.264/AVC delivered a 50% bitrate efficiency over MPEG2, while HEVC delivers a 50% advantage over H.264/AVC, and AV1 is well on track to beat HEVC by 50%. But with each standard comes an ever-higher order of computing complexity. The most significant inhibitor to a service adopting a more efficient video codec is the limitation on how much computing resources can be spent on video encoding tasks.

Environmental and Economic Considerations for the Cloud.

Dedicated video encoding and image processing using ASIC-based video encoders can radically reduce the hardware required for video processing, enabling the video encoding operation to be decentralized and operated at regional points of presence to create a balance between Cloud computing and the economic realities of operating a consumer service at hyper-scale.

For services needing to scale, hurdles include cost, quality of service, visual quality, and motion-to-photon latency. Unfortunately, the cost of deployment and operations increases sharply as compute resources move closer to the user. Making cloud computing affordable for interactive applications requires network topologies that are able to balance the economics of high-scale with the need to drive low latency and excellent visual quality. ASIC-based encoding has enabled a new generation of video transcoders that can improve encoding server density by a factor of ten while reducing power consumption and improving the environmental impact by a factor of twenty.

NETINT is a pioneer in ASIC-based encoding with a family of dedicated video transcoders that are plug-n-play to your existing x86 and Arm-based servers. These innovative video transcoding solutions can radically reduce your server footprint for video encoding and they are a unique solution that can scale economically to deliver any native desktop, mobile, or head-mounted display application from the cloud, while simultaneously minimizing environmental impacts and cutting TCO.

Encoding technology refreshes are seldom CFO driven. For European data centers, over the next few years, they may need to be as reducing power consumption in data centers becomes a primary focus.

Few European consumers or businesses need to be reminded that they are in the midst of a power crisis. But a recent McKinsey & Company article entitled Four themes shaping the future of the stormy European power market provides interesting insights into the causes of the crisis and its expected duration. Engineering and technical leaders, don’t stop reading because this crisis will impact the architecture and technology decisions you may be making.

The bottom line, according to McKinsey? Buckle up, Europe, “With the frequency of high-intensity heat waves expected to increase, additional outages of nuclear facilities planned in 2023, and further expected reductions in Russian gas imports, we expect that wholesale power prices may not reduce substantially (defined as returning to three times higher than pre-crisis levels) until at least 2027.” If you haven’t been thinking about steps your organization should take to reduce power consumption and carbon emissions, now is the time.

https://youtu.be/yiYSoUB4yXc

HARD QUESTIONS ON HOT TOPICS – EUROPEAN ENERGY CRISIS AS PER MCKINSEY REPORT
WATCH THE FULL CONVERSATION ON YOUTUBE: https://youtu.be/yiYSoUB4yXc

The Past

The war in Ukraine is the most obvious contributor to the energy crisis, but McKinsey identifies multiple additional contributing factors. Significantly, even before the War, Europe was in the midst of “structural challenges” caused by its transition from carbon-emitting fossil fuels to cleaner and more sustainable sources like wind, solar, and hydroelectric.

Then, in 2022, the shock waves began. Prior to the invasion of Ukraine in February, Russia supplied 30 percent of Europe’s natural gas, which dropped by as much as 50% in 2022, and is expected to decline further. This was exacerbated by a drop of 19% in hydroelectric power caused by drought and a 14% drop in nuclear power caused by required maintenance that closed 32 of France’s 56 reactors. As a result, “wholesale prices of both electricity and natural gas nearly quadrupled from previous records in the third quarter of 2022 compared with 2021, creating concerns for skyrocketing energy costs for consumers and businesses.”

Figure 1. As most European consumers and businesses know, prices skyrocketed in 2022
and are expected to remain high through 2027 and beyond.

Four key themes

Looking ahead, McKinsey identifies four key themes it expects to shape the market’s evolution over the next five years.

• Increase in Required Demand

McKinsey sees power usage increasing from 2,900 terawatt-hours (TWh) in 2021 to 3,700 TWh in 2030, driven by multiple factors. For example, the switch to electric cars and other modes of transportation will increase power consumption by 14% annually. In addition, the manufacturing sector, which needs power for electrolysis, will increase to 200 TWh by 2030.

• The Rise of Intermittent Renewable Energy Sources

By 2030, wind and solar power will provide 60% of Europe’s energy, double the share in 2021. This will require significant new construction but could also face challenges like supply chain issues, material shortages, and a scarcity of suitable land and talent.

• Balancing Intermittent Energy Sources

McKinsey sees the energy market diverging into two types of sources; intermittent sources like solar, wind, and hydroelectric, and dispatchable sources like coal, natural gas, and nuclear that can be turned on and off to meet peak requirements. Over the next several years, McKinsey predicts that “a gap will develop between peak loads and the dispatchable power capacity that can be switched on to meet it.”

To close the gap, Europe has been aggressively developing clean energy sources of dispatchable capacity, including utility-scale battery systems, biomass, and hydrogen. In particular, hydrogen is set to play a key role in Europe’s energy future, as a source of dispatchable power and as a means to store energy from renewable sources.

All these sources must be further implemented and massively scaled, with “build-outs remaining highly uncertain due to a reliance on supportive regulations, the availability of government incentives, and the need for raw materials that are in short supply, such as lithium ion.”

• New and evolving markets and rules

Beyond temporary measures designed to reduce costs for energy consumers, European policymakers are considering several options to reform how the EU energy market operates. These include

• A central buyer model: A single EU or national regulatory agency would purchase electricity from dispatchable sources at fixed prices under long-term contracts and sell it to the market at average cost prices.
• Decoupled day-ahead markets: Separate zero marginal cost energy resources (wind, solar) and marginal cost resources (coal) into separate markets to prioritize dispatching of renewables.
• Capacity remuneration mechanism: Grid operator provides subsidies to producers based on forecast cost of keeping power capacity in the market to ensure a steady supply of dispatchable electricity and protect consumers.

McKinsey closes on a positive note, “Although the European power market is experiencing one of its most challenging periods, close collaboration among stakeholders (such as utilities, suppliers, and policy makers) can enable Europe’s green-energy transition to continue while ensuring a stable supply of power.”

The future of the European power market is complex and subject to many challenges, but policymakers and stakeholders are working to address them and find solutions to ensure a stable and affordable energy system for consumers and businesses.

In the meantime, the mandate for data centers isn’t new as video engineers are being asked to reduce power consumption to save OPEX, reduce carbon footprint to ensure ESG metrics are hit by the company, and minimize the potential disruption of energy instability.

If you’re in this mode, NETINT’s ASIC-based transcoders can help by offering the lowest available power draw of any silicon solution (CPU, GPU, FPGA), and thus the highest possible density.