The AV1 Codec: Assessing its Evolution Since Launch

AV1 is an emerging and promising codec that is the first high-efficiency video processing codec with a royalty-free license from Alliance for Open Media (AOMedia). This firm counts Amazon, Apple, Facebook, Google, Intel, Microsoft, and Netflix among its founding members. Depending on which stage in the media pipeline they are deployed in, video codecs can be categorized into three types considering the kind of data processing involved. The three categories are capture, editing, and release. This article explores the fundamentals of video codecs, especially in the context of OTT and social media streaming, as well as how far could a royalty-free processor take your business in terms of efficiency and profitability.

Decoding the Video Codecs

As the name suggests, a capture codec captures as much information as possible. At this stage of the media pipeline, from an acquisition perspective, production needs the full dynamic range of luminance and chrominance captured as best as possible for further downstream processing. Capture codecs such as ProRes RAW are necessary to pull off this task.

An editing codec such as Avid’s DNxHR, commonly used in post-production, is designed to be easy to edit while producing smaller file sizes than capture codecs. Release codecs, aka the final output, need to be the best-looking image at the smallest file size since the file will mostly be played by viewers linearly with no image manipulation, unlike capture and editing codecs.

Open-source Nature of AV1

The AV1 codec was finalized by the AOM in 2018 and is intended to be used as a royalty-free release codec. The royalty-free bit matters considerably because, in the past, usage of licensed codecs resulted in a payout of hefty royalties to the patent licensing pools (e.g., MPEG-LA or Access Advance) that are responsible for collecting royalties on behalf of the companies that created the overall codec design.

Depending on the codec being used, e.g., HEVC, a company would end up paying royalties whether it was, say, an OTT streaming service requiring the payment of royalties on a per-subscriber basis (subjected to a yearly cap). Or if it were a mobile/STB/TV manufacturer, it would have to pay per-device basis. In the case of AV1, there are zero royalties to be paid.

How AV1 Stacks Up Against VVC

• VP9 and HEVC are about even in terms of quality.
• AV1 is about 35%–38% more efficient than HEVC (H.265)
• VVC, another popular codec, is 3%–5% more efficient than AV1.

VVC (aka H.266) is not royalty free, unlike AV1. VVC, while based on H.265, offers anywhere from 30%-50% efficiency over H.265. DVB has also adopted VVC as its core specifications for video and audio coding in its media delivery toolbox as of 2022.

Video Codec Release Cycles

With each generation of video codecs, there is typically a 5–10x increase in the encoder complexity, while the corresponding increase in decoding complexity tends to be a factor of two. The complexity increase adds to the cost due to enhanced CPU usage and time to encode/decode. When evaluating codec choices, OTT streamers prefer encoding using codecs where video can be decoded by consumers quickly in software or for which there is widespread hardware decoder support. A rule of thumb tends to be when a codec enjoys about 30% hardware device penetration, it becomes commercially viable for an OTT streamer to re-encode the entire catalog to exploit the accompanying benefits.

It is worth noting that when HEVC (H.265) was standardized in 2013 by the ITU & MPEG, it still took HEVC roughly seven years to find a ubiquitous presence in hardware decoding, even in low-end mobile phones in India circa 2020.

AV1 Usage on Social Media

One of the world’s leading social networks first launched AV1 in 2019 to serve obscure video use cases. In 2022, they set goals around relaunching AV1 internally and started with iOS due to the powerful CPUs available on iOS devices.

Being a very narrow set of alternatives, this made it easy to test with only a few generations of phones. The tests began with encoding videos on Reels with the internal findings being that, on average, they used 10% fewer bits compared to VP9. The ultimate goal for this social network is to deploy AV1 across all video applications and possibly be the only codec of choice.

AV1 Usage on OTT Streaming Platforms

One of the world's largest OTT streaming platforms launched AV1 on Android. 30% of the video on demand (VoD) catalog is now encoded in AV1. Their roadmap involves using AV1 offshoot AVIF for images. In February 2020, this OTT platform started streaming AV1 to its Android mobile app The Android launch leveraged the open-source software decoder dav1d built by the VideoLAN, VLC, and FFmpeg communities.

As a best practice, this streaming platform:

1. Encodes source content at the highest frame rate and resolution. So, if a production house delivers a movie at 4k resolution and HFR frame rate, the AV1 encoding is done at 4k HFR.

2. Uses a bit depth of 10 bits even if the AV1 main profile allows for 8-bit encoding. This helps reduce banding artifacts.

3. When streaming, since it's all about balancing video buffering, play delay, and picture quality, this platform now sees an average of 30% reduction in bandwidth per viewing session as well as a 2% reduction in start time.

Striking the Right Balance

New generations of video encoders allow for high-quality video encodes at significantly reduced bit rates. However, this requires increased computing resources that also strain data center capacity and power grids. This causes tension between the dual aims of reducing the bandwidth requirements of video on one hand and curbing energy/power consumption on the other.

Companies tackle this challenge by making cost/benefit decisions on a per-video basis, ensuring that videos predicted to get the most watch time are assigned the most computing resources. Streaming and social networks tend to achieve this by using an older codec, such as H.264, for the less popular videos and newer codecs, like libvpx or SVT-AV1, for the most popular videos.

With the increased adoption of highly efficient software and hardware, could AV1 decoders in mobile devices witness a transition? A change where the entire streaming industry transitions from its current fragmented form, where many generations of codecs exist and compete with one another, to one with royalty-free and energy-efficient solutions. Only time could be the judge of that, but the emerging prominence of royalty-free alternatives is opening doors to endless possibilities.