Video Encoding Tested: AMD GPUs Still Lag Behind Nvidia, Intel

The best graphics cards aren’t just for playing games. Artificial intelligence training and inference, professional applications, video encoding and decoding can all benefit from having a better GPU. Yes, games still get the most attention, but we like to look at the other aspects as well. Here we’re going to focus specifically on the video encoding performance and quality that you can expect from various generations of GPU.

Generally speaking, the video encoding/decoding blocks for each generation of GPU will all perform the same, with minor variance depending on clock speeds for the video block. We’ve checked the RTX 3090 Ti and RTX 3050 as an example — the fastest and slowest GPUs from Nvidia’s Ampere RTX 30-series generation — and found effectively no difference. Thankfully, that leaves us with fewer GPUs to look at than would otherwise be required.

We’ll test Nvidia’s RTX 4090, RTX 3090, and GTX 1650 from team green, which covers the Ada Lovelace, Turing/Ampere (functionally identical), and Pascal-era video encoders. For Intel, we’re looking at desktop GPUs, with the Arc A770 as well as the integrated UHD 770. AMD ends up with the widest spread, at least in terms of speeds, so we ended up testing the RX 7900 XTX, RX 6900 XT, RX 5700 XT, RX Vega 56, and RX 590. We also wanted to check how the GPU encoders fare against CPU-based software encoding, and for this we used the Core i9-12900K and Core i9-13900K.

Video Encoding Test Setup

Most of our testing was done using the same hardware we use for our latest graphics card reviews, but we also ran the CPU test on the 12900K PC that powers our 2022 GPU benchmarks hierarchy. As a more strenuous CPU encoding test, we also ran the 13900K with a higher-quality encoding preset, but more on that in a moment.

TOM’S HARDWARE TEST EQUIPMENT

For our test software, we’ve found ffmpeg nightly (opens in new tab) to be the best current option. It supports all of the latest AMD, Intel, and Nvidia video encoders, can be relatively easily configured, and it also provides the VMAF (opens in new tab) (Video Multi-Method Assessment Fusion) functionality that we’re using to compare video encoding quality. We did, however, have to use the last official release, 5.1.2, for our Nvidia Pascal tests (the nightly build failed on HEVC encoding).

We’re doing single-pass encoding for all of these tests, as we’re using the hardware provided by the various GPUs and it’s not always capable of handling more complex encoding instructions. GPU video encoding is generally used for things like livestreaming of gameplay, while if you want best quality you’d generally need to opt for CPU-based encoding with a high CRF (Constant Rate Factor) of 17 or 18, though that of course results in much larger files and higher average bitrates. There are still plenty of options that are worth discussing, however.

AMD, Intel, and Nvidia all have different “presets” for quality, but what exactly these presets are or what they do isn’t always clear. Nvidia’s NVENC in ffmpeg uses “p4” as its default. And switching to “p7” (maximum quality) did little for the VMAF scores, while dropping encoding performance by anywhere from 30 to 50 percent. AMD opts for a “-quality” setting of “speed” for its encoder, but we also tested with “balanced” — and like Nvidia, the maximum setting of “quality” reduced performance a lot but only improved VMAF scores by 1~2 percent. Lastly, Intel seems to use a preset of “medium” and we found that to be a good choice — “veryslow” took almost twice as long to encode with little improvement in quality, while “veryfast” was moderately faster but degraded quality quite a bit.

Ultimately, we opted for two sets of testing. First, we have the default encoder settings for each GPU, where the only thing we specified was the target bitrate. Even then, there are slight differences in encoded file sizes (about a +/-5% spread). Second, after consulting with the ffmpeg subreddit (opens in new tab), we attempted to tune the GPUs for slightly more consistent encoding settings, specifying a GOP size equal to two seconds (“-g 120” for our 60 fps videos). AMD was the biggest benefactor of our tuning, trading speed for roughly 5-10 percent higher VMAF scores. But as you’ll see, AMD still trailed the other GPUs.

There are many other potential tuning parameters, some of which can change things quite a bit, others which seem to accomplish very little. We’re not targeting archival quality, so we’ve opted for faster presets that use the GPUs, but we may revisit things in the future. Sound off in our comments if you have alternative recommendations for the best settings to use on the various GPUs, with an explanation of what the settings do. It’s also unclear how the ffmpeg settings and quality compare to other potential encoding schemes, but that’s beyond the scope of this testing.

Here are the settings we used, both for the default encoding as well as for the “tuned” encoding.

AMD:
Default: ffmpeg -i [source] -c:v [h264/hevc/av1]_amf -b:v [bitrate] -y [output]
Tuned: ffmpeg -i [source] -c:v [h264/hevc/av1]_amf -b:v [bitrate] -g 120 -quality balanced -y [output]

Intel:
Default: ffmpeg -i [source] -c:v [h264/hevc/av1]_qsv -b:v [bitrate] -y [output]
Tuned: ffmpeg -i [source] -c:v [h264/hevc/av1]_amf -b:v [bitrate] -g 120 -preset medium -y [output]

Nvidia:
Default: ffmpeg -i [source] -c:v [h264/hevc/av1]_nvenc -b:v [bitrate] -y [output]
Tuned: ffmpeg -i [source] -c:v [h264/hevc/av1]_nvenc -b:v [bitrate] -g 120 -no-scenecut 1 -y [output]

Most of our attempts at “tuned” settings didn’t actually improve quality or encoding speed, and some settings seemed to cause ffmpeg (or our test PC) to just break down completely. The main thing about the above settings is that it keeps the key frame interval constant, and potentially provides for slightly higher image quality.

Again, if you have some better settings you’d recommend, post them in the comments; we’re happy to give them a shot. The main thing is that we want bitrates to stay the same, and we want reasonable encoding speeds of at least real-time (meaning, 60 fps or more) for the latest generation GPUs. That said, for now our “tuned” settings ended up being so close to the default settings, with the exception of the AMD GPUs, that we’re just going to show those charts.

With the preamble out of the way, here are the results. We’ve got four test videos, taken from captured gameplay of Borderlands 3, Far Cry 6, and Watch Dogs Legion. We ran tests at 1080p and 4K for Borderlands 3 and at 4K with the other two games. We also have three codecs: H.264/AVC, H.265/HEVC, and AV1. We’ll have two charts for each setting and codec, comparing quality using VMAF and showing the encoding performance.

For reference, VMAF scores (opens in new tab) follow a scale from 0 to 100, with 60 rated as “fair,” 80 as “good,” and 100 as “excellent.” (None of our results fall below 60, but 40 is “poor” and 20 is “bad.”) In general, scores of 90 or above are desirable, with 95 or higher being mostly indistinguishable from the original source material.

Video Encoding Quality and Performance at 1080p: Borderlands 3

Starting out with a video of Borderlands 3 at 1080p, there are a few interesting trends that we’ll see elsewhere. We’re discussing the “tuned” results, as they’re slightly more apples to apples among the GPUs. Let’s start with AVC/H.264, which remains the most popular codec. Even though it’s been around for about a decade and has since been surpassed in quality, its effectively royalty-free status made for widespread adoption.

AMD’s GPUs rank at the bottom of our charts on this test, with the Polaris, Vega, and RDNA 1 generations of hardware all performing about the same in terms of quality. RDNA 2 improved the encoding hardware, and scores with lower bitrates improved by a few points. RDNA 3 appears to use the identical logic for H.264, with tied scores on the 6900 XT and 7900 XTX that are a bit higher than the other AMD cards at 3Mbps, but the 6Mbps and 8Mpbs results are effectively unchanged.

Nvidia’s Ampere and Ada H.264 implementations come out as the top solutions, in terms of quality, with Intel’s Arc just a hair behind — you’d be hard pressed to tell the difference in practice. Our test settings also put the CPU roughly at the level of Nvidia’s older Pascal-era encoder, at least for these 1080p tests.

As far as performance goes, which is something of a secondary concern provided you can break 60 fps for livestreaming, the RTX 4090 was fastest at nearly 500 fps, about 15% ahead of the RTX 3090 and RX 7900 XTX. Intel Arc comes in third place, then Nvidia Pascal (or at least the GTX 1650’s version of Pascal — a GTX 1080 Ti had the same quality but was a bit slower). Interestingly, the Core i9-13900K can encode at 1080p faster than the old RX Vega 56, which in turn ends up being another big step up compared to the RX 590.

Switching to HEVC provides a solid bump in image quality, and it’s been around long enough now that virtually any modern PC should have good GPU encoding support. The only thing really hindering its adoption in more applications is royalties, and a lot of software opted to stay with H.264 to avoid that.

As far as the ranking go, AMD’s various GPUs all fall below the rest of the options again in this test, but they’re quite a bit closer than in H.264. The RDNA 3 video engine is slightly better than the RDNA 1/2 version, which in turn slightly outperforms the older Vega and Polaris cards.

For our tested settings, the CPU and Nvidia Pascal results are basically tied, coming in 3~5% above the 7900 XTX. Intel’s Arc A770 just barely edges out the RTX 4090/3090 in pure numbers, with a 0.5–1.9% lead, depending on the bitrate. Also note that with scores of 95–96 at 8Mbps, the Arc, Ada, and Ampere GPUs all effectively achieve encodings that would be virtually indistinguishable from the source material.

Performance is a bit interesting, especially looking at the generations of hardware. The older GPU models were generally faster at HEVC than H.264 encoding, sometimes by a fairly sizable margin — look at the RX 590 and GTX 1650. But the RTX 3090 and 4090 are faster at H.264, and the Ampere generation at least (RTX 30) seems to have deprioritized HEVC speed compared to Pascal.

The worst VMAF results are now above 70, with the Ada/Ampere and Arc cards all staying above 80, even with a 3Mbps bitrate. But let’s see if AV1 can do any better.

In a word: Nope. AV1 and HEVC basically deliver equivalent quality overall, or nearly so. There are some differences, but a lot of that comes down to the various implementations of the codecs.

AMD’s RDNA 3 for example does better with AV1 than with HEVC by about two points. Nvidia’s Ada cards also do better with AV1 than HEVC by 1–2 points, while Intel’s Arc GPUs go the other way and score 1–2 points lower in AV1 versus HEVC.

Given the cost of RX 7900-series and RTX 40-series cards, the only option right now for a lot of people will be the Arc GPUs. The A380 incidentally delivered identical quality results to the A770 (down to six decimal places), and it was actually slightly faster — probably because the A380 card had a factory overclock. Alternatively, you could opt for CPU-based AV1 encoding, which matches or exceeds the GPU encoders and can still be done at over 200 fps on a 13900K.

That’s assuming you even want to bother with AV1, which at this stage still doesn’t have as widespread of support as HEVC. It’s also interesting how demanding HEVC encoding can be on CPU, where AV1 by comparison looks pretty “easy.” In some cases (at 4K), the CPU encoding of AV1 was actually faster than H.264 encoding, though that will obviously depend on the settings used for each.

Video Encoding Quality and Performance at 4K: Borderlands 3

Moving up to 4K encoding, the bitrate demands with H.264 become significantly higher than at 1080p. Intel’s Arc nearly makes it to an 80 score with just 8Mbps, but Nvidia’s Ada/Ampere/Turing GPUs (which had effectively the same VMAF results) land at around 73, while the Pascal generation encoder along with all of AMD’s GPUs falls below 70.

AMD has made improvements, though, with RDNA 3 — the first significant changes we’ve seen so far. It nearly matches our CPU-based “medium quality” preset result at higher bitrates and does beat it at 8Mbps. If you use older AMD GPUs, though, you’ll want 16Mbps bitrates to get up to 80 or higher, which is basically the minimum we’d want for regular use.

Encoding performance at 4K is a lot more demanding, as expected. Most of the solutions are roughly 1/3 as fast as at 1080p, and in some cases it can even be closer to 1/4 the performance. AMD’s older Polaris GPUs can’t manage even 30 fps based on our testing, while the Vega cards fall below 60 fps. The 13900K, however, does plug along at around 80 fps, though if you’re playing a game you may not want to devote that much CPU horsepower to the task.

HEVC and AV1 encoding were designed to scale up to 4K and beyond, and our charts prove this point nicely. All of the GPUs break 80 on the VMAF scale with 12Mbps, and at 16Mbps you can get over 90. AV1, perhaps because it’s newer and thus doesn’t have the backlog of older, inferior implementations, starts at 80 on the AMD 7900 XTX with 8Mbps and scales to above 90 as well.

But if you’re after image quality, AMD’s solutions still fall behind. They can be reasonably fast at encoding, but even Nvidia’s Pascal era hardware generally delivers superior results compared to anything AMD has, at least with our ffmpeg testing. Actually, the 7900 XTX does basically tie the GTX 1650 on quality, if we compare AMD’s AV1 against the Nvidia’s HEVC, but that’s still a bit ludicrous for hardware from seven years ago to match a modern solution.

AMD has informed us that it’s working with ffmpeg to get some quality improvements into the code, and we’ll have to see how that goes. We don’t know if it will improve quality a lot, bringing AMD up to par with Nvidia, or if it will only be one or two points. Still, every little bit of improvement is good.

HEVC and AV1 aren’t really any better than H.264 from the performance perspective. AMD’s 7900 XTX does hit 200 fps with AV1, the fastest of the bunch by a fairly wide margin, which again leaves us some hope for quality improvements. Otherwise, for 4K HEVC, you’ll need a Pascal or later Nvidia card, or an RDNA or later AMD card, or of course Intel’s Arc.

Video Encoding Quality and Performance at 4K: Far Cry 6

The results with our other two videos at 4K aren’t substantially different from what we saw with Borderlands 3 at 4K. Far Cry 6 ended up with higher VMAF scores across all three codecs, with some minor variations in performance, but that’s about all we have to add to the story.

Actually, we didn’t include the results in our charts, but we did test Intel’s UHD Graphics 770 — the integrated graphics on 12th Gen Alder Lake and 13th Gen Raptor Lake CPUs. The quality is slightly lower than Arc, especially at the lowest bitrates, falling between Nvidia’s Pascal and Turing/Ampere generation in terms of VMAF scores.

Performance on the other hand was a lot lower than we expected, far less than half of what the Arc can do. Granted, Arc has two video engines, but the UHD 770 only managed 46 fps at 4K with H.264, and that dropped to around 25 fps with HEVC encoding. For HEVC encoding, the 12900K CPU encoding was faster, and with AV1 it was about three times as fast as the GPU HEVC result while still delivering better fidelity.

Video Encoding Quality and Performance at 4K: Watch Dogs Legion

Watch Dogs Legion repeats the story, but again with increased VMAF scores. That’s probably due to there being less rapid movement in the camera, so the frame to frame changes are smaller than that allows for better compression and thus higher objective quality within the same amount of bitrate.

Performance ends up nearly the same as Far Cry 6 in most cases, and there’s some variability between runs that we’re not really taking into account — we’d estimate about a 3% margin of error for the FPS, since we only did each encode once per GPU.

If you’d like to see the original video files, without any compression, I’ve created this personally hosted BitTorrent magnet link:
magnet:?xt=urn:btih:8a4710b78cba5c8b3c26bb95af5e306c9a4d0ebb&dn=Tom%27s%20Hardware%20Video%20Encoding%20Tests

If you’d like to do your own testing, the above BitTorrent magnet link will be kept active for the coming months — join and share if you can, as this is seeding from my own home network and I’m limited to 20Mbps upstream. If you previously downloaded my Arc A380 (opens in new tab) testing, note that nearly all of the files are completely new as I recorded new videos at a higher 100Mbps bitrate for these tests. (The Borderlands 3
source videos are the same, however.)

The files are all spread across folders, so you can download just the stuff you want if you’re not interested in the entire collection. And if you do want to enjoy all 350+ videos for whatever reason, it’s a 16.4GiB download. Cheers!

For those that don’t want to download all of the files, I’ll provide a limited set of screen captures from the Borderlands 3 1080p and 4K videos for reference below. I’m only going to do screens from recent GPUs, as otherwise this gets to be very time consuming.

Video Encoding Image Quality Screenshots

The above gallery shows screengrabs for just one frame from the AMD RX 7900 XTX, Intel Arc A770, Nvidia RTX 4090, and CPU encodes on the Core i9-13900K. Of course a single frame isn’t the point, and there can be differences in where key frames are located, but that’s why we used the “-g 120” option so that, in theory, all of the encoders are putting key frames at the same spot. Anyway, if you want to browse through them, we recommend viewing the full images in separate tabs.

We’ve only elected to do screen grabs for the lowest bitrates, and that alone took plenty of time. You can at least get some sense of what the encoded videos look like, but the VMAF scores arguably provided a better guide as to what the overall experience of watching a stream using the specified encoder would look like.

Video Encoding Showdown: Closing Thoughts

After running and rerunning the various encodes multiple times, we’ve done our best to try and level the playing field, but it’s still quite bumpy in places. Maybe there’s some options that can improve quality without sacrificing speed that we’re not familiar with, but if you’re not planning on putting a ton of time into figuring such things out, these results should provide a good baseline of what to expect.

For all the hype about AV1 encoding, in practice it really doesn’t look or feel different from HEVC. The only real advantage is that AV1 is supposed to be royalty free, but if you’re archiving your own movies you can certainly stick with using HEVC and you won’t miss out on anything. Maybe AV1 will take over going forward, just like H.264 became the de facto video standard for the past decade or more. Certainly it has some big names behind it, and now all three of the major PC GPU manufacturers have accelerated encoding support.

From an overall quality and performance perspective, Nvidia’s latest Ada Lovelace NVENC hardware comes out as the winner, but right now it’s only available with GPUs that start at $799 — for the RTX 4070 Ti. It should eventually arrive in 4060 and 4050 variants, and those are already shipping in laptops, though there’s an important caveat: the 40-series cards with 12GB or more VRAM have dual NVENC blocks, while the other models will only have a single encoder block, which could mean about half the performance compared to our tests here.

Right behind Nvidia in terms of quality and performance, at least as far as video encoding is concerned, Intel’s Arc GPUs also provide great quality and performance for streaming purposes. You can also use them for archiving, sure, but that’s likely not the key draw. Nvidia definitely supports more options for tuning, however, and seems to be getting more software support as well.

AMD’s GPUs meanwhile continue to lag behind their competition. The RDNA 3-based RX 7900 cards deliver the highest-quality encodes we’ve seen from an AMD GPU to date, but that’s not saying a lot. In fact, at least with the current version of ffmpeg, quality and performance are about on par with what you could get from a GTX 10-series GPU back in 2016 — except without AV1 support, naturally, since that wasn’t a thing back then.

Also, few people are going to buy a graphics card purely for its video encoding prowess, so check our GPU benchmarks and our Stable Diffusion tests to see how the various cards stack up in other areas. Next up, we need to run updated numbers for Stable Diffusion, but that’s a story for another day.