Fable Legends: AMD and Nvidia go head-to-head in latest DirectX 12 benchmark

Fable Legends: AMD and Nvidia go head-to-head in latest DirectX 12 benchmark
As DirectX 12 and Windows 10 roll out across the PC ecosystem, the number of titles that support Microsoft’s new API is steadily growing.

As DirectX 12 and Windows 10 roll out across the PC ecosystem, the number of titles that support Microsoft’s new API is steadily growing. Last month, we previewed Ashes of the Singularity and its DirectX 12 performance; today we’re examining Microsoft’s Fable Legends. This upcoming title is expected to debut on both Windows PCs and the Xbox One and is built with Unreal Engine 4.

Like Ashes, Fable Legends is still very much a work-in-progress. Unlike Ashes of the Singularity, which can currently be bought and played, Microsoft chose to distribute a standalone benchmark for its first DirectX 12 title. The test has little in the way of configurable options and performs a series of flybys through complex environments. Each flyby highlights a different aspect of the game, including its day/night cycle, foliage and building rendering, and one impressively ugly troll. If Ashes of the Singularity gave us a peek at how DX12 would handle several dozen units and intense particle effects, Fable Legends looks more like a conventional first-person RPG or FPS.

There are other facets to Fable Legends that make this a particularly interesting match-up, even if it’s still very early in the DX12 development cycle. Unlike Ashes of the Singularity, which is distributed through Oxide, this is a test distributed directly by Microsoft. It uses the Unreal 4 engine — and Nvidia and Epic, Unreal’s developer, have a long history of close collaboration. Last year, Nvidia announced GameWorks support for UE4, and the UE3 engine was an early supporter of PhysX on both Ageia PPUs and later, Nvidia GeForce cards.

Test setup

We tested the GTX 980 Ti and Radeon Fury X in Windows 10 using the latest version of the operating system. Our testbed was an Asus X99-Deluxe motherboard with a Core i7-5960X, 16GB of DDR4-2667 memory. We tested an AMD-provided beta driver for the Fury X and with Nvidia’s latest WHQL-approved driver, 355.98. NVidia hasn’t released a beta Windows 10 driver since last April, and the company didn’t contact us to offer a specific driver for the Fable Legends debut.

The benchmark itself was provided by Microsoft and can run in a limited number of modes. Microsoft provided three presets — a 720p “Low” setting, a 1080p “Ultra” and a 4K “Ultra” benchmark. There are no user-configurable options besides enabling or disabling V-Sync (we tested with V-Sync disabled) and the ability to specify low settings or ultra settings. There is no DX11 version of the benchmark. We ran all three variants on both the Fury X and GTX 980 Ti.

Test Results (Original and Amended):

Once other sites began posting their own test results, it became obvious that our own 980 Ti and Fury X benchmarks were both running more slowly than they should have. It’s normal to see some variation between review sites, but gaps of 15-20% in a benchmark with no configurable options? That meant a different problem. Initial retests confirmed the figures shown below, even after wiping and reinstalling drivers.

The next thing to check was power management — and this is where we found our smoking gun. We tested Windows 10 in its “Balanced” power configuration, which is our standard method of testing all hardware. While we sometimes increase to “High Performance” in corner cases or to measure its impact on power consumption, Windows can generally be counted on to handle power settings, and there’s normally no performance penalty for using this mode.

Imagine our surprise, then, to see the following when we fired up the Fable benchmark:

Click to enlarge

The benchmark is actively running in the screenshot above, with power conservation mode and clock speed visible at the same time. And while CPU clock speed isn’t the determining factor in most titles, clocking down to 1.17GHz is guaranteed to have an impact on overall frame rates. Switching to “High Performance” pegged the CPU clock between 3.2 and 3.3GHz — exactly where we’d expect it to be. It’s not clear what caused this problem — it’s either a BIOS issue with the Asus X99-Deluxe or an odd driver bug in Windows 10, but we’ve retested both GPUs in High Performance mode.

These new results are significantly different from our previous tests. 4K performance is unchanged, and the two GPUs still tie, but 1080p performance improves by roughly 8% on the GTX 980 Ti and 6% on the Fury X. Aftermarket GTX 980 Ti results show higher-clocked manufacturing variants of that card as outperforming the R9 Fury X, and those are perfectly valid data points — if you want to pay the relatively modest price premium for a high-end card with more clock headroom, you can expect a commensurate payoff in this test. Meanwhile, the R9 Fury X no longer wins 720p as it did before. Both cards are faster here, but the GTX gained much more from the clock speed boost, leaping up 27%, compared to just 2% for AMD. While this conforms to our general test trends in DX11, in which AMD performs more capably at higher resolutions, it’s still unusual to see only one GPU respond so strongly to such ludicrously low clock speeds.

These new runs, like the initials, were performed multiple times. We ran the benchmark 4x on each card, at each quality preset, but threw out the first run in each case. We also threw out runs that appeared unusually far from the average.

Why include AMD results?

In our initial coverage for this article, we included a set of AMD-provided test results. This was mostly done for practical reasons — I don’t actually have an R9 390X, 390, or R9 380, and therefore couldn’t compare performance in the midrange graphics stack. Our decision to include this information “shocked” Nvidia’s PR team, which pointed out that no other reviewer had found the R9 390 winning past the GTX 980.

Implications of impropriety deserve to be taken seriously, as do charges that test results have misrepresented performance. So what’s the situation here? While we may have shown you chart data before, AMD’s reviewer guide contains the raw data values themselves. According to AMD, the GTX 980 scored 65.36 FPS in the 1080p Ultra benchmark using Nvidia’s 355.98 driver (the same we driver we tested). Our own results actually point to the GTX 980 being slightly slower — when we put the card through its paces for this section of our coverage, it landed at 63.51 FPS. Still, that’s just a 3% difference.

It’s absolutely true that Tech Report’s excellent coverageshows the GTX 980 beating past the R9 390  (TR was the only website to test an R9 390 in the first place). But that doesn’t mean AMD’s data is non-representative. Tech Report notes that it useda Gigabyte GTX 980, with a base clock of 1228MHz and a boost clock of 1329MHz. That’s 9% faster than the clocks on my own reference GTX 980 (1127MHz and 1216MHz respectively).

Multiply our 63.51 FPS by 1.09x, and you end up with 69 FPS — exactly what Tech Report reported for the GTX 980. And if you have an NV GTX 980 clocked at this speed, yes, you will outperform a stock-clocked R9 390. That, however, doesn’t mean that AMD lied in its test results. A quick trip to Neweggreveals that GTX 980s ship in a variety of clocks, from a low of 1126MHz to a high of 1304MHz. That, in turn, means that the highest-end GTX 980 is as much as 15% faster than the stock model. Buyers who tend to buy on price are much more likely to end up with cards at the base frequency, the cheapest EVGA GTX 980 is $459, compared to $484 for the 1266MHz version.

There’s no evidence that AMD lied or misconstrued the GTX 980’s performance. Neither did Tech Report. Frankly, we prefer testing retail hardware when such equipment is available, but since GPU vendors tend to charge a premium for higher-clocked GPUs, it’s difficult to select any single card and declare it representative.

Amended Conclusion:

Nvidia’s overall performance in Fable Legends remains excellent, though whether Team Red or Green wins is going to depend on which type of card, specifically, you’ve chosen to purchase. The additional headroom left in many of Nvidia’s current designs is a feature, not a bug, and while it makes it more difficult to point at any single point and declare it representative of GTX 980 Ti or 980 performance, we suspect most enthusiasts appreciate the additional headroom.

The power issues that forced a near-total rewrite of this story, however, also point to the immaturity of the DirectX 12 ecosystem. Whether you favor AMD or Nvidia, it’s early days for both benchmarks and GPUs, and we wouldn’t recommend making drastic decisions around expected future DirectX 12 capability. There are still unanswered questions and unclear situations surrounding certain DirectX 12 features, like asynchronous computing on Nvidia cards, but the overall performance story from Team Red vs. Team Green is positive. The fact that a stock R9 390, at $329, outperforms a stock GTX 980 with an MSRP of $460, however, is a very nice feather in AMD’s cap.

As with Ashes of the Singularity, the usual caveats apply. These are pre-launch titles and early drivers on a still-young operating system. So far, however, the DX12 results we’ve seen have been very positive for AMD — lending credence to the company’s longstanding argument that GCN would fare well under the new API.

After results from other sites began to go live, it became apparent that our performance figures for Fury X and the 980 Ti were oddly low. We’ve inserted a section above (and new benchmarks) to explain what happened and examine the new data. Relative performance between lower-end AMD and Nvidia cards is also now addressed.