Note: I wrote a guide for the benchmarking, available here.
I decided to run some of my own tests in Dota 2 to see what kind of difference I am actually seeing, the results are surprising.
With many thanks to the person behind the recent video I covered, I was able to get a fair bit of help from him on twitter. I used the same demo file he used to run my tests. This demo file is heavier than the one used by Phoronix and others. Every demo file will give you a different result remember.
All tests run with "desktop friendly fullscreen" mode, not the borderless window mode which is a different option again.
Each test was run four times with the first removed to allow for all systems cache to be built for more accurate results. Tests done using the Dota 2 built-in benchmark tool using "timedemo benchmarkfilename", I simply ran Dota 2 with these launch options (console + API):
or
The benchmark does change graphical settings when you switch resolutions, so don't "do a Liam" and not notice. Make 100% sure the settings match after changing resolutions.
Note: As this is a benchmark, please remember this is probably different to what you will achieve. Unless you have the exact same setup your results will vary.
System
Linux Mint 17.3 64bit (Moving back over to Ubuntu soon, but that's another story)
Linux driver: 364.19 (latest available for Mint)
Intel i7 5960x 3.0ghz, 8 cores
Nvidia 980ti
Results
So in all the Linux tests, OpenGL outperformed Vulkan in Dota 2. Not by much at all in 1080p, but 4K is a different story.
I would love to run some Windows 10 tests in comparison, but Windows wanted to update which slowed everything down, Steam kept going to a 0b/s download for Dota 2 and it's just...so slow. I will hopefully do a Windows 10 comparison when Windows actually plays nicely. I was on it for half an hour and it only got to 40% done on updates, so it would probably take all night to do that + the tests.
Finally, just to remind you that Dota 2 isn't the best benchmark for Vulkan, Valve said it themselves that Dota 2 isn't doing all that much. Vulkan is early, Vulkan in Dota 2 is also very early. Article taken from GamingOnLinux.com.
I decided to run some of my own tests in Dota 2 to see what kind of difference I am actually seeing, the results are surprising.
With many thanks to the person behind the recent video I covered, I was able to get a fair bit of help from him on twitter. I used the same demo file he used to run my tests. This demo file is heavier than the one used by Phoronix and others. Every demo file will give you a different result remember.
All tests run with "desktop friendly fullscreen" mode, not the borderless window mode which is a different option again.
Each test was run four times with the first removed to allow for all systems cache to be built for more accurate results. Tests done using the Dota 2 built-in benchmark tool using "timedemo benchmarkfilename", I simply ran Dota 2 with these launch options (console + API):
+con_enable 1 -vulkanor
+con_enable 1 -glThe benchmark does change graphical settings when you switch resolutions, so don't "do a Liam" and not notice. Make 100% sure the settings match after changing resolutions.
Note: As this is a benchmark, please remember this is probably different to what you will achieve. Unless you have the exact same setup your results will vary.
System
Linux Mint 17.3 64bit (Moving back over to Ubuntu soon, but that's another story)
Linux driver: 364.19 (latest available for Mint)
Intel i7 5960x 3.0ghz, 8 cores
Nvidia 980ti
Results
So in all the Linux tests, OpenGL outperformed Vulkan in Dota 2. Not by much at all in 1080p, but 4K is a different story.
I would love to run some Windows 10 tests in comparison, but Windows wanted to update which slowed everything down, Steam kept going to a 0b/s download for Dota 2 and it's just...so slow. I will hopefully do a Windows 10 comparison when Windows actually plays nicely. I was on it for half an hour and it only got to 40% done on updates, so it would probably take all night to do that + the tests.
Finally, just to remind you that Dota 2 isn't the best benchmark for Vulkan, Valve said it themselves that Dota 2 isn't doing all that much. Vulkan is early, Vulkan in Dota 2 is also very early. Article taken from GamingOnLinux.com.
0 Likes
Some you may have missed, popular articles from the last month:
The comments on this article are closed.
It's okay, different graphical patterns use different features on the GPU. The benchmarks we do can widely differ based on the demos we use, the state of implementation of drivers for a certain GPU architecture etc, driver versions we're using etc.
That all is perfectly okay. All we know now: It's not where we'd like to see Vulkan yet. There is room for improvement in drivers and engines. That's all we can say.
Last edited by STiAT on 15 June 2016 at 10:57 pm UTC
That all is perfectly okay. All we know now: It's not where we'd like to see Vulkan yet. There is room for improvement in drivers and engines. That's all we can say.
Last edited by STiAT on 15 June 2016 at 10:57 pm UTC
0 Likes
Quoting: liamdaweI wouldn't just use the ones the Valve dev pointed out, there aren't any that are the "best".No, not the "best" indeed, they are just meant to illustrate scenarios where Vulkan is efficient.
Ideally I would use more demos with specific scenarios or a single one but much longer.
For the moment though is too early (I think) to spend so much time for a benchmark that tomorrow might be obsolete.
0 Likes
On a GTX 980 over here, I got capped at 120, with the occasional drop to 119.
0 Likes
Quoting: aFoxNamedMorrisOn a GTX 980 over here, I got capped at 120, with the occasional drop to 119.Are you using the "+fps_max 0" argument?
0 Likes
I have no solution for any of this, and I don't even play Dota. But my limited experience with Vulkan has shown me that it runs much more smoothly, without hitches like I tend to get with OpenGL. In this case, I'm referring to the Talos Principle. I'm guessing it takes stress off the CPU and relies more on the GPU. I have a GTX 970 but a pretty old i5 CPU. The less stress on my CPU, the better (at least until I can upgrade).
Maybe, even though you're seeing slightly lower FPS, it might still be smoother performance and better use of the GPU. I'm totally speculating, but just wondering.
Maybe, even though you're seeing slightly lower FPS, it might still be smoother performance and better use of the GPU. I'm totally speculating, but just wondering.
1 Likes, Who?
cRaZy-bisCuiT Jun 16, 2016
Quoting: wojtek88Hmmm hd to 4k comparison got you weird results. 4 times more pixels to render and only like 2-5 fps less?Due to CPU limitation! In 1080p the CPU may allready limit the scenario while the GPU gets bored. Therfor (with a powerful GPU) there's not much of a difference. With better vulkan optimisation of the game and driver the performance cap may be raised a bit. Therfor the gap might get bigger.
Quoting: liamdaweA GTX 1080 is a more powerful card than my Nvidia 980ti.True indeed, still the 1080 driver is not well-optimised since it's very new. The difference will grow. Except for the fact the game is CPU limited anyways.
1 Likes, Who?
Quoting: melkemindMaybe, even though you're seeing slightly lower FPS, it might still be smoother performance and better use of the GPU. I'm totally speculating, but just wondering.
This is actually true. FPS tends to fluctuate a lot less on Vulkan. Also input lag is greatly reduced, which really makes a huge difference in competitive games and VR.
Also, I don't think it's because of less stress on CPU (unless you have a really bad cooling) but rather how Vulkan is multi threaded.
Last edited by Blodoffer on 16 June 2016 at 4:01 am UTC
0 Likes
There are new NVIDIA drivers out with Added support for the following GPUs:
GeForce GTX 1080
GeForce GTX 1070
and many bug fixes...
http://www.nvidia.co.uk/download/driverResults.aspx/104314/en-uk
Also can someone please post in details step by step how I can do this Dota 2 benchmarks?
I would like to do some benchmarks with my GTX980
Is weird that I cant find any info on this , on every benchmark site noone explains how they did their benchmarks step by step.
GeForce GTX 1080
GeForce GTX 1070
and many bug fixes...
http://www.nvidia.co.uk/download/driverResults.aspx/104314/en-uk
Also can someone please post in details step by step how I can do this Dota 2 benchmarks?
I would like to do some benchmarks with my GTX980
Is weird that I cant find any info on this , on every benchmark site noone explains how they did their benchmarks step by step.
0 Likes
Quoting: wolfyrionAlso can someone please post in details step by step how I can do this Dota 2 benchmarks?I am writing out a guide at the moment, I was also frustrated by the lack of a good guide to it all.
I would like to do some benchmarks with my GTX980
Is weird that I cant find any info on this , on every benchmark site noone explains how they did their benchmarks step by step.
Last edited by Liam Dawe on 16 June 2016 at 6:48 am UTC
1 Likes, Who?
Listening to a few Dota developer videos they state that Dota is not really draw call limited. It doesn't make enough calls to throttle DirectX or OpenGL, so you won't see much of a leap in performance with Vulkan.
From the benchmarks between OpenGL and Vulkan on Linux it would suggest the problem is driver (or render implementation) bound. As for the performance variance between Windows and Linux, a mixture of driver, compositor, X11, and a whole heap of other things. I suspect some developers will look into what is causing this variance.
Bare in mind this is from a game in-which the DirectX 9 implementation is still the fastest, even compared to DX10/11 renderers.
From the benchmarks between OpenGL and Vulkan on Linux it would suggest the problem is driver (or render implementation) bound. As for the performance variance between Windows and Linux, a mixture of driver, compositor, X11, and a whole heap of other things. I suspect some developers will look into what is causing this variance.
Bare in mind this is from a game in-which the DirectX 9 implementation is still the fastest, even compared to DX10/11 renderers.
0 Likes
Patreon
PayPal
See more from me