Use Reddit? Join us on our very own subreddit: https://www.reddit.com/r/Linuxstuff/
You can sign up to get a daily email of our articles, see the Mailing List page!
Support us on Patreon to keep GamingOnLinux alive. This ensures we have no timed articles and no paywalls. Just good, fresh content! Alternatively, you can support us on Paypal and Liberapay!

OpenGL Multi-threading, what it is and what it means

Posted by , | Views: 19,058
Disclaimer: this information is all easily available around, I’m trying to condense it a bit, and focus on a particular topic for discussion. I may gloss over, or simplify information, but the centrals ideas should apply.

There’s been a fair bit of talk recently about attempting to multi-thread using OpenGL so I thought I’d write a bit more about what “multi-threading” an OpenGL game is, what’s normally done, and how it compares to multi-threading in Vulkan.

Some History

For those not aware, OpenGL is, in computing terms, old. It was designed before multiple CPU cores were even available to the general consumer, and long before just about every part of a graphics pipeline was programmable.
The central concept of OpenGL is a state machine. This has done it very well for a long time, but a single OpenGL context (state machine) is based on sequential inputs from the application - the application calls the API, and the OpenGL implementation reacts. State is changed, rendering commands issued, resources loaded, and so on.


State of OpenGL

Being state based, and because any of the API calls has the potential to change state, this makes multi-threaded access to an OpenGL context very difficult - indeed. Let's say one thread is handling "x" and only meant for it, but another thread accesses it and changes "x" to "x+1", the original thread then tries to do something else and it doesn't know it has changed. It’s not permitted in most cases and can result in undefined behaviour like that. There is some exception to this: contexts are allowed to share certain data such as texture information and vertex buffers, but more on that later.
Furthermore about a state based design, is that OpenGL implementations must ensure that the state is always valid. It must ensure that data is correctly bound, in range, and that nothing will break the system. Up to this point, everything is CPU-side still. If everything is okay, the implementation may then generate commands that can be sent to the GPU itself.

Drivers can do some fancy things behind the scenes of course, but the end result, as presented to the application, is the same. Accept an API command, modify and verify state, send hardware commands to the GPU.

Recent versions of OpenGL have been a big help in cutting out a lot of the overhead. Checking state validity can be decreased, greatly reducing the time from API call to GPU command, but it’s still very much a case of having to do that in a single thread.


Threading

So how can developers “multi-thread” OpenGL? It is possible to have multiple contexts in multiple threads, and use them to load texture data, update vertex buffers, possibly compile new shaders, in different threads. The tricky part about this is that sharing this information between OpenGL contexts is dependent on the drivers behaving themselves, in addition to the application not trying (by accident or intent) to do anything strange, so it’s often unstable. It can be quite the adventure getting a game running with this approach, and the runtime improvements are often simply not even worth the effort - it can often run worse if drivers need to synchronise data between contexts often enough. For the curious, things like editors with multiple rendering windows do this, but that’s a different scenario - each window isn’t trying to interfere with every other window while rendering, so multi-threading doesn’t normally come into play.

This leads to the second approach to multi-threading OpenGL: developers don’t! If OpenGL works best by submitting commands sequentially on the thread where a context is active, then that’s simply the best thing to do. Nothing stops a game developer making their own queue of OpenGL API calls they want to perform though, and creating that can be done by multi-threading. To give an example, if a game has a big list of objects, there’s going to be a bit of processing to do when deciding whether to draw each object or not; for each object, the game decides if the object might be visible first, and only try to render if it will actually be seen. The check for each object takes time, but processing each object is independent. So the list can be split into multiple sub-lists, and each sub-list given to a separate thread to run a visibility check on. Each thread will have it’s own rendering list to which objects that should be rendered are added. When done, each rendering list can be iterated over in turn and objects submitted to OpenGL in a single thread. This is a very simple example, but there’s normally quite a fair amount of similar logic in deciding what to render. So it’s not multi-threading OpenGL, but rather multi-threading in deciding how to use OpenGL.


Vulkan

Before I mentioned that OpenGL verifies state information and then generates commands to the GPU.

Firstly, once a developer has finished making everything work, then all that verification is still done, but isn’t actually required. It’s useful during development, but later it’s (hopefully!) a waste of time. So even on a dedicated thread submitting commands to OpenGL, there’s quite the overhead for the final task of actually sending commands to the GPU itself. It would be nice if there was some way to pre-build a list of commands to send to the GPU that were known to be valid.
Secondly, as with the example above of a game splitting object visibility checks into multiple sub-lists, it would also be nice if multiple GPU command lists could be created on separate threads, and then submitted to the GPU in turn. They are separate after all, and don’t require GPU access to actually prepare.
This is essentially what Vulkan allows. There are some requirements: all the state must be known up-front, and prepared for well before it’s time to actually render something. The flip side is that there is much, much less driver overhead, and the API itself can be used multi-threaded. Actual submission of commands to the GPU is still done sequentially, in a single thread, however there’s very little overhead; all error checking has been done, and it’s just sending commands directly to the GPU (feeding the beast).
There are other areas of Vulkan that lend themselves nicely to application level multi-threading, but I won’t cover them here. Suffice to say that Vulkan does not contain a central state machine, and instead tries to keep everything as isolated and contained as possible, meaning things like building a shader don’t block loading a texture, making multi-threaded designs easier to achieve.


Not Always Applicable

On a final note: when porting games, the way a game handles its data is not always compatible with some of the multi-threading ideas mentioned above. It can’t be expected in every game. In addition, it might simply be easier in time and effort (not to mention with testing and stability) to run things in a single thread anyway. Not as efficient, but possibly less error prone and faster to bring a port.
31 Likes, Who?
We do often include affiliate links to earn us some pennies. We are currently affiliated with GOG and Humble Store. See more information here.
The comments on this article are closed.
19 comments
Page: 1/2»
  Go to:

orlfman 10 February 2017 at 9:23 pm UTC
correct me if i'm wrong, but hasn't nvidia supported opengl multi-threading for awhile now in a form of their own extension? or am i thinking of something else?
mirv 10 February 2017 at 9:41 pm UTC
View PC info
  • Supporter
Indeed, nvidia did introduce some extensions that basically tried to morph OpenGL into something Vulkan-like, but there was still some constraints on that and I'm not sure it took off anywhere other than demos. I suspect they did it more for marketing purposes than anything else, and my own guess is that they were feeding their internal Vulkan work into OpenGL extensions as a showcase.

If you're referring to the driver flag which can be activated, that's something different. Nvidia do support some internal driver "modes" where they can use worker threads to even out the workload, but must synchronise everything at certain times.
If the application loads a texture and then loads another texture, there's no reason to wait for the first texture to be loaded before starting on the second, for example. The driver can hide this away - but if the application does anything that needs access to the first texture, the driver must wait until the texture is actually ready. From the application's perspective, it's still single-thread synchronised, as the application doesn't know what the driver has been up to.
While nvidia have put in an amazing amount of work with that, there are cases where performance is worse, or it just crashes outright.
tuubi 10 February 2017 at 10:09 pm UTC
View PC info
  • Supporter
Interesting stuff, mirv. Maybe write about your adventures in Vulkan land at some point?
mirv 10 February 2017 at 10:25 pm UTC
View PC info
  • Supporter
tuubiInteresting stuff, mirv. Maybe write about your adventures in Vulkan land at some point?

It started with "tap tap tap...blank screen...crap!...reboot".
sarmad 10 February 2017 at 10:37 pm UTC
Thanks for the great explanation.
drmoth 11 February 2017 at 8:45 am UTC
Thanks for the great article! Maybe add the GL_THREADED_OPTIMIZATION nvidia stuff to the main article too.
jnrivers 11 February 2017 at 10:13 am UTC
Great information for the layman. Thanks.
Ray54 11 February 2017 at 12:47 pm UTC
View PC info
  • Supporter
Well explained mirv. I knew parts of it, but you have brought the issues together very well. Single large state machine representations were all the rage back in the 1990's, so were used for Unix Workstation Graphics (OpenGL), telephone switches, etc. However, concurrency (e.g. multi-threading) was always a major problem, with deadlock, livelock and unreachable states. Is it now the case that the software tools (e.g. Vulkan validation layer and C++ debuggers) have improved so much that ordinary game writers can be expected to write and successfully debug complex concurrency issues?
Corralx 11 February 2017 at 1:27 pm UTC
Ray54Well explained mirv. I knew parts of it, but you have brought the issues together very well. Single large state machine representations were all the rage back in the 1990's, so were used for Unix Workstation Graphics (OpenGL), telephone switches, etc. However, concurrency (e.g. multi-threading) was always a major problem, with deadlock, livelock and unreachable states. Is it now the case that the software tools (e.g. Vulkan validation layer and C++ debuggers) have improved so much that ordinary game writers can be expected to write and successfully debug complex concurrency issues?

Sadly, it's not the case. Concurrency issues are still the most difficult and subtle to debug.
The issue with all the threading related bugs (deadlocks, livelocks, race conditions or just performance degradation) is that there's no tool made specifically for that. Common debuggers are great to spot bugs in your program's logic for example, but give you little to no help when you stumble on concurrency issues.
There are some tools that can help you with that, either at runtime (like Valgrind's tool, Helgrind) or analysing the source code (not many options for C/C++, tho), but the overhead/cost of the analysis is really huge and typically infeasible for projects as big as modern AAA games.
So in the end, when you're debugging threading issues, you are pretty much on your own. It's you, your brain and your experience (and a lot of headache)
One of the biggest stopper when debugging those issues is the indeterminism of the issues, ie. every run yields a different outcome and it's difficult to relate them and find the root of the problem. Luckily for this we have some working solutions that help greatly.
mirv 11 February 2017 at 1:31 pm UTC
View PC info
  • Supporter
Concurrency is still very much a hassle when it comes to debugging. A lot of tools have become better in that they can look at the state of multiple threads and see what's going on, but it's still going to give a developer a lot of grief if not designed correctly at the start.
Don't forget that devs have been getting used to multiple threads for a long time with networking, AI, audio, game logic, file I/O, etc, so a lot of experience there can now map to graphics as well. Not to say it couldn't before, it's just less of a hassle now.
  Go to:
While you're here, please consider supporting GamingOnLinux on Patreon or Liberapay. We have no adverts, no paywalls, no timed exclusive articles. Just good, fresh content. Without your continued support, we simply could not continue!

We also accept Paypal donations and subscriptions! If you already are, thank you!
Livestreams & Videos
Community Livestreams
  • Story Time: „The Secret of Monkey Island“
  • Date:
See more!
Popular this week
View by Category
Contact
Latest Comments
Latest Forum Posts