Confused on Steam Play and Proton? Be sure to check out our guide.
We do often include affiliate links to earn us some pennies. See more here.

Recently it was noticed that users on more bleeding-edge Linux distributions that updated saw Easy Anti-Cheat no longer working on Linux, the culprit was glibc and now a Valve developer has spoken out about it.

Writing in a small thread on Twitter, Valve developer Pierre-Loup Griffais said:

Unfortunate that upstream glibc discussion on DT_HASH isn't coming out strongly in favor of prioritizing compatibility with pre-existing applications. Every such instance contributes to damaging the idea of desktop Linux as a viable target for third-party developers.

Our thoughts on the topic from this prior compatibility issue in BlueZ apply more than ever: https://github.com/bluez/bluez/commit/35a2c50437cca4d26ac6537ce3a964bb509c9b62#commitcomment-56028543
It is unfortunately yet another entry in a growing list over the years.

We understand that working with a focus on compatibility requires more resources and more engineering trade-offs, but strongly believe it is nonetheless the way to go. We are very interested in helping with any underlying resource constraints.

This prompted CodeWeavers (who work on Wine and with Valve on Proton) developer Arek Hiler to write a blog post titled "Win32 Is The Only Stable ABI on Linux" and their ending statement is something people should think on:

I think this whole situation shows why creating native games for Linux is challenging. It’s hard to blame developers for targeting Windows and relying on Wine + friends. It’s just much more stable and much less likely to break and stay broken.

Hiler certainly isn't the only one to think like that, with another CodeWeavers developer Andrew Eikum mentioning on Hacker News some time ago:

As a long-time Linux dev (see my profile), I have also found this to be true. Linux userland APIs are unstable and change all the time. Some transitions that come to mind that have affected me personally: ALSA->pulse; libudev->libudev2->systemd; gstreamer 0.10->1.0. All of those changes required modifications to my software, and the backwards-compat tools that are provided are buggy and insufficient. Meanwhile, you can still write and run winmm[1] applications on Windows 10, and they will work in almost all cases. It's simply the case that the win32 API is more stable than Linux userland APIs, so it's entirely plausible that games will run better in Wine, which shares that stable ABI, than they will on Linux, especially as time goes on and Linux userland shifts yet again.

[1] winmm dates to the Windows 3.x days!

Situations like this can be pretty messy and this is not a case of open source versus secret closed source anti-cheat stuff either, since the glibc issue affected a Native Linux game (Shovel Knight) and Linux software libstrangle. No doubt there are other things yet to be discovered that were broken by the change.

It is of course also a case that Linux distributions need to ensure they do quality assurance testing, especially for gaming which can end up showing up issues quite easily and that bleeding-edge distributions can and clearly do end up breaking things by pulling new software in so quickly.

Article taken from GamingOnLinux.com.
43 Likes
About the author -
author picture
I am the owner of GamingOnLinux. After discovering Linux back in the days of Mandrake in 2003, I constantly came back to check on the progress of Linux until Ubuntu appeared on the scene and it helped me to really love it. You can reach me easily by emailing GamingOnLinux directly. Find me on Mastodon.
See more from me
The comments on this article are closed.
117 comments
Page: «10/12»
  Go to:

JordanPlayz158 Aug 18, 2022
Quoting: ShabbyXIt's made exactly so that libraries *can* break ABI if they have to, without the world imploding. glibc made a breaking change, and no matter how small, they should have made an incompatible version change. Yes that would still be inconvenient, but at least it's detectable and fixable. Imagine if python3 did all its backward incompatible things but still called itself python2.
Yes, that's exactly the point we were making, they didn't treat it as a breaking change as they just incremented the patch number by 1 rather than the major. Not to mention a lot of people have said it wasn't very clear and I could buy that as documentation doesn't seem to be a strong point in either linux or c in general (or I could be wrong (or xdg-desktop-portal could be a bad example of a c library) but when I attempted to make an application using xdg-desktop-portal they had the equivalent of a javadoc but that was about it, there are no examples provided in the repo for how to use any of the objects (or it wasn't clearly labeled or findable), there were no tutorials online and I looked into other repos (that are flatpaks as they must use xdg-desktop-portal to my knowledge) and it was a mess because different applications use different languages or sometimes used a middle man library or some combination of the two which made it difficult to know how to use the interface)
F.Ultra Aug 18, 2022
View PC info
  • Supporter
Quoting: JordanPlayz158
Quoting: minidouI fail to see how glibc devs could be blamed for removing a function deprecated for 17 years (though I can agree a major version bump should have been called for). Do videogame devs need that much nursing ? EAC linux lib is a recent piece of software, how many deprecated dependencies do they use ?
Quoting: TermyI tend to agree with the criticism about unstable APIs for many cases - but in this case, i really blame Epic for using a function deprecated for almost two decades in a recent piece of software.
From everything I've read from other people, it wasn't obvious that the function was depreciated. You shouldn't need to search up, "is x function depreciated glibc" for every function you use, it should be immediately obvious (like most languages I know have comments or a depreciation tag), some say that all distros somehow knew about this but you shouldn't need to be familiar with linux development or need to be in the right communities to know about a depreciated function.

Small nitpick here, but DT_HASH is not a libc function, it's a linker section in the ELF header of a shared object. The function that utilizes this section is dlopen() and that function have not changed and it works, it's only applications that tries to read the ELF data manually that have to handle DT_HASH and DT_GNU_HASH, and by all means it looks like the glibc devs "incorrectly" thought that dlopen was their ABI/API and not the actual linker section in the ELF headers.
slaapliedje Aug 18, 2022
Quoting: F.Ultra
Quoting: Beamboom
Quoting: F.UltraNew releases of WINE/Proton breaks games that worked with the older version from time to time so yes they are wrong in that WIN32 is the only stable ABI on Linux since it clearly isn't stable.

It's not a either/or, binary reality discussed here. It's not a question if this or that ever breaks at all, ever.

So how am I then to interpret CodeWeavers "WIN32 is the only stable ABI on Linux" when it have broken more times for me on my Linux system than glibc have ever done, yet this change by glibc triggers CodeWeavers to make that tweet?!
Of course the Codeweavers dev would say that. It's kind of their business to sell Win32 compatibility to Linux users. I thought I'd be nice and throw some money their way for the mac version of Crossover, but for the most part I've found newer wine is just more capable.
F.Ultra Aug 18, 2022
View PC info
  • Supporter
Quoting: JordanPlayz158
Quoting: ShabbyXIt's made exactly so that libraries *can* break ABI if they have to, without the world imploding. glibc made a breaking change, and no matter how small, they should have made an incompatible version change. Yes that would still be inconvenient, but at least it's detectable and fixable. Imagine if python3 did all its backward incompatible things but still called itself python2.
Yes, that's exactly the point we were making, they didn't treat it as a breaking change as they just incremented the patch number by 1 rather than the major. Not to mention a lot of people have said it wasn't very clear and I could buy that as documentation doesn't seem to be a strong point in either linux or c in general (or I could be wrong (or xdg-desktop-portal could be a bad example of a c library) but when I attempted to make an application using xdg-desktop-portal they had the equivalent of a javadoc but that was about it, there are no examples provided in the repo for how to use any of the objects (or it wasn't clearly labeled or findable), there were no tutorials online and I looked into other repos (that are flatpaks as they must use xdg-desktop-portal to my knowledge) and it was a mess because different applications use different languages or sometimes used a middle man library or some combination of the two which made it difficult to know how to use the interface)

glibc is very well documented, as is most C libraries on Linux (don't know where this misconception comes from), they didn't see the ELF sections as part of the glibc ABI/API though which is why this went under the radar for developers that isn't on the glibc mailing lists.

This was not a code change, it was a change in the automake configure script to no longer pass in the argument to ld to include the old hash section when building glibc.

Normally no library/application dev ever sees a change like this as a ABI/API change, but it looks like the glibc devs will do from now on.
JordanPlayz158 Aug 18, 2022
Quoting: F.Ultra
Quoting: JordanPlayz158
Quoting: minidouI fail to see how glibc devs could be blamed for removing a function deprecated for 17 years (though I can agree a major version bump should have been called for). Do videogame devs need that much nursing ? EAC linux lib is a recent piece of software, how many deprecated dependencies do they use ?
Quoting: TermyI tend to agree with the criticism about unstable APIs for many cases - but in this case, i really blame Epic for using a function deprecated for almost two decades in a recent piece of software.
From everything I've read from other people, it wasn't obvious that the function was depreciated. You shouldn't need to search up, "is x function depreciated glibc" for every function you use, it should be immediately obvious (like most languages I know have comments or a depreciation tag), some say that all distros somehow knew about this but you shouldn't need to be familiar with linux development or need to be in the right communities to know about a depreciated function.

Small nitpick here, but DT_HASH is not a libc function, it's a linker section in the ELF header of a shared object. The function that utilizes this section is dlopen() and that function have not changed and it works, it's only applications that tries to read the ELF data manually that have to handle DT_HASH and DT_GNU_HASH, and by all means it looks like the glibc devs "incorrectly" thought that dlopen was their ABI/API and not the actual linker section in the ELF headers.
Thank you for correcting the information, I am not a C developer nor have made linux applications "natively" so thank you for clearing that up
JordanPlayz158 Aug 18, 2022
Quoting: F.Ultra
Quoting: JordanPlayz158
Quoting: ShabbyXIt's made exactly so that libraries *can* break ABI if they have to, without the world imploding. glibc made a breaking change, and no matter how small, they should have made an incompatible version change. Yes that would still be inconvenient, but at least it's detectable and fixable. Imagine if python3 did all its backward incompatible things but still called itself python2.
Yes, that's exactly the point we were making, they didn't treat it as a breaking change as they just incremented the patch number by 1 rather than the major. Not to mention a lot of people have said it wasn't very clear and I could buy that as documentation doesn't seem to be a strong point in either linux or c in general (or I could be wrong (or xdg-desktop-portal could be a bad example of a c library) but when I attempted to make an application using xdg-desktop-portal they had the equivalent of a javadoc but that was about it, there are no examples provided in the repo for how to use any of the objects (or it wasn't clearly labeled or findable), there were no tutorials online and I looked into other repos (that are flatpaks as they must use xdg-desktop-portal to my knowledge) and it was a mess because different applications use different languages or sometimes used a middle man library or some combination of the two which made it difficult to know how to use the interface)

glibc is very well documented, as is most C libraries on Linux (don't know where this misconception comes from), they didn't see the ELF sections as part of the glibc ABI/API though which is why this went under the radar for developers that isn't on the glibc mailing lists.

This was not a code change, it was a change in the automake configure script to no longer pass in the argument to ld to include the old hash section when building glibc.

Normally no library/application dev ever sees a change like this as a ABI/API change, but it looks like the glibc devs will do from now on.
Well I just had one example experience (with xdg-desktop-portal, apologizes if they are an exception and not the norm for the standards of C libraries), I try to stay away from languages like C just as it doesn't really seem to click with me like a lot of other languages have (not to mention there are languages like Rust that some say will replace C and while I don't like rust either, it is leagues better for clicking with me). Also yeah, that makes a lot more sense as I was very confused by DT_HASH being in a build config file and not a function in the code when I saw the commit so again, thank you for clearing that up.
F.Ultra Aug 18, 2022
View PC info
  • Supporter
Quoting: slaapliedje
Quoting: F.Ultra
Quoting: Beamboom
Quoting: F.UltraNew releases of WINE/Proton breaks games that worked with the older version from time to time so yes they are wrong in that WIN32 is the only stable ABI on Linux since it clearly isn't stable.

It's not a either/or, binary reality discussed here. It's not a question if this or that ever breaks at all, ever.

So how am I then to interpret CodeWeavers "WIN32 is the only stable ABI on Linux" when it have broken more times for me on my Linux system than glibc have ever done, yet this change by glibc triggers CodeWeavers to make that tweet?!
Of course the Codeweavers dev would say that. It's kind of their business to sell Win32 compatibility to Linux users. I thought I'd be nice and throw some money their way for the mac version of Crossover, but for the most part I've found newer wine is just more capable.

I know, it just bothers me that there are whole reddit threads now filled with (very outspoken) people that believe them. Basically this whole thing is nothing with a storm in a glass, for a short period there existed a breakage for people using bleeding edge rolling distros, but thanks to this all being FOSS those distros could quickly patch the DT_HASH section back in and later glibc themselves fixed it upstream (yes they where initially a bit grumpy about it, which is understandably considering that they did it not for laughs but for actual benefits and have been waiting 16 years to do it. I also think that people put way to much emphasis on devs being grumpy, part of a devs job is to reject patches and being grumpy, otherwise things like glibc would be several GB by now).
F.Ultra Aug 18, 2022
View PC info
  • Supporter
Quoting: JordanPlayz158
Quoting: F.Ultra
Quoting: JordanPlayz158
Quoting: ShabbyXIt's made exactly so that libraries *can* break ABI if they have to, without the world imploding. glibc made a breaking change, and no matter how small, they should have made an incompatible version change. Yes that would still be inconvenient, but at least it's detectable and fixable. Imagine if python3 did all its backward incompatible things but still called itself python2.
Yes, that's exactly the point we were making, they didn't treat it as a breaking change as they just incremented the patch number by 1 rather than the major. Not to mention a lot of people have said it wasn't very clear and I could buy that as documentation doesn't seem to be a strong point in either linux or c in general (or I could be wrong (or xdg-desktop-portal could be a bad example of a c library) but when I attempted to make an application using xdg-desktop-portal they had the equivalent of a javadoc but that was about it, there are no examples provided in the repo for how to use any of the objects (or it wasn't clearly labeled or findable), there were no tutorials online and I looked into other repos (that are flatpaks as they must use xdg-desktop-portal to my knowledge) and it was a mess because different applications use different languages or sometimes used a middle man library or some combination of the two which made it difficult to know how to use the interface)

glibc is very well documented, as is most C libraries on Linux (don't know where this misconception comes from), they didn't see the ELF sections as part of the glibc ABI/API though which is why this went under the radar for developers that isn't on the glibc mailing lists.

This was not a code change, it was a change in the automake configure script to no longer pass in the argument to ld to include the old hash section when building glibc.

Normally no library/application dev ever sees a change like this as a ABI/API change, but it looks like the glibc devs will do from now on.
Well I just had one example experience (with xdg-desktop-portal, apologizes if they are an exception and not the norm for the standards of C libraries), I try to stay away from languages like C just as it doesn't really seem to click with me like a lot of other languages have (not to mention there are languages like Rust that some say will replace C and while I don't like rust either, it is leagues better for clicking with me). Also yeah, that makes a lot more sense as I was very confused by DT_HASH being in a build config file and not a function in the code when I saw the commit so again, thank you for clearing that up.

no probs, xdg-desktop-portal looks to be a 3d party tool for flatpak and unfortunately many such well intended extra tools are very rarely well documented.

To experience the glibc documentation just open a terminal and type in say "man dlopen" and see just how much information they have on that single function.
slaapliedje Aug 18, 2022
Quoting: F.Ultra
Quoting: JordanPlayz158
Quoting: ShabbyXIt's made exactly so that libraries *can* break ABI if they have to, without the world imploding. glibc made a breaking change, and no matter how small, they should have made an incompatible version change. Yes that would still be inconvenient, but at least it's detectable and fixable. Imagine if python3 did all its backward incompatible things but still called itself python2.
Yes, that's exactly the point we were making, they didn't treat it as a breaking change as they just incremented the patch number by 1 rather than the major. Not to mention a lot of people have said it wasn't very clear and I could buy that as documentation doesn't seem to be a strong point in either linux or c in general (or I could be wrong (or xdg-desktop-portal could be a bad example of a c library) but when I attempted to make an application using xdg-desktop-portal they had the equivalent of a javadoc but that was about it, there are no examples provided in the repo for how to use any of the objects (or it wasn't clearly labeled or findable), there were no tutorials online and I looked into other repos (that are flatpaks as they must use xdg-desktop-portal to my knowledge) and it was a mess because different applications use different languages or sometimes used a middle man library or some combination of the two which made it difficult to know how to use the interface)

glibc is very well documented, as is most C libraries on Linux (don't know where this misconception comes from), they didn't see the ELF sections as part of the glibc ABI/API though which is why this went under the radar for developers that isn't on the glibc mailing lists.

This was not a code change, it was a change in the automake configure script to no longer pass in the argument to ld to include the old hash section when building glibc.

Normally no library/application dev ever sees a change like this as a ABI/API change, but it looks like the glibc devs will do from now on.
Exactly this! It's a compile flag. Nothing more.
JordanPlayz158 Aug 18, 2022
Quoting: F.Ultra
Quoting: JordanPlayz158
Quoting: F.Ultra
Quoting: JordanPlayz158
Quoting: ShabbyXIt's made exactly so that libraries *can* break ABI if they have to, without the world imploding. glibc made a breaking change, and no matter how small, they should have made an incompatible version change. Yes that would still be inconvenient, but at least it's detectable and fixable. Imagine if python3 did all its backward incompatible things but still called itself python2.
Yes, that's exactly the point we were making, they didn't treat it as a breaking change as they just incremented the patch number by 1 rather than the major. Not to mention a lot of people have said it wasn't very clear and I could buy that as documentation doesn't seem to be a strong point in either linux or c in general (or I could be wrong (or xdg-desktop-portal could be a bad example of a c library) but when I attempted to make an application using xdg-desktop-portal they had the equivalent of a javadoc but that was about it, there are no examples provided in the repo for how to use any of the objects (or it wasn't clearly labeled or findable), there were no tutorials online and I looked into other repos (that are flatpaks as they must use xdg-desktop-portal to my knowledge) and it was a mess because different applications use different languages or sometimes used a middle man library or some combination of the two which made it difficult to know how to use the interface)

glibc is very well documented, as is most C libraries on Linux (don't know where this misconception comes from), they didn't see the ELF sections as part of the glibc ABI/API though which is why this went under the radar for developers that isn't on the glibc mailing lists.

This was not a code change, it was a change in the automake configure script to no longer pass in the argument to ld to include the old hash section when building glibc.

Normally no library/application dev ever sees a change like this as a ABI/API change, but it looks like the glibc devs will do from now on.
Well I just had one example experience (with xdg-desktop-portal, apologizes if they are an exception and not the norm for the standards of C libraries), I try to stay away from languages like C just as it doesn't really seem to click with me like a lot of other languages have (not to mention there are languages like Rust that some say will replace C and while I don't like rust either, it is leagues better for clicking with me). Also yeah, that makes a lot more sense as I was very confused by DT_HASH being in a build config file and not a function in the code when I saw the commit so again, thank you for clearing that up.

no probs, xdg-desktop-portal looks to be a 3d party tool for flatpak and unfortunately many such well intended extra tools are very rarely well documented.

To experience the glibc documentation just open a terminal and type in say "man dlopen" and see just how much information they have on that single function.
Yeah, I tried that then I thought `man elf` and there is a page, is this from glibc's documentation or something else as if they have control over that man page then they should definitely change this portion
 
       d_tag  This member may have any of the following values:

              DT_NULL     Marks end of dynamic section

              DT_NEEDED   String table offset to name of a needed library

              DT_PLTRELSZ Size in bytes of PLT relocation entries

              DT_PLTGOT   Address of PLT and/or GOT

              DT_HASH     Address of symbol hash table

to
       d_tag  This member may have any of the following values:
              ...
              DT_HASH     [s]Address of symbol hash table[/s] PLEASE FOR THE LOVE OF GOD DO NOT USE THIS AND INSTEAD USE DT_GNU_HASH IN THE FUTURE, PLEASE USE ANYTHING OTHER THAN THIS

and then problem solved but it is possible this man page is not in their control but if it is, I propose to add my addition to it lol
While you're here, please consider supporting GamingOnLinux on:

Reward Tiers: Patreon. Plain Donations: PayPal.

This ensures all of our main content remains totally free for everyone! Patreon supporters can also remove all adverts and sponsors! Supporting us helps bring good, fresh content. Without your continued support, we simply could not continue!

You can find even more ways to support us on this dedicated page any time. If you already are, thank you!
The comments on this article are closed.