Confused on Steam Play and Proton? Be sure to check out our guide.

Recently it was noticed that users on more bleeding-edge Linux distributions that updated saw Easy Anti-Cheat no longer working on Linux, the culprit was glibc and now a Valve developer has spoken out about it.

Writing in a small thread on Twitter, Valve developer Pierre-Loup Griffais said:

Unfortunate that upstream glibc discussion on DT_HASH isn't coming out strongly in favor of prioritizing compatibility with pre-existing applications. Every such instance contributes to damaging the idea of desktop Linux as a viable target for third-party developers.

Our thoughts on the topic from this prior compatibility issue in BlueZ apply more than ever: https://github.com/bluez/bluez/commit/35a2c50437cca4d26ac6537ce3a964bb509c9b62#commitcomment-56028543
It is unfortunately yet another entry in a growing list over the years.

We understand that working with a focus on compatibility requires more resources and more engineering trade-offs, but strongly believe it is nonetheless the way to go. We are very interested in helping with any underlying resource constraints.

This prompted CodeWeavers (who work on Wine and with Valve on Proton) developer Arek Hiler to write a blog post titled "Win32 Is The Only Stable ABI on Linux" and their ending statement is something people should think on:

I think this whole situation shows why creating native games for Linux is challenging. It’s hard to blame developers for targeting Windows and relying on Wine + friends. It’s just much more stable and much less likely to break and stay broken.

Hiler certainly isn't the only one to think like that, with another CodeWeavers developer Andrew Eikum mentioning on Hacker News some time ago:

As a long-time Linux dev (see my profile), I have also found this to be true. Linux userland APIs are unstable and change all the time. Some transitions that come to mind that have affected me personally: ALSA->pulse; libudev->libudev2->systemd; gstreamer 0.10->1.0. All of those changes required modifications to my software, and the backwards-compat tools that are provided are buggy and insufficient. Meanwhile, you can still write and run winmm[1] applications on Windows 10, and they will work in almost all cases. It's simply the case that the win32 API is more stable than Linux userland APIs, so it's entirely plausible that games will run better in Wine, which shares that stable ABI, than they will on Linux, especially as time goes on and Linux userland shifts yet again.

[1] winmm dates to the Windows 3.x days!

Situations like this can be pretty messy and this is not a case of open source versus secret closed source anti-cheat stuff either, since the glibc issue affected a Native Linux game (Shovel Knight) and Linux software libstrangle. No doubt there are other things yet to be discovered that were broken by the change.

It is of course also a case that Linux distributions need to ensure they do quality assurance testing, especially for gaming which can end up showing up issues quite easily and that bleeding-edge distributions can and clearly do end up breaking things by pulling new software in so quickly.

Article taken from GamingOnLinux.com.
43 Likes
We do often include affiliate links to earn us some pennies. We are currently affiliated with GOG and Humble Store. See more here.
About the author -
author picture
I am the owner of GamingOnLinux. After discovering Linux back in the days of Mandrake in 2003, I constantly came back to check on the progress of Linux until Ubuntu appeared on the scene and it helped me to really love it. You can reach me easily by emailing GamingOnLinux directly.
See more from me
123 comments
Page: «11/13»
  Go to:

Quoting: F.Ultra
Quoting: JordanPlayz158
Quoting: minidouI fail to see how glibc devs could be blamed for removing a function deprecated for 17 years (though I can agree a major version bump should have been called for). Do videogame devs need that much nursing ? EAC linux lib is a recent piece of software, how many deprecated dependencies do they use ?
Quoting: TermyI tend to agree with the criticism about unstable APIs for many cases - but in this case, i really blame Epic for using a function deprecated for almost two decades in a recent piece of software.
From everything I've read from other people, it wasn't obvious that the function was depreciated. You shouldn't need to search up, "is x function depreciated glibc" for every function you use, it should be immediately obvious (like most languages I know have comments or a depreciation tag), some say that all distros somehow knew about this but you shouldn't need to be familiar with linux development or need to be in the right communities to know about a depreciated function.

Small nitpick here, but DT_HASH is not a libc function, it's a linker section in the ELF header of a shared object. The function that utilizes this section is dlopen() and that function have not changed and it works, it's only applications that tries to read the ELF data manually that have to handle DT_HASH and DT_GNU_HASH, and by all means it looks like the glibc devs "incorrectly" thought that dlopen was their ABI/API and not the actual linker section in the ELF headers.
Thank you for correcting the information, I am not a C developer nor have made linux applications "natively" so thank you for clearing that up
Quoting: F.Ultra
Quoting: JordanPlayz158
Quoting: ShabbyXIt's made exactly so that libraries *can* break ABI if they have to, without the world imploding. glibc made a breaking change, and no matter how small, they should have made an incompatible version change. Yes that would still be inconvenient, but at least it's detectable and fixable. Imagine if python3 did all its backward incompatible things but still called itself python2.
Yes, that's exactly the point we were making, they didn't treat it as a breaking change as they just incremented the patch number by 1 rather than the major. Not to mention a lot of people have said it wasn't very clear and I could buy that as documentation doesn't seem to be a strong point in either linux or c in general (or I could be wrong (or xdg-desktop-portal could be a bad example of a c library) but when I attempted to make an application using xdg-desktop-portal they had the equivalent of a javadoc but that was about it, there are no examples provided in the repo for how to use any of the objects (or it wasn't clearly labeled or findable), there were no tutorials online and I looked into other repos (that are flatpaks as they must use xdg-desktop-portal to my knowledge) and it was a mess because different applications use different languages or sometimes used a middle man library or some combination of the two which made it difficult to know how to use the interface)

glibc is very well documented, as is most C libraries on Linux (don't know where this misconception comes from), they didn't see the ELF sections as part of the glibc ABI/API though which is why this went under the radar for developers that isn't on the glibc mailing lists.

This was not a code change, it was a change in the automake configure script to no longer pass in the argument to ld to include the old hash section when building glibc.

Normally no library/application dev ever sees a change like this as a ABI/API change, but it looks like the glibc devs will do from now on.
Well I just had one example experience (with xdg-desktop-portal, apologizes if they are an exception and not the norm for the standards of C libraries), I try to stay away from languages like C just as it doesn't really seem to click with me like a lot of other languages have (not to mention there are languages like Rust that some say will replace C and while I don't like rust either, it is leagues better for clicking with me). Also yeah, that makes a lot more sense as I was very confused by DT_HASH being in a build config file and not a function in the code when I saw the commit so again, thank you for clearing that up.
F.Ultra 18 Aug
View PC info
  • Supporter
Quoting: slaapliedje
Quoting: F.Ultra
Quoting: Beamboom
Quoting: F.UltraNew releases of WINE/Proton breaks games that worked with the older version from time to time so yes they are wrong in that WIN32 is the only stable ABI on Linux since it clearly isn't stable.

It's not a either/or, binary reality discussed here. It's not a question if this or that ever breaks at all, ever.

So how am I then to interpret CodeWeavers "WIN32 is the only stable ABI on Linux" when it have broken more times for me on my Linux system than glibc have ever done, yet this change by glibc triggers CodeWeavers to make that tweet?!
Of course the Codeweavers dev would say that. It's kind of their business to sell Win32 compatibility to Linux users. I thought I'd be nice and throw some money their way for the mac version of Crossover, but for the most part I've found newer wine is just more capable.

I know, it just bothers me that there are whole reddit threads now filled with (very outspoken) people that believe them. Basically this whole thing is nothing with a storm in a glass, for a short period there existed a breakage for people using bleeding edge rolling distros, but thanks to this all being FOSS those distros could quickly patch the DT_HASH section back in and later glibc themselves fixed it upstream (yes they where initially a bit grumpy about it, which is understandably considering that they did it not for laughs but for actual benefits and have been waiting 16 years to do it. I also think that people put way to much emphasis on devs being grumpy, part of a devs job is to reject patches and being grumpy, otherwise things like glibc would be several GB by now).
F.Ultra 18 Aug
View PC info
  • Supporter
Quoting: JordanPlayz158
Quoting: F.Ultra
Quoting: JordanPlayz158
Quoting: ShabbyXIt's made exactly so that libraries *can* break ABI if they have to, without the world imploding. glibc made a breaking change, and no matter how small, they should have made an incompatible version change. Yes that would still be inconvenient, but at least it's detectable and fixable. Imagine if python3 did all its backward incompatible things but still called itself python2.
Yes, that's exactly the point we were making, they didn't treat it as a breaking change as they just incremented the patch number by 1 rather than the major. Not to mention a lot of people have said it wasn't very clear and I could buy that as documentation doesn't seem to be a strong point in either linux or c in general (or I could be wrong (or xdg-desktop-portal could be a bad example of a c library) but when I attempted to make an application using xdg-desktop-portal they had the equivalent of a javadoc but that was about it, there are no examples provided in the repo for how to use any of the objects (or it wasn't clearly labeled or findable), there were no tutorials online and I looked into other repos (that are flatpaks as they must use xdg-desktop-portal to my knowledge) and it was a mess because different applications use different languages or sometimes used a middle man library or some combination of the two which made it difficult to know how to use the interface)

glibc is very well documented, as is most C libraries on Linux (don't know where this misconception comes from), they didn't see the ELF sections as part of the glibc ABI/API though which is why this went under the radar for developers that isn't on the glibc mailing lists.

This was not a code change, it was a change in the automake configure script to no longer pass in the argument to ld to include the old hash section when building glibc.

Normally no library/application dev ever sees a change like this as a ABI/API change, but it looks like the glibc devs will do from now on.
Well I just had one example experience (with xdg-desktop-portal, apologizes if they are an exception and not the norm for the standards of C libraries), I try to stay away from languages like C just as it doesn't really seem to click with me like a lot of other languages have (not to mention there are languages like Rust that some say will replace C and while I don't like rust either, it is leagues better for clicking with me). Also yeah, that makes a lot more sense as I was very confused by DT_HASH being in a build config file and not a function in the code when I saw the commit so again, thank you for clearing that up.

no probs, xdg-desktop-portal looks to be a 3d party tool for flatpak and unfortunately many such well intended extra tools are very rarely well documented.

To experience the glibc documentation just open a terminal and type in say "man dlopen" and see just how much information they have on that single function.
slaapliedje 18 Aug
View PC info
  • Supporter Plus
Quoting: F.Ultra
Quoting: JordanPlayz158
Quoting: ShabbyXIt's made exactly so that libraries *can* break ABI if they have to, without the world imploding. glibc made a breaking change, and no matter how small, they should have made an incompatible version change. Yes that would still be inconvenient, but at least it's detectable and fixable. Imagine if python3 did all its backward incompatible things but still called itself python2.
Yes, that's exactly the point we were making, they didn't treat it as a breaking change as they just incremented the patch number by 1 rather than the major. Not to mention a lot of people have said it wasn't very clear and I could buy that as documentation doesn't seem to be a strong point in either linux or c in general (or I could be wrong (or xdg-desktop-portal could be a bad example of a c library) but when I attempted to make an application using xdg-desktop-portal they had the equivalent of a javadoc but that was about it, there are no examples provided in the repo for how to use any of the objects (or it wasn't clearly labeled or findable), there were no tutorials online and I looked into other repos (that are flatpaks as they must use xdg-desktop-portal to my knowledge) and it was a mess because different applications use different languages or sometimes used a middle man library or some combination of the two which made it difficult to know how to use the interface)

glibc is very well documented, as is most C libraries on Linux (don't know where this misconception comes from), they didn't see the ELF sections as part of the glibc ABI/API though which is why this went under the radar for developers that isn't on the glibc mailing lists.

This was not a code change, it was a change in the automake configure script to no longer pass in the argument to ld to include the old hash section when building glibc.

Normally no library/application dev ever sees a change like this as a ABI/API change, but it looks like the glibc devs will do from now on.
Exactly this! It's a compile flag. Nothing more.
Quoting: F.Ultra
Quoting: JordanPlayz158
Quoting: F.Ultra
Quoting: JordanPlayz158
Quoting: ShabbyXIt's made exactly so that libraries *can* break ABI if they have to, without the world imploding. glibc made a breaking change, and no matter how small, they should have made an incompatible version change. Yes that would still be inconvenient, but at least it's detectable and fixable. Imagine if python3 did all its backward incompatible things but still called itself python2.
Yes, that's exactly the point we were making, they didn't treat it as a breaking change as they just incremented the patch number by 1 rather than the major. Not to mention a lot of people have said it wasn't very clear and I could buy that as documentation doesn't seem to be a strong point in either linux or c in general (or I could be wrong (or xdg-desktop-portal could be a bad example of a c library) but when I attempted to make an application using xdg-desktop-portal they had the equivalent of a javadoc but that was about it, there are no examples provided in the repo for how to use any of the objects (or it wasn't clearly labeled or findable), there were no tutorials online and I looked into other repos (that are flatpaks as they must use xdg-desktop-portal to my knowledge) and it was a mess because different applications use different languages or sometimes used a middle man library or some combination of the two which made it difficult to know how to use the interface)

glibc is very well documented, as is most C libraries on Linux (don't know where this misconception comes from), they didn't see the ELF sections as part of the glibc ABI/API though which is why this went under the radar for developers that isn't on the glibc mailing lists.

This was not a code change, it was a change in the automake configure script to no longer pass in the argument to ld to include the old hash section when building glibc.

Normally no library/application dev ever sees a change like this as a ABI/API change, but it looks like the glibc devs will do from now on.
Well I just had one example experience (with xdg-desktop-portal, apologizes if they are an exception and not the norm for the standards of C libraries), I try to stay away from languages like C just as it doesn't really seem to click with me like a lot of other languages have (not to mention there are languages like Rust that some say will replace C and while I don't like rust either, it is leagues better for clicking with me). Also yeah, that makes a lot more sense as I was very confused by DT_HASH being in a build config file and not a function in the code when I saw the commit so again, thank you for clearing that up.

no probs, xdg-desktop-portal looks to be a 3d party tool for flatpak and unfortunately many such well intended extra tools are very rarely well documented.

To experience the glibc documentation just open a terminal and type in say "man dlopen" and see just how much information they have on that single function.
Yeah, I tried that then I thought `man elf` and there is a page, is this from glibc's documentation or something else as if they have control over that man page then they should definitely change this portion
 
       d_tag  This member may have any of the following values:

              DT_NULL     Marks end of dynamic section

              DT_NEEDED   String table offset to name of a needed library

              DT_PLTRELSZ Size in bytes of PLT relocation entries

              DT_PLTGOT   Address of PLT and/or GOT

              DT_HASH     Address of symbol hash table

to
       d_tag  This member may have any of the following values:
              ...
              DT_HASH     [s]Address of symbol hash table[/s] PLEASE FOR THE LOVE OF GOD DO NOT USE THIS AND INSTEAD USE DT_GNU_HASH IN THE FUTURE, PLEASE USE ANYTHING OTHER THAN THIS

and then problem solved but it is possible this man page is not in their control but if it is, I propose to add my addition to it lol
F.Ultra 18 Aug
View PC info
  • Supporter
Quoting: EagleDelta
Quoting: BlackBloodRumIt's all on usage case. For a server, stable is better.

What is the definition of "stable" here? For things like RHEL, CentOS, Debian Stable, SLES, sometimes even Ubuntu Server "stable" generally means very, very old versions of software that many times is no longer maintained by the upstream developers. In many cases when I was a SysAdmin, that meant we had to install external YUM or APT repos to install the software that the project or company "supported" because what was included in the distribution was no longer supported by the community or the company behind the FOSS project and as such, support was entirely on us to manage.

As for enterprise, the advent of patterns like containerization (Docker, containerd, Flatpak, etc) has enabled the ability to have a stable OS with newer (or older) libraries running inside the application "container". However, in the case of glibc, I'm not sure that's an option. Especially on Desktops. glibc is so core to the OS that the container would have to ship GFX drivers as well that are built on the shipped glibc version separate from the host OS and anything else that entails.

When it comes to core system functionality, compatibility will always win out from the non-technical customers/users. There's a reason Microsoft has not successfully moved away from win32 yet, despite the amount of times they've wanted to and tried.

Stable means that the API/ABI are unchanged for the lifecycle of the distribution so that enterprises can build software for say RHEL7 and their customers can then be sure that this software will work exactly identical on RHEL7.0 as it will on RHEL7.9 even though there are 5 years between the two releases.

While the included software might be versions that are no longer being maintained upstream they are being maintained by the distribution developers (this is why a subscription to RHEL does not come cheap) and security features are backported, which is why even if it might look like you run PHP v3.4 you do infact run a highly patched version of PHP that is just as secure as PHP 8.x but walks and talks like 3.4

That you had to import external repositories with newer software was a decision made by your company developers to use that newer software instead of the one provided by the distro of choice. Either the devs have to be kept in check (they will always want the new shiny) or the choice of distribution have to match what they need/want, the kind of combination that your workplace tried is usually a road to unnecessary complexity and problems.
F.Ultra 18 Aug
View PC info
  • Supporter
Quoting: JordanPlayz158
Quoting: F.Ultra
Quoting: JordanPlayz158
Quoting: F.Ultra
Quoting: JordanPlayz158
Quoting: ShabbyXIt's made exactly so that libraries *can* break ABI if they have to, without the world imploding. glibc made a breaking change, and no matter how small, they should have made an incompatible version change. Yes that would still be inconvenient, but at least it's detectable and fixable. Imagine if python3 did all its backward incompatible things but still called itself python2.
Yes, that's exactly the point we were making, they didn't treat it as a breaking change as they just incremented the patch number by 1 rather than the major. Not to mention a lot of people have said it wasn't very clear and I could buy that as documentation doesn't seem to be a strong point in either linux or c in general (or I could be wrong (or xdg-desktop-portal could be a bad example of a c library) but when I attempted to make an application using xdg-desktop-portal they had the equivalent of a javadoc but that was about it, there are no examples provided in the repo for how to use any of the objects (or it wasn't clearly labeled or findable), there were no tutorials online and I looked into other repos (that are flatpaks as they must use xdg-desktop-portal to my knowledge) and it was a mess because different applications use different languages or sometimes used a middle man library or some combination of the two which made it difficult to know how to use the interface)

glibc is very well documented, as is most C libraries on Linux (don't know where this misconception comes from), they didn't see the ELF sections as part of the glibc ABI/API though which is why this went under the radar for developers that isn't on the glibc mailing lists.

This was not a code change, it was a change in the automake configure script to no longer pass in the argument to ld to include the old hash section when building glibc.

Normally no library/application dev ever sees a change like this as a ABI/API change, but it looks like the glibc devs will do from now on.
Well I just had one example experience (with xdg-desktop-portal, apologizes if they are an exception and not the norm for the standards of C libraries), I try to stay away from languages like C just as it doesn't really seem to click with me like a lot of other languages have (not to mention there are languages like Rust that some say will replace C and while I don't like rust either, it is leagues better for clicking with me). Also yeah, that makes a lot more sense as I was very confused by DT_HASH being in a build config file and not a function in the code when I saw the commit so again, thank you for clearing that up.

no probs, xdg-desktop-portal looks to be a 3d party tool for flatpak and unfortunately many such well intended extra tools are very rarely well documented.

To experience the glibc documentation just open a terminal and type in say "man dlopen" and see just how much information they have on that single function.
Yeah, I tried that then I thought `man elf` and there is a page, is this from glibc's documentation or something else as if they have control over that man page then they should definitely change this portion
 
       d_tag  This member may have any of the following values:

              DT_NULL     Marks end of dynamic section

              DT_NEEDED   String table offset to name of a needed library

              DT_PLTRELSZ Size in bytes of PLT relocation entries

              DT_PLTGOT   Address of PLT and/or GOT

              DT_HASH     Address of symbol hash table

to
       d_tag  This member may have any of the following values:
              ...
              DT_HASH     [s]Address of symbol hash table[/s] PLEASE FOR THE LOVE OF GOD DO NOT USE THIS AND INSTEAD USE DT_GNU_HASH IN THE FUTURE, PLEASE USE ANYTHING OTHER THAN THIS

and then problem solved but it is possible this man page is not in their control but if it is, I propose to add my addition to it lol

Yes the elf man page looks to come from not glibc since it does not cover DT_GNU_HASH, it looks to be from the kernel and have a git page here: https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git and I can see that DT_GNU_HASH is still not in the elf(5) page in MAIN/HEAD.

However I guess that this could be problematic on another level as well since not only is glibc not the only libc out there for Linux but it is also not the only shared object and this man page covers the ELF format itself and not just the one supported by the glibc ld binary.
Eike 18 Aug
View PC info
  • Supporter Plus
Quoting: F.Ultra
Quoting: JordanPlayz158
Quoting: minidouI fail to see how glibc devs could be blamed for removing a function deprecated for 17 years (though I can agree a major version bump should have been called for). Do videogame devs need that much nursing ? EAC linux lib is a recent piece of software, how many deprecated dependencies do they use ?
Quoting: TermyI tend to agree with the criticism about unstable APIs for many cases - but in this case, i really blame Epic for using a function deprecated for almost two decades in a recent piece of software.
From everything I've read from other people, it wasn't obvious that the function was depreciated. You shouldn't need to search up, "is x function depreciated glibc" for every function you use, it should be immediately obvious (like most languages I know have comments or a depreciation tag), some say that all distros somehow knew about this but you shouldn't need to be familiar with linux development or need to be in the right communities to know about a depreciated function.

Small nitpick here, but DT_HASH is not a libc function, it's a linker section in the ELF header of a shared object. The function that utilizes this section is dlopen() and that function have not changed and it works, it's only applications that tries to read the ELF data manually that have to handle DT_HASH and DT_GNU_HASH, and by all means it looks like the glibc devs "incorrectly" thought that dlopen was their ABI/API and not the actual linker section in the ELF headers.

I find this neither nitpcking nor small! So it isn't an actual API change, but considered an internal data invisible to almost every user program.
dibz 18 Aug
Quoting: shorbergI wasn't going to comment on this because it is way off-topic, but having thought about it for a while I decided to do so anyway, in the interest of public education on neural diversity. Liam or moderators may remove this if they think it inappropriate.

Quoting: dibzYeah, one of the best and worst things about Linux is the vocal community. And let's be honest, the people that flock tend to have very strong personalities -- borderline autistic at times.
Autistic is not the word you are looking for here. It is a common and widespread misconception, but it really isn't what you are looking for to describe the individuals you are hinting at. Autistics are often asocial, generally not antisocial. Typically, if an autistic person learns they have said or done something that have hurt someone, they will feel intense remorse.

A "borderline autistic" person is a perfectly average person, you likely won't even notice anything different except perhaps slightly increased anxiety compared to baseline human their age. At most you might describe them as someone who likes to stay home and read a book on a Saturday every now and then.

They are not people who will try to rally the masses, they will probably just stand at the sideline wishing they could contribute more. There are exceptions though, a certain fellow countrywoman is a strong personality who does quite a bit to hold people accountable for the climate.

Quoting wikipedia,
QuoteThe autism spectrum is a range of neurodevelopmental conditions generally characterized by difficulties in social interactions and communication, repetitive behaviours, intense interests, and unusual responses to sensory stimuli.
The difficulty in social interaction and communication refers to discomfort and avoidance and reduced body language. But many autistics won't put themselves in most social situations to begin with, either because of a lack of interest in social activities or because it confuses them. Even mailing-lists, or forum posting is something that scares many.

Source: I've done a lot of reading on autism, I have talked to a lot of professionals, I have friends who are autistic and I am autistic myself. I am however far from an expert, and open to learn more.

PS. What I have described as autistics above is not a set of all people with autism spectrum disorder, it is a generalised set of the ones who are high-functioning enough that you are likely to have been in contact with them in something as complex as technical online discussions.

I appreciate the explanations. Though, my use of "Borderline Autistic" is actually close to home for me. Without airing my laundry on the internet, I will say that I used the term in the context that I learned it in the first place -- in a healthcare setting when diagnosing one of my children who I could absolutely see being a "strongly opinionated linux user" like many of us, heh.
While you're here, please consider supporting GamingOnLinux on:

Reward Tiers: Patreon. Plain Donations: PayPal.

This ensures all of our main content remains totally free for everyone with no article paywalls. We also don't have tons of adverts, there's also no tracking and we respect your privacy. Just good, fresh content. Without your continued support, we simply could not continue!

You can find even more ways to support us on this dedicated page any time. If you already are, thank you!
Login / Register

Or login with...
Sign in with Steam Sign in with Twitter Sign in with Google
Social logins require cookies to stay logged in.