Don't want to see articles from a certain category? When logged in, go to your User Settings and adjust your feed in the Content Preferences section where you can block tags!
We do often include affiliate links to earn us some pennies. See more here.

Shader cache downloads being a nuisance? Valve may have solved it

By - | Views: 72,618

Valve's shader cache system can be extremely helpful for Linux gaming, however it can also at times be quite problematic but Valve may have solved a big problem with it.

The idea behind the system is that you're supposed to see small regular downloads for the shader cache of each game, as Valve continue to build it up and then spread it out to everyone to help improve game performance on Linux and Steam Deck. However, over the years quite a lot of people have reported how it kept giving full downloads of the whole thing instead of being incremental.

Writing on the GitHub bug report from 2021 today, Valve developer Pierre-Loup Griffais said:

Shader cache downloads are designed to be incremental. Reports here indicate a daily download of the full shader cache file size, which is inconsistent with expectations - a small download and full disk write is expected instead.

After looking at the logs shared on this issue and on #9200, we've investigated and found a server-side problem with the system that keeps track of the ordering of shaders in those updates.

We've just rolled out an initial fix and will continue to monitor the situation. If the issue persists, please comment on this report and attach your shader_log[.previous].txt as indicated above.

Hopefully now fewer people (or no one) will see the issue but do let them know on the bug report if it happens to you.

Article taken from GamingOnLinux.com.
Tags: Misc, Steam
37 Likes
About the author -
author picture
I am the owner of GamingOnLinux. After discovering Linux back in the days of Mandrake in 2003, I constantly came back to check on the progress of Linux until Ubuntu appeared on the scene and it helped me to really love it. You can reach me easily by emailing GamingOnLinux directly. Find me on Mastodon.
See more from me
The comments on this article are closed.
22 comments
Page: «2/3»
  Go to:

Bogomips Feb 16, 2023
My internet connection is still ADSL with a narrow bandwith, and I have disabled the pre-caching a long time ago, because there is no benefits for me I only play a handful of games from my library ten times bigger. So a per-game option would be nice.
F.Ultra Feb 16, 2023
View PC info
  • Supporter
Any one knows how this all works with games like Callisto Protocol where Steam downloads shader caches only for the game to then upon launch to rebuild the shaders anyway, sounds like the two systems are fighting each other and that perhaps the steam shader cache should be disabled for games like this, or does it still help in some way that I don't understand?
MayeulC Feb 16, 2023
Quoting: F.UltraAny one knows how this all works with games like Callisto Protocol where Steam downloads shader caches only for the game to then upon launch to rebuild the shaders anyway, sounds like the two systems are fighting each other and that perhaps the steam shader cache should be disabled for games like this, or does it still help in some way that I don't understand?

I haven't played that specific game, but lots of things can happen.

Firstly, having a cache should accelerate your launch times, then, because cached shaders are not recompiled.

Secondly, drivers tend to not compile shaders until they are actually used, which cause stutter, but reduce compilation time a lot (a whole lot of shaders are apparently unused, usually). That one is debatable, but this is where the steam shader cache usually helps. There are ways to force it, so it could be what the game is doing here.

Indeed, if the game recompiles everything, you should just expect a very minor speed improvement on launch (assuming background processing of shaders is disabled, for a "fair" comparison: otherwise, launch speed should increase a lot).
CatKiller Feb 16, 2023
View PC info
  • Supporter Plus
For those that are interested in the whys of the shader problem, this article is a good read.
rcrit Feb 16, 2023
View PC info
  • Supporter Plus
Seems like there is more room for improvement. I leave steam running all the time, sometimes for weeks, with an occasional scheduled download but nothing major. Every time I restart the client I get 20+ downloads immediately, plus recompilation and today was no different. I restarted just now and got 21 downloads for a total of 2.6GB (with associated fossilize_replay processes).

Or maybe this is its last gasp and next time things will be golden.
timekills Feb 17, 2023
Quoting: Guestshader cache was broken for me. it just kept downloading and piling up to 30GB in total, so I disabled it, and I didn't notice any game-breaking stutters, probably because my 16 threads CPU dealt with compiling shaders fast enough. If shaders can be shipped precompiled, why don't games do it?

if shaders should be compiled for your hardware ans software specifically, why even bothering with this precompiled cache system?

Short answer to the two questions is
1: As you mention in the 2nd question, it is both hardware and OS dependent. The combination of those that each player could have is obviously unknown to the game developer, as well as a large variable so adding those to the game would include a SIGNIFICANT file size increase for everyone with most of that data irrelevant to each user.

2: Of equal significance, the cache is more required for Linux as the programs and graphics are designed to work with (typically) Windows drivers. So pre-adding them as above is not useful for the majority of people. However, Steam building then for Linux already removes one major variable, and with the Steam Deck builds removes all variables as the hardware is consistent. They have the info on your system, so know which cache is applicable whether Steam Deck or PC.

Aside: this is a common issue for later console system emulation, and why some emulators can support shared graphical caches and some do not. I.E. how much a hardware/OS difference impacts the emulation ability determinesv whether it's even viable to share.
Korhaka Feb 17, 2023
Steam → Settings → Shader Pre-Caching → Allow background processing of Vulkan shaders

Since I enabled that I had pretty much forgot that shader pre-caching was a thing.
14 Feb 18, 2023
View PC info
  • Supporter Plus
Quoting: Schattenspiegelok so now we have only a few hundred kb downloads and half an hour of shader pre-compiling in the background...for however many people use this feature and dare to have more than one game installed ... this seems a bit wasteful energy wise, does it not?
It would happen during gameplay anyway, so it's only a waste if you don't play the games you have installed. If that's true, then you are wasting money and material on your hard drive too.

Your same thought has crossed my mind, too. It's partly the reason why I only keep games installed which I've played within the last 12 months.
14 Feb 18, 2023
View PC info
  • Supporter Plus
Quoting: rcritEvery time I restart the client I get 20+ downloads immediately, plus recompilation and today was no different. I restarted just now and got 21 downloads for a total of 2.6GB (with associated fossilize_replay processes).

Or maybe this is its last gasp and next time things will be golden.
I noticed the same happening to me. I'm not sure if it has gone away because I haven't been able to get on Steam for a while. But you're not alone!
rcrit Feb 18, 2023
View PC info
  • Supporter Plus
Quoting: 14
Quoting: rcritEvery time I restart the client I get 20+ downloads immediately, plus recompilation and today was no different. I restarted just now and got 21 downloads for a total of 2.6GB (with associated fossilize_replay processes).

Or maybe this is its last gasp and next time things will be golden.
I noticed the same happening to me. I'm not sure if it has gone away because I haven't been able to get on Steam for a while. But you're not alone!

Thanks for that!

I restarted the client just now to apply the latest beta update and for the first time in recent memory not a single download was done, so I have hope that things are indeed fixed.
While you're here, please consider supporting GamingOnLinux on:

Reward Tiers: Patreon. Plain Donations: PayPal.

This ensures all of our main content remains totally free for everyone! Patreon supporters can also remove all adverts and sponsors! Supporting us helps bring good, fresh content. Without your continued support, we simply could not continue!

You can find even more ways to support us on this dedicated page any time. If you already are, thank you!
The comments on this article are closed.