Update: I spoke with Valve about the issue and they confirmed change is coming — "Yes, we're aware of the issue and will soon be shipping an update to the SteamOS Beta to address it. After the update, Firefox will be installed as a flatpak, making it easier to keep up to date."
For all the pros of the Steam Deck like the full KDE Plasma desktop mode, there's a bit of a sore spot with Valve being too slow to update some of the included software.
Right now, the version of Firefox that is available on the Steam Deck is stuck at 96.0.3 even if you've opted into the very latest SteamOS 3.3 Beta. This is actually rather important, because web browsers are constantly under attack and new vulnerabilities appear all the time. With this version of Firefox, there's various flaws still present in it.
It's not entirely clear why Valve don't just remove the Firefox package included directly in SteamOS, and have the Flatpak version installed instead like they give you the option for with Chrome with their built-in installer that keeps it up to date too.
For now if you want a secure browsing experience on the Steam Deck you should either install and run the Firefox version available via Discover as Flatpak or use Chrome that Valve has an easy installer for either via Discover or through the Non-Steam section in Gaming Mode.
I've reached out to Valve directly about this, will update if / when they reply.
Need help installing software on the Steam Deck? See our guide example video below:
Hat tip to Quinn.