This engine naturally isn't the new Serious Engine 4 that runs The Talos Principle and, as noted on the studio's website, isn't capable of rendering modern AAA graphics. Rather this is an old engine that they used for Serious Sam: The First Encounter and Serious Sam: The Second Encounter.
It's worth noting, however, that the engine in its current state is heavily tied to the Windows ecosystem and utilizes DirectX 8 and Visual Studio 2013 or 2015 is required for building the project. The assets of the games that run on this engine are also still proprietary. However, it wouldn't be the first time miracles have been made with opened up source code and I do expect someone will eventually port it to OpenGL.
You can read the official announcement here and find the source code on their Github.

I am a Finnish YouTuber and apparently responsible for the GOL Cast. I also do some programming every now and then, for example Castawesome livestreaming app is a creation of mine. My motto is "DON'T BUY BEFORE IT'S OUT".