Support us on Patreon to keep GamingOnLinux alive. This ensures all of our main content remains free for everyone. Just good, fresh content! Alternatively, you can donate through PayPal. You can also buy games using our partner links for GOG and Humble Store.
We do often include affiliate links to earn us some pennies. See more here.

An Update On The Open Source Xoreos Project For Neverwinter Nights 2

By - | Views: 20,954
xoreos is a FLOSS project aiming to reimplement BioWare's Aurora engine (and derivatives), covering their games starting with Neverwinter Nights and potentially up to Dragon Age II. This post gives a short update on the current progress.

Note: This is a cross-post of a news item on the xoreos website.


During the last 2 weeks, I implemented basic support for Neverwinter Nights 2 areas and campaigns/modules. In particular:

  • Reading the list of modules and the starting module in a campaign
  • Reading module areas and starting conditions
  • Loading general area information like music
  • Loading area geometry and objects
    • Tiles
    • Terrain
    • Doors
    • Placeables
    • Environmental objects
    • Creatures

  • Flying through the areas in a "spectator mode"
  • Highlighting potentially useable objects
  • Hooking up a debug console
    • listcampaign/loadcampaign commands
    • listmodules/loadmodule commands
    • listareas/gotoarea commands



As a result, areas in Neverwinter Nights 2 are now in a similarly supported state as areas in the two Knights of the Old Republic games.

image image

So, here's a few interesting things and/or issues with them.

Tiles and Metatiles

Like the first Neverwinter Nights game, areas in Neverwinter Nights 2 are (or can be, more about that later) made up of tiles. Meaning, a grid of models define the general structure: piece of indoors floor with walls, corners, windows and/or doors. Unlike the first game, though, there's also metatitles, which are bigger models spanning the space of several regular tiles. This is used, for example, for the piece of the cave where you wake up in the Mask of the Betrayer expansion.

xoreos correctly loads and displays both tiles and metatiles.

image

Terrain

Also unlike the first game, areas in the second game can also use terrain instead (or maybe even additionally to) tiles. The terrain is floor geometry unique to the area, and consists of patches of land and water, together with walk mesh definitions. Terrain is used very often in outdoor areas, to give those areas a more unique and less flat feeling.

xoreos correctly loads and displays the land and water patches, in a very rudimentary and stubby way.

image

Environmental Object

Neverwinter Nights 2 adds a new area object type: environmental objects. These seem to be quite similar to regular placeables, only that they are static and can't ever be moved. Apparently, their geometry is also baked into the area walk meshes.

As an extra weird bit, environmental objects store their position and orientation differently than all the other objects.

xoreos correctly loads and displays environmental objects in areas.

Trees

Neverwinter Nights 2 also adds new, non-static trees, courtesy of the SpeedTree middleware. Tree definitions seem to consist of just a few parameters and a random seed, the rest is done by the SpeedTree library.

xoreos does not load or display those trees yet. In fact, that will be a major task in the future. Luckily, trees aren't essential; them missing is not a showstopper.

Creature Models and Animations

xoreos also partially loads and displays the various creatures (animals, NPCs, monsters, ...) defined in the area files. There's one caveat, though: the different model parts (head, hair, gloves, boots) aren't actually attached to the body yet. They're loaded as separate models and those are already placed correctly for default character dimensions. Of course, this would fail with scaled models (which we do not yet support).

The underlying problem is this: where the character models in Neverwinter Nights and Knights of the Old Republic contain special nodes telling the engine where to place the different body parts, Neverwinter Nights 2 has no such thing. Instead, each skin geometry vertex has a "bone index" attribute that, I assume, are indices into the Granny 3D animation files.

Which means: to properly attach the body parts, and to animate the models, we first need to figure those files out.

image

Tile and Object Tinting

A lot of objects in Neverwinter Nights 2 areas use tint maps. These are special textures with intensity values in their color channels. When an object is loaded from the area definition, those also contain 3 color values, and those color values are combined with the intensity values in the tint map to create a unique tinting of the object. This gives you the ability to create blue-ish caskets with yellow-ish metal rings, purple carpets, green vases and the like, without having to manually create textures for each color combination.

Moreover, tiles also use this tinting.

Currently, xoreos can evaluate and use those tinting, but entirely in software. For each object, an uncompressed RGBA texture is created, and the color values are mixed on loading. Of course, this has certain drawbacks:

First of all, it's slow. But more damning, it takes up a lot of extra memory, both on the GPU and in the system's RAM. As a ballpark, a common large-ish area creates 1GB of extra textures; even more when there's a lot of objects around.

This is, of course, unacceptable. In the future, this tinting should most probably be done with shaders on the GPU.

image image

image image

Terrain Tinting

There is another tinting mechanism: terrain geometry uses a similar approach, only more extended. Instead of just 3 mix channels, there's 6 (using 2 mixing textures) and instead of just mixing colors, textures are mixing. I.e.: a terrain patch can specify up to 6 texture that will be mixed according to the color values in 2 mix textures. The result is a layered texture that removes all visible texture tiling effects.

Additionally, each patch also has a color value to modify the texture, increasing the flexibility again.

At the moment, xoreos does none of this, for obvious reasons. Instead, the singular color value is applied to each patch, creating a more flat look.

Doors and Chests

Currently, xoreos does not display most doors and chests entirely correctly: doors are missing their, well, door part (only the door frame and knob are visible) and chests are missing their lid. I have no idea if that's connected to the animation system, tinting, or what.

image

All in all, I'm pretty happy with the progress so far. :) Article taken from GamingOnLinux.com.
0 Likes
About the author -
Geek. Atheist+. Leftist. Metal-Head. Discordian. Lefty.
ScummVM dev, xoreos lead.
Free software zealot.
See more from me
The comments on this article are closed.
20 comments
Page: 1/2»
  Go to:

Anorelsan Feb 1, 2015
Impressive work! :D
chrisq Feb 1, 2015
Please keep going, I'd love to see these games native in linux!
coryrj19951 Feb 1, 2015
Keep up the good work! Can't wait for it to be finished so I can play NWN2, and the other Aurora engine games natively!
commodore256 Feb 1, 2015
Good job, Sven!
D34VA_ Feb 2, 2015
I'm eagerly awaiting the day I can run KOTOR/KOTOR2 native. Here's to Dr. McCoy and Xoreos!
xeranas Feb 2, 2015
I wish that there would be more explanations behind these engines reimplementations. How end-user will benefit from it? Do we see 'Linux' icon for mentioned games on Steam? Or we need do some 'hacking' in order to run games.
DrMcCoy Feb 2, 2015
Thanks for the kind words, everyone. :)

But keep in mind that the goal to play the games natively on GNU/Linux is still a long way off. Especially Dragon Age is still missing most of the basics. That's why I really need more people to join my efforts!

Quoting: xeranasHow end-user will benefit from it?

There's several way users will benefit from this:
  • The games will continue to run on modern hardware and operating systems. In many cases, games are pretty awful at compatibility. For xoreos' target, this is currently less of an issue than for games supported by, for example, ScummVM. But even there, there's problems: one example is the Mac OS X version of Neverwinter Nights. It is a PPC binary only; it won't work on modern Intel Macs (Rosetta stopped working with 10.7) [1]

  • The games will also work on other operating systems, like GNU/Linux and Mac OS X, without the need for Wine. This is a potential performance increase

  • There's the potentially for support on mobile platforms

  • For some of the games, there's still a vibrant modding community. This will open up more modding opportunities, giving the games a new life, new adventures, etc.


Sure, it is unlikely that original rights holders will acknowledge this. But never say never: just look at the recent official rereleases of LucasArts games using ScummVM.

And yes, getting them to work might still be a bit hacky, and more difficult than just pressing "Play". But again, look at ScummVM and ResidualVM, those already make it more easier than for example Wine and DOSBox. It is a future goal of mine to make playing the games with xoreos as easy and unhacky as possible.

Still, I admit that this project is also a fun hobby, learning excercise and puzzle for me. :)

[1] See also the recent push of the Internet Archive to archive old DOS games for a tangential reason
Sabun Feb 2, 2015
That is amazing work. May I ask how you procured the knowledge to reach this level? Were you a computer science student?

To be able to re-implement an engine like this, with how far you've already reached, it really baffles my mind how you grasp it all. Keep up the good work :)
DrMcCoy Feb 2, 2015
Well, I've been programming since I was 8. I started with GW-BASIC on Novell DOS, in the early 90s. I learned Visual Basic, Delphi, Turbo Pascal, C and C++ later, and I had some contact with assembly (mostly Z80, for the TI-83+ calculator).

As for reverse-engineering / reimplementing, I learned a lot when I joined the ScummVM team in January 2006 and took over the engine for Coktel Vision games. I was giving the freeware version of IDA, the partially commented IDA database of Gobliiins, the ScummVM code and the task to reverse engineer Gobliins 2 to add support to ScummVM.

I of course asked a lot of questions, but many thing you just have to figure out for yourself. So I grabbed a copy of the "The Art of Assembly Language" book to get some understanding of the DOS specific stuff, and dug into the disassemblies of Gobliiins and Gobliins 2. It took me half a year until I had some understanding of what Gobliins 2 did, and then lots of fiddling and mistakes. And I was a student then, with a lot of time to waste. Preliminary support was announced in December 2006.

So yeah, lots of privilege due to early contact with programming, and a lot of banging my head against the wall until stuff makes sense. :)
DrMcCoy Feb 2, 2015
Oh, and of course, for xoreos, I can build upon what other people in the modding communities have already figured out . Really, this is essential, having open communities where information flows freely. I am very grateful for those people.

Unfortunately, not all people in the modding communities see it this way and guard their knowledge with their life. This makes me sad.
While you're here, please consider supporting GamingOnLinux on:

Reward Tiers: Patreon. Plain Donations: PayPal.

This ensures all of our main content remains totally free for everyone! Patreon supporters can also remove all adverts and sponsors! Supporting us helps bring good, fresh content. Without your continued support, we simply could not continue!

You can find even more ways to support us on this dedicated page any time. If you already are, thank you!
The comments on this article are closed.