Building Mesa from source

From GamingOnLinux.com Linux Games Wiki
Revision as of 01:15, 29 December 2017 by Shmerl (Talk | contribs) (64-bit)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

There are times when Mesa introduces some new feature that didn't yet make it into your distro. Some games can become playable with that change, or it can be simply a performance optimization that speeds up already working games - either way, you might be interested in running such latest development version of Mesa for various reasons. At the same time you don't want to mess up your system with unstable graphics stack. This guide will explain how to use such Mesa for playing games while keeping your system Mesa intact.

Note: This process has distro specific parts. You'd need to adapt it, to use with another distro. Feel free to expand this guide with a section which describes how you did it on your distro of choice.

To automate all the steps below, I made a script which I use form inside the VM (for cleaner separation). But if you want to understand the idea better, read on.

Building Mesa on Debian testing

You probably want to build both 64-bit and 32-bit Mesa libraries, since the later are needed for 32-bit applications, including many older games, especially those run through Wine.

64-bit

First install various development tools and dependencies. Note: keep track of any installed / removed packages (copy them from apt-get lists for NEW and save in some file). You don't want to clutter, and let alone mess up your system. After you finished building all you need (both 64-bit and 32-bit), it's a good idea to return the system to the previous state. I.e. remove all the packages that got installed. To keep things tidy, you can do this whole thing inside a VM, which you can simply discard once you are done. This will avoid any risk of messing up your main system installation.

sudo apt-get install build-essential git
sudo apt-get build-dep mesa

Make / select yourself some directory (let's say $HOME/build) and start working from there. Clone the repo and prepare to configure it. This guide assumes using latest master branch. If you want to build a specific version instead of using master, you'd need to switch to it using git once it's cloned

build_dir="${HOME}/build/mesa"
llvm_ver="5.0"
cpuarch="znver1" # I set it for Zen, but you can use "native" or anything else you like.
build_threads=$(nproc)
arch_dir["32"]="x86"
arch_dir["64"]="x86_64"
arch_vk["32"]="i686"
arch_vk["64"]="x86_64"

mkdir -p "$build_dir"
cd $(dirname "$build_dir")

git clone git://anongit.freedesktop.org/mesa/mesa $(basename "$build_dir")
cd "$build_dir"

Now, you need to configure Mesa. There are multitude of ways to do it. Since I was building for Debian, I checked how the official distro package is configured. An easy way to do it is to first open Debian tracker page for Mesa. From there find the link "buildd: logs". Once on that page, find the row with amd64 architecture, and click on "Installed" which will open a full Debian build log for the package (you can also select Debian type, such as sid for unstable here, or anything else). Once in the log, find string /configure. That would take you right away to configuration part. Copy that string and analyze it if you want.

I adjusted it a bit. For example I planned to install resulted libraries in /opt/mesa-master. So I changed --prefix=/usr to --prefix=/opt/mesa-master. And to simplify things, --libdir=\${prefix}/lib/x86_64-linux-gnu --libexecdir=\${prefix}/lib/x86_64-linux-gnu to --libdir=\${prefix}/x86_64 --libexecdir=\${prefix}/x86_64 (note the explicit directory for x86_64, that's because afterwards, same process will be repeated for 32-bit build).

Also, you can remove some drivers from the list which you don't need to speed up compilation. That's up to you what to pick. I changed: --with-dri-drivers= nouveau i915 i965 r200 radeon to --with-dri-drivers= i965 and --with-gallium-drivers= nouveau svga virgl r600 r300 radeonsi swrast to --with-gallium-drivers= radeonsi. I also changed optimization to -O3. That's it, run the resulting configure command:

./configure --build=x86_64-linux-gnu --prefix=${mesa_dir} --includedir=\${prefix}/include --mandir=\${prefix}/share/man --infodir=\${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --disable-silent-rules --libdir=\${prefix}/${arch_dir["64"]} --libexecdir=\${prefix}/${arch_dir["64"]} --disable-maintainer-mode --disable-dependency-tracking --enable-dri "--with-dri-drivers= i965" --with-dri-driverdir=/usr/lib/x86_64-linux-gnu/dri --with-dri-searchpath=/usr/lib/x86_64-linux-gnu/dri:\\\$\${ORIGIN}/dri:/usr/lib/dri "--with-vulkan-drivers= intel radeon" --enable-libglvnd --enable-osmesa --enable-glx-tls --enable-shared-glapi --enable-texture-float --disable-xvmc --disable-omx-bellagio --enable-driglx-direct --enable-gbm --enable-dri3 "--with-platforms=x11,surfaceless wayland drm" --enable-xa --enable-llvm --enable-opencl --enable-opencl-icd ac_cv_path_LLVM_CONFIG=llvm-config-${llvm_ver} --enable-vdpau --enable-va --enable-gallium-extra-hud --enable-lmsensors "--with-gallium-drivers= radeonsi" --disable-gles1 --enable-gles2 "CFLAGS=-g -O3 -fdebug-prefix-map=${HOME}/build=. -march=${cpuarch} -fstack-protector-strong -Wformat -Werror=format-security -Wall" "CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2" "CXXFLAGS=-g -O3 -fdebug-prefix-map=${HOME}/build=. -march=${cpuarch} -fstack-protector-strong -Wformat -Werror=format-security -Wall" "FCFLAGS=-g -O3 -fdebug-prefix-map=${HOME}/build=. -fstack-protector-strong" "FFLAGS=-g -O3 -fdebug-prefix-map=${HOME}/build=. -march=${cpuarch} -fstack-protector-strong" "GCJFLAGS=-g -O3 -fdebug-prefix-map=${HOME}/build=. -fstack-protector-strong" LDFLAGS=-Wl,-z,relro "OBJCFLAGS=-g -O3 -fdebug-prefix-map=${HOME}/build=. -march=${cpuarch} -fstack-protector-strong -Wformat -Werror=format-security" "OBJCXXFLAGS=-g -O3 -fdebug-prefix-map=${HOME}/build=. -march=${cpuarch} -fstack-protector-strong -Wformat -Werror=format-security"

If all is correct, it will succeed and Mesa is ready for building:

make -j${build_threads}

After it's all built, publish the result to your target directory (in my case /opt/mesa-master):

mv -v x86_64/gallium/* x86_64/
rmdir x86_64/gallium

sudo mkdir /opt/mesa-master
sudo mv -v x86_64 /opt/mesa-master/
sudo cp src/intel/vulkan/intel_icd.x86_64.json /opt/mesa-master/x86_64/
sudo cp src/amd/vulkan/radeon_icd.x86_64.json /opt/mesa-master/x86_64/
sudo chown -R root:root /opt/mesa-master

Now you are ready to test your freshly built Mesa.

OpenGL:

LD_LIBRARY_PATH=/opt/mesa-master/x86_64:$LD_LIBRARY_PATH LIBGL_DRIVERS_PATH=/opt/mesa-master/x86_64 EGL_DRIVERS_PATH=/opt/mesa-master/x86_64 glxinfo | grep string

You'll get something like:

...
OpenGL renderer string: AMD Radeon (TM) RX 480 Graphics (POLARIS10 / DRM 3.19.0 / 4.14.0-2-amd64, LLVM 5.0.1)
OpenGL core profile version string: 4.5 (Core Profile) Mesa 17.4.0-devel (git-420627e6e7)
OpenGL core profile shading language version string: 4.50
...

Vulkan:

LD_LIBRARY_PATH=/opt/mesa-master/x86_64:$LD_LIBRARY_PATH VK_ICD_FILENAMES=/opt/mesa-master/x86_64/radeon_icd.x86_64.json vulkaninfo | more
===========
VULKAN INFO
===========

Vulkan API Version: 1.0.61

INFO: [loader] Code 0 : Found ICD manifest file /opt/mesa-master/x86_64/radeon_icd.x86_64.json, version "1.0.0"

Instance Extensions:
====================
Instance Extensions     count = 10
        VK_KHR_external_fence_capabilities  : extension revision  1
        VK_KHR_external_memory_capabilities : extension revision  1
        VK_KHR_external_semaphore_capabilities: extension revision  1
        VK_KHR_get_physical_device_properties2: extension revision  1
        VK_KHR_get_surface_capabilities2    : extension revision  1
        VK_KHR_surface                      : extension revision 25
        VK_KHR_wayland_surface              : extension revision  6
        VK_KHR_xcb_surface                  : extension revision  6
        VK_KHR_xlib_surface                 : extension revision  6
        VK_EXT_debug_report                 : extension revision  8
WARNING: radv is not a conformant vulkan implementation, testing use only.
...
  • Note the environment variables needed to point the application to the custom driver. You'd need to set those for any game you want to run with this Mesa.
  • In the Vulkan example I assume radv (AMD) usage. If you are using Intel, change VK_ICD_FILENAMES=/opt/mesa-master/x86_64/radeon_icd.x86_64.json to VK_ICD_FILENAMES=/opt/mesa-master/x86_64/intel_icd.x86_64.json.
  • To run some Vulkan demos (to test if the driver even works), you can use vulkan-smoketest or builds from here. For example Viking Village demo has both 32-bit and 64-bit versions, so you can test both builds of the driver with it.

32-bit

After you are done with 64-bit, clean your build:

make clean
autoreconf -vfi

Using the above trick with sudo apt-get build-dep mesa wouldn't work for 32-bit (you could in theory try adding -a i386 to that command), because not all needed packages support multiarch in Debian yet. Configuration and building steps will fail however if all needed dependencies are not installed. You can figure them out one by one (this stuff can change in the future), by simply not installing anything first and running configure and then make. Analyzing errors you can figure out what packages are needed. This is a bit tedious, here is what I had to do:

sudo apt-get purge llvm-${llvm_ver}-dev libclang-${llvm_ver}-dev libsensors4-dev
sudo apt-get autoremove --purge

sudo apt-get install llvm-${llvm_ver}-dev:i386 libclang-${llvm_ver}-dev:i386 gcc-multilib g++-multilib libdrm-dev:i386 libexpat1-dev:i386 libxcb-dri3-dev:i386 libxcb-present-dev:i386 libxshmfence-dev:i386 libxext-dev:i386 libxdamage-dev:i386 libx11-xcb-dev:i386 libxcb-glx0-dev:i386 libxcb-dri2-0-dev:i386 libxxf86vm-dev:i386 libwayland-dev:i386 libsensors4-dev:i386 libelf-dev:i386 zlib1g-dev:i386 libglvnd-core-dev:i386

Note, that some packages are not multilib compatible yet, so they'll try to remove their 64-bit counterparts. That's why to resolve some conflicts, you need to explicitly purge them first, like above. Here are some examples:

  • libsensors4-dev:i386 doesn't support multilib - removes libsensors4-dev (64-bit).
  • llvm-${llvm_ver}-dev:i386 doesn't support multilib - removes llvm-${llvm_ver} llvm-${llvm_ver}-dev llvm-${llvm_ver}-runtime (64-bit).
  • libclang-${llvm_ver}-dev:i386 doesn't support multilib - removes libclang-${llvm_ver}-dev libclang-common-${llvm_ver}-dev (64-bit).

Now to configure Mesa, start again with the build log, but for the 32-bit version of the package. Find i386 architecture on the buildd page, and open "Installed" again. Find /configure. This string however is used to build on 32-bit system proper. We need to adjust it to cross compile from 64-bit to 32-bit. The key parts here are adding -m32 to compiler invocations, and setting --build to 64-bit to trigger cross compilation. I.e. that's the most relevant part: CC="gcc -m32" CXX="g++ -m32" --build=x86_64-linux-gnu --host=i686-pc-linux-gnu That's what I came up with:

./configure CC="gcc -m32" CXX="g++ -m32" --build=x86_64-linux-gnu --host=i686-pc-linux-gnu --prefix=${mesa_dir} --includedir=\${prefix}/include --mandir=\${prefix}/share/man --infodir=\${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --disable-silent-rules --libdir=\${prefix}/${arch_dir["32"]} --libexecdir=\${prefix}/${arch_dir["32"]} --disable-maintainer-mode --disable-dependency-tracking --enable-dri "--with-dri-drivers= i965" --with-dri-driverdir=/usr/lib/i386-linux-gnu/dri --with-dri-searchpath=/usr/lib/i386-linux-gnu/dri:\\\$\${ORIGIN}/dri:/usr/lib/dri "--with-vulkan-drivers= intel radeon" --enable-libglvnd --enable-osmesa --enable-glx-tls --enable-shared-glapi --enable-texture-float --disable-xvmc --disable-omx-bellagio --enable-driglx-direct --enable-gbm --enable-dri3 "--with-platforms=x11,surfaceless wayland drm" --enable-xa --enable-llvm --enable-opencl --enable-opencl-icd ac_cv_path_LLVM_CONFIG=llvm-config-${llvm_ver} --enable-vdpau --enable-va --enable-gallium-extra-hud --enable-lmsensors "--with-gallium-drivers= radeonsi" --disable-gles1 --enable-gles2 "CFLAGS=-g -O3 -march=${cpuarch} -fstack-protector-strong -Wformat -Werror=format-security -Wall" "CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2" "CXXFLAGS=-g -O3 -march=${cpuarch} -fstack-protector-strong -Wformat -Werror=format-security -Wall" "FCFLAGS=-g -O3 -march=${cpuarch} -fstack-protector-strong" "FFLAGS=-g -O3 -fdebug-prefix-map=${HOME}/build=. -march=${cpuarch} -fstack-protector-strong" "GCJFLAGS=-g -O3 -fdebug-prefix-map=${HOME}/build=. -march=${cpuarch} -fstack-protector-strong" LDFLAGS=-Wl,-z,relro "OBJCFLAGS=-g -O3 -fdebug-prefix-map=${HOME}/build=. -fstack-protector-strong -Wformat -Werror=format-security" "OBJCXXFLAGS=-g -O3 -fdebug-prefix-map=${HOME}/build=. -march=${cpuarch} -fstack-protector-strong -Wformat -Werror=format-security"
make -j${build_threads}

Once succeeded, publish the result and clean up ownership:

mv -v x86/gallium/* x86/
rmdir x86/gallium

sudo mv -v x86 /opt/mesa-master/
sudo cp src/intel/vulkan/intel_icd.i686.json /opt/mesa-master/x86/
sudo cp src/amd/vulkan/radeon_icd.i686.json /opt/mesa-master/x86/
sudo chown -R root:root /opt/mesa-master

You should be ready to use it with some 32-bit applications.

OpenGL (Shadow Tactics: Blades of the Shogun, 32-bit GOG version):

LD_LIBRARY_PATH=/opt/mesa-master/x86:$LD_LIBRARY_PATH LIBGL_DRIVERS_PATH=/opt/mesa-master/x86 EGL_DRIVERS_PATH=/opt/mesa-master/x86 ./start.sh

Vulkan (Vikings Village demo 32-bit):

LD_LIBRARY_PATH=/opt/mesa-master/x86:$LD_LIBRARY_PATH VK_ICD_FILENAMES=/opt/mesa-master/x86/radeon_icd.i686_64.json ./vv.x86

You can run playonlinux like this with both custom Vulkan and OpenGL support (this will allow you running 32-bit games in Wine):

LD_LIBRARY_PATH=/opt/mesa-master/x86:$LD_LIBRARY_PATH LIBGL_DRIVERS_PATH=/opt/mesa-master/x86 EGL_DRIVERS_PATH=/opt/mesa-master/x86 VK_ICD_FILENAMES=/opt/mesa-master/x86/radeon_icd.i686_64.json playonlinux

For 64-bit ones, change accordingly.

Again, in the Vulkan examples I assume radv (AMD) usage. If you are using Intel, change VK_ICD_FILENAMES=/opt/mesa-master/x86/radeon_icd.i686.json to VK_ICD_FILENAMES=/opt/mesa-master/x86/intel_icd.i686.json.