3.9.8. SGX Build Guide

The graphics stack on SGX has been frozen. The last major change occured in release 09.01. This will capture the build process and some of the caviots of this legacy driver.

This driver has been reworked as part of the kernel 6.1 support effort to mimic the build and structure of rogue. The stack consists of 3 main components with varying classifications:

  • The kernel module (KM) (Open Source, but out-of-tree)

  • The GLES 2.0 implementation (UM) (Closed Source, binary release)

  • The mesa shim (Open Source, but out-of-tree)

All 3 of these components are required for GLES support.

3.9.8.1. SGX Kernel Module

The kernel module is responsible for:

  • Memory and power management

  • Monitoring device state

  • Orchestrating hardware recoveries

  • Creating an interface for the userspace server

The source for the component is available at:

3.9.8.1.1. Building the SGX Kernel Module

The latest branch contains the most bugfixes and has been adapted for the widest general compatibility in modern applications. This is 1.17.4948957/mesa/k6.1, and despite being tagged for kernel 6.1 it is also interoperable with kernel 6.6.

The INSTALL file gives a brief overview of how to build and install the module. This gives you the common variables, but it does leave out some of the more important ones for the build. The following variables are key and must match with the values used in the UM release:

  • BUILD – Options: release or debug

  • PVR_BUILD_DIR – Options: <platform>_linux, for example ti572x_linux

  • WINDOW_SYSTEM – Options: lws-generic, xorg, wayland, …

These variables must be set for the build. To check for updated build variables please see the EXTRA_OEMAKE variable in the ti-sgx-ddk-km recipe in meta-ti. This will always contain the latest required variables to compile the driver.

As previously mentioned these build options must match with the corresponding UM build options. Currently we only release binaries with BUILD=release and WINDOW_SYSTEM=lws-generic. The UM section will go previous exceptions to this.

3.9.8.2. SGX Userspace Components

The proprietary userspace components are required for all interactions with the GPU. It contains the firmware, GLES implementation, and the primary control server. Unlike rogue, the firmware is packaged as part of the control sever library.

The binary release for this component is available at:

3.9.8.2.1. Installing the SGX Userspace Components

The repository contains binaries for each platform in release specific branches. The newest release, which matches with the above mentioned kernel module branch is 1.17.4948957/mesa/glibc-2.35. This branch deviates from the existing naming pattern to indicate that unlike previous releases it required a new version of mesa and was linked against glibc-2.35.

In the repository the following structure can be seen:

.
├── LICENSE
├── Makefile
├── README.md
└── targetfs
    ├── common
    │   ├── 50-pvrsrvctl.rules.template
    │   ├── etc
    │   └── pvrsrvctl.service.template
    ├── ti335x_linux
    │   └── lws-generic
    ├── ti343x_linux
    │   └── lws-generic
    ├── ti437x_linux
    │   └── lws-generic
    ├── ti443x_linux
    │   └── lws-generic
    ├── ti572x_linux
    │   └── lws-generic
    └── ti654x_linux
        └── lws-generic

Under the targetfs directory there is a subdirectory for each PVR_BUILD_DIR build variable. Under that directory is a subdirectory corresponding to the WINDOW_SYSTEM build variable. Finally, under that directory is a subdirectory corresponding to the BUILD build variable.

This is reflected in the Makefile as:

PRODUCTDIR := ./targetfs/${TARGET_PRODUCT}/${WINDOW_SYSTEM}/${BUILD}

The Makefile simply unpacks this directory structure and installs the corresponding files into DESTDIR in the install step. Do not worry about the clean step, as this is used for development.

Unlike rogue, there is an additional common directory that contains startup script templates. These templates are automatically modified by the Makefile as part of the install step.

These are not strictly required, but you will need to manually start the device with pvrsrvctl if you choose not to use these templates. Please note that this binary is, and always has been brittle. It likely will not realize what kernel you are running so you should specify --no-module when interacting with it.

3.9.8.3. SGX Mesa Components

Mesa, at this point in time, is a collection of GFX tools and utilities for setting up and interacting with rendering contexts. It contains everything from a DRI “megadriver” to full GLES/GL implementations. If you’re interested in learning GFX under Linux it’s worth familiarizing yourself with everything else it provides.

For us, the important part is that DRI “megadriver.” This is the mechanism used to determine what GLES / GL implementation is picked when you bind one of the previously mentioned API to a EGL context. This is also where things get tricky.

Historically there has been some issues with embedded GFX because, unlike your standard PC GPU, we tend to mix and match actual Graphics Processing Units and Display Controllers. The megadriver uses the display device name to coordinate between API implementations. As such, we need a shim to act as a DRI driver and coordinate the link with the SGX GLES implementation.

This shim, currently, is provided in the form of a Gallium Frontend. This is the main reason for the fork and the 60 odd patches we carry at the following repo:

There are also other nice-to-have features there such as additional pixel formats, minor fixups, and a few performance tweaks IMG have picked up over the years, but the main reason we need it is for that shim.

3.9.8.3.1. Building the SGX Mesa Components

We recommend following the Mesa build guide for general options. Right now the mesa components use a standard interface that allows you to pick any powervr/* branch equal to or greater than 22.3.5.

The only necessary build options are:

  • -Dgallium-drivers=sgx – This is a comma separated list, just make sure sgx is present in it.

  • -Dgallium-sgx-alias= – This should match the display controller you want to bind to. This can either be tilcdc, tidss, or omapdrm depending on the device.

This will produce 3 important files relevant to the shim mechanism we discussed earlier:

  • sgx_dri.so – Main DRI interface

  • pvr_dri.so – GPU interface that points back to sgx_dri.so. This is required in the case an application attempts to interact with the GPU directly because the DRI device is still registered with the name pvr.

  • <display>_dri.so – Display controller interface that points back at sgx_dri.so. Will be named after the value specified with -Dgallium-sgx-alias.

3.9.8.4. Using the SGX Stack

Assuming you’re using the SDK or you’ve built and installed the above correctly, you should see a message similar to the following in dmesg after the kernel module is loaded:

[   17.344567] [drm] Initialized pvr 1.17.4948957 20110701 for 56000000.gpu on minor 1

If the module loads but does not detect the device, make sure your device tree has defined the node properly, corresponding to one of the values of powervr_id_table in the services4/srvkm/env/linux/module.c.

Upon starting the userspace daemon you should see the following message:

[   29.277564] PVR_K: UM DDK-(4948957) and KM DDK-(4948957) match. [ OK ]

You should now be able to issue a simple test. We recommend glmark2-es2-drm. You should see the following indicating you are using the correct driver:

root@am335x-evm:~# glmark2-es2-drm
MESA: info: Loaded libpvr_dri_support.so
=======================================================
    glmark2 2021.12
=======================================================
    OpenGL Information
    GL_VENDOR:     Imagination Technologies
    GL_RENDERER:   PowerVR SGX 530
    GL_VERSION:    OpenGL ES 2.0 build 1.17@4948957
=======================================================

The GL_VENDOR should report Imagination Technologies with the renderer corresponding to the graphics processor in that device.