Accelerated Graphics for Embedded Systems
What is it ?
We have heard a lot about graphics support in embedded systems, it provides a medium of interface to interact with underlying hardware, well this is partially true! Then what does graphics actually do ? Graphics provides a way to display basic Graphical User Interface (GUI). The embedded industry has evolved starting from liquid crystal displays to full-color LCD and now E-Ink displays to save some power as and when needed. Graphics can be processed at either software level or hardware level, as for software graphics support the hardware does not participate in processing of data and hence one can observe lag and delay in the output displayed on the monitor or LCD. To overcome the deficiencies found in the software rendering, one has to move to a hardware that comprises of a dedicated Graphics Processing Unit (GPU) to acheive faster frame rate support and different surface overlays for most efficient output devices.
Its the most recent embedded solutions that incorporate on-board graphics support in the SOC unlike older generation that provided graphics support through external cards on high speed bus expansions.
Why is it needed ?
The primary fact remains same in the embedded world, that is high speed processing with lowest latency and low consumption of power. To achieve what is in demand, we have to choose little or reasonably expensive solutions that provides dedicated graphical processing unit and the software will be quite different than the traditional systems in-order to drive this unit. We need GPU for graphics data processing at hardware level and such systems provide high speed output results. Modern embedded and handheld devices usually are designed to include at least one display unit of various sizes and corresponding display resolutions and GPU’s for these output devices should processes the data almost instantaneously and must be capable enough to drive the display device with ease. The real time embedded machines like ATM, Vending, smart meters, medical instruments and security systems need instant output with zero lag and delays, that is where the embedded machines with dedicated GPUs will play a major role and stand-out as primary choice for such critical gadgets.
How it can be used ?
The basic graphics system architecture consists of Geometric transformations, lighting calculations, rasterization and post-rasterization that is finally delivered to display screens. The precise way to define the key components of this system are the graphics unit, Linux device driver and OpenGL framework libraries that primarily works on DRM or KMS based systems in recent days, or traditionally framebuffers were used to directly interact with the device and just need the driver along with the OpenGL-Framework libs for rendering the graphics data. The framebuffers provide trivial APIs that are capable enough to configure the graphics mode on the main output for the graphics chip. Framebuffers usually exposes the device file /dev/fb(n) which user-space performs the reads/writes and ioctls on.
The DRM (Direct Rendering Manager) drivers were created to support the DRI (Direct rendering infrastructure) and pass them to GEM or TTM code, which passes the data/instruction to the graphics card. DRI can be used to accelerate all sorts of graphics but in general it is used for opengl.
EGL is an API for managing windows and buffers and is assumed as a helper for OpenGL framework. An OpenGL implementation is mainly responsible for generating appropriate blocks of data for a card to process and can be associated with card specific instructions. OpenGL needs required buffers to be passed in as parameters. Therefore the EGL was introduced as standard API which may have different implementation for different windowing system. The DRI is mainly responsible for provision of hardware acceleration for Mesa implementation of OpenGL. Also DRI is used to provide OpenGL acceleration on framebuffer console without any display server running.
The latest graphics stack on Linux is much more complex then how is interpreted, a frame-buffer is most simple one that is used in embedded systems.
What is coming next ?
The framebuffer is ancient method of displaying content on the monitor or a console, but there are other popular methods that provide better features and implementations to acheive the same in a mature way like DRM which provides a emulated fbdev in order to keep the legacy applications running without breaking the workflow. The KMS is also another widely used solution for accelerated graphics apart from DRM which extends a bit interms of usability such as KMS (Kernel Mode Setting) and UMS (User-space Mode Setting) to provide controls in either kernel space or user space. The following are key factors before choosing a display server and underlying subsystem:
1. Community maintanance
2. Individual control of display units
3. Feature set
4. Complexity
5. User-space graphics support
The FBDev is less actively maintained and may be deprecated in coming days, due to the fact that it provides in-adequate features, So most of the latest developers are now moving towards DRM/KMS due to the fact that it provides features like overlays and hw cursors. Modern display servers like X11 and Wayland have been working quite well with DRM and preferred with choice but weston as compositor still needs some polishing and updates to support GUI frameworks like GTK3 without many tweaks and hacks.
The OpenGL for embedded systems has come a long way and is been used as trusted graphics rendering API for 2D and much widely used 3D graphics, it is designed for smartphones, tablets and video game consoles which believe in high performance and low power consumption that is leading towards advanced graphics like Vulkan.
