Dev:Cairo render migration

This page is to describe the Cairo render migration process from the actual Synfig software render.

Rendering plugin system

• A plugin system for rendering methods (can be extended to support rendering methods other than Cairo, such as OpenGL)
  • In target.h, there is an option to specify the rendering method.
    • SOFTWARE is the old renderer, OPENGL is unsupported but kept for consistency with Uiomae's branch, and CAIRO is the one nikitakit added
  • Uiomae made a number of changes to the individual targets to support new way of receiving data
    • Each target is now annotated with the kind of color data that it accepts. (See the enum PixelFormat)
    • A few targets may have been mis-annotated, in which case the resulting images will turn out wierd (or segfault the renderer). The PNG target should work properly, though. If a target is mis-annotated then it would fall back to software render
    • The render-to-screen targets defined in Synfig Studio do not support these methods, so they cannot currently render using Cairo.

Cairo rendering system

How the cairo rendering system works

• Nikitakit added a virtual method cairo_render to the Layer class. It receives a Cairo rendering context (cr) in addition to the Synfig context.
• The default implementation in the layer.cpp simply renders the layer using the software renderer, copies the output onto a cairo image surface, and then paints it onto the context.
  • This involves one copy (with a color conversion, see below) and one paint operation - not very efficient!
• Each layer can override the method cairo_render to specify an actual cairo renderer.
• We should start the cairo conversion by implementing cairo_render for a few common layers, and figuring out how to do painting and blend methods properly.
  • Cairo does not support as many blend methods as Synfig, but it does have quite a few (see [1])

    Note

    cairomm does not provide wrappers for all of them. You may need to get the constants from the C API.

Cairo image surfaces, and color conversions

• Cairo has a number of underlying "surfaces" that redirect the paint operation to different targets. There are PNG, SVG, OpenGL, direct-to-screen, and a couple others.
• Nikitakit use the image surfaces because of the need to perform operations with color buffers.
  • The image surfaces have a variety of data formats (see [2])

Data formats

The only one among them, however, that includes both alpha and RGB data channels is Cairo::FORMAT_ARGB32.

This format uses 8 bits per channel. One thing to watch out for is that professional photographers tend to want higher bit depth, especially for print. However, this may not apply to on-screen animation.

But this cairo data format is NOT the data format that is used by the default Synfig renderer! This is probably the most confusing part of the whole rendering framework.

There are two key differences between the formats:

Color order

ARGB32 is (8 bits of alpha value, 8 bits of red, 8 bits of green, 8 bits of blue).

IIRC Synfig's renderer is RGBA32 (the alpha channel is at the end, not at the beginning)

Stride concept in Cairo

Synfig packs 32-bit pixel values back-to-back in a memory buffer: immediately following the last pixel in one row is the first pixel in the next row.

That is, the pixel in row `r' and column `c' can be found at memory location (width * 32bits * r) + (32bits * c)

but Cairo sometimes pads zeros between rows in order to align the data for better hardware acceleration. So in cairo, the pixel in row `r' and column `c' can be found at memory location (stride * r) + (32bits * c), where stride >= width * 32bits. Therefore the default implementation of cairo_render must perform a color conversion between the two data formats. This definitely has a negative performance impact.

In addition, each of the Synfig render targets expects color data in a specific format, which is NOT the same as Synfig's format, NOR is it the same as cairo's data format. Here Nikitakit uses Uiomae's work on defining which color format a target accepts, and convert the cairo data format to that format

Ideally, we would want to take advantage of cairo's native render-to-png, render-to-svg, and render-to-screen surfaces instead of manually calling libpng, etc. Therefore we should consider having each target return a cairo surface, draw the image onto the cairo surface, and then pass the finalized color surface to the target

However, that would be a completely different model from the software renderer. We would need a good way to integrate them, or else scrap our custom rendering framework and use Cairo's image surfaces, even in our software renderer (as a final step).

Implementing process

1. Implement cairo_render for at least a few key layers (namely: Paste canvas, and either rectangle or Bline). Figure out how to do color composition correctly
2. Decide whether to keep the current color conversion system, or figure out how to integrate cairo's native surfaces
   • Either integrate cairo image surfaces into all targets, including the ones in Synfig Studio
   • Or specify the color conversion needed for the Synfig Studio renderer.

     Note

     both require modification/reimplementation of tiling, since all renderers in core are not tile renderers)

3. Make sure all render targets work properly
4. Implement the rest of the layers in Cairo

   Note

   Since this is mostly independent, parallelized work, it's possible to have a code sprint where different people take care of different layers

5. Figure out what parts, if any, of the custom rendering framework we want to scrap in favor of using library code
6. Optimize: for example, don't re-render a layer if it doesn't change from one frame to the next