Flutter (97): Drawing (IV) Compositing

Time: Column:Mobile & Frontend views:197

This section introduces flushCompositingBits().

Now, let’s review Flutter’s rendering pipeline:

void drawFrame() {
  pipelineOwner.flushLayout();
  pipelineOwner.flushCompositingBits();
  pipelineOwner.flushPaint();
  renderView.compositeFrame();
  ... // omitted for brevity
}

Among these, only flushCompositingBits() has not been introduced yet. To understand flushCompositingBits(), it’s essential to know what a Layer is and how the Layer tree is constructed. To make this easier to understand, let’s start with a demo.

1 CustomRotatedBox

We’ll implement a CustomRotatedBox, which will rotate its child element 90 degrees clockwise. To achieve this effect, we can directly use the canvas transformation feature. Here’s the core code:

class CustomRotatedBox extends SingleChildRenderObjectWidget {
  CustomRotatedBox({Key? key, Widget? child}) : super(key: key, child: child);

  @override
  RenderObject createRenderObject(BuildContext context) {
    return CustomRenderRotatedBox();
  }
}

class CustomRenderRotatedBox extends RenderBox
    with RenderObjectWithChildMixin<RenderBox> {

  @override
  void performLayout() {
    _paintTransform = null;
    if (child != null) {
      child!.layout(constraints, parentUsesSize: true);
      size = child!.size;
      // Calculate the rotation matrix based on the child’s size
      _paintTransform = Matrix4.identity()
        ..translate(size.width / 2.0, size.height / 2.0)
        ..rotateZ(math.pi / 2) // Rotate 90 degrees
        ..translate(-child!.size.width / 2.0, -child!.size.height / 2.0);
    } else {
      size = constraints.smallest;
    }
  }

  @override
  void paint(PaintingContext context, Offset offset) {
    if (child != null) {
      // Adjust the rotation matrix based on the offset
      final Matrix4 transform =
          Matrix4.translationValues(offset.dx, offset.dy, 0.0)
            ..multiply(_paintTransform!)
            ..translate(-offset.dx, -offset.dy);
      _paint(context, offset, transform);
    } else {
      //...
    }
  }

  void _paint(PaintingContext context, Offset offset, Matrix4 transform) {
    // To avoid interfering with other nodes drawn on the same layer, 
    // we need to call save before drawing the child and restore after
    // for display. Interested readers can refer to the Canvas API documentation.
    context.canvas
      ..save()
      ..transform(transform.storage);
    context.paintChild(child!, offset);
    context.canvas.restore();
  }
  ... // omitted unrelated code
}

Next, we’ll write a demo to test it:

class CustomRotatedBoxTest extends StatelessWidget {
  const CustomRotatedBoxTest({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Center(
      child: CustomRotatedBox(
        child: Text(
          "A",
          textScaleFactor: 5,
        ),
      ),
    );
  }
}

The result is shown in Figure , where "A" has been successfully rotated:

Flutter (97): Drawing (IV) Compositing

Now, let’s add a RepaintBoundary to CustomRotatedBox and try again:

@override
Widget build(BuildContext context) {
  return Center(
    child: CustomRotatedBox(
      child: RepaintBoundary( // Adding a RepaintBoundary
        child: Text(
          "A",
          textScaleFactor: 5,
        ),
      ),
    ),
  );
}

After running this, as shown in Figure , “A” stands upright again!

Flutter (97): Drawing (IV) Compositing

Let’s analyze the reason: based on the knowledge from the previous section, we can easily draw the Layer tree structure before and after adding the RepaintBoundary, as illustrated in Figure :

Flutter (97): Drawing (IV) Compositing

After adding the RepaintBoundary, the CustomRotatedBox still holds OffsetLayer1:

void _paint(PaintingContext context, Offset offset, Matrix4 transform) {
    context.canvas // This canvas corresponds to PictureLayer1
      ..save()
      ..transform(transform.storage);
    // The child node is a repaint boundary node and will be drawn on PictureLayer2 in the new OffsetLayer2
    context.paintChild(child!, offset);
    context.canvas.restore();
  }
  ... // omitted unrelated code
}

Clearly, the canvas performing the rotation transformation in CustomRotatedBox corresponds to PictureLayer1, while the drawing of Text("A") uses the canvas of PictureLayer2. They belong to different layers. We can see that the PictureLayers of parent and child are "separated," so CustomRotatedBox won’t affect Text("A"). So, how do we address this issue?

As mentioned in previous sections, many container components come with transformation effects. The container layer that has rotation transformation is the TransformLayer. Therefore, in CustomRotatedBox, we can do the following before drawing the child node:

  1. Create a TransformLayer (let's call it TransformLayer1) and add it to the Layer tree.

  2. Next, create a new PaintingContext that binds to TransformLayer1.

  3. The child node will be drawn using this new PaintingContext.

After these operations, the layers where descendant nodes are drawn will be children of the TransformLayer, allowing us to apply transformations to all child nodes collectively. Figure  shows the Layer tree structure before and after adding TransformLayer1.

Flutter (97): Drawing (IV) Compositing

This is essentially a layer compositing process: creating a new ContainerLayer, then passing this ContainerLayer to the child nodes, ensuring that the layers of descendant nodes belong to this ContainerLayer. Thus, applying transformations to this ContainerLayer will affect all its descendant nodes. To clarify, in this section, "layer compositing" specifically refers to this process.

In different contexts, "layer compositing" can refer to various things. For instance, Skia ultimately renders by composing multiple layers into a final bitmap; the process of layering all drawing results can also be called layer compositing.

Now, let’s look at the specific code implementation. Since layer composition is a standard process (the only difference is which type of ContainerLayer to use as the parent), the PaintingContext provides a pushLayer method to execute the composition. Here’s the implementation:

void pushLayer(ContainerLayer childLayer, PaintingContextCallback painter, Offset offset, { Rect? childPaintBounds }) {
  
  if (childLayer.hasChildren) {
    childLayer.removeAllChildren();
  }
  // The following two lines are standard operations for adding a new layer to the Layer tree.
  stopRecordingIfNeeded();
  appendLayer(childLayer);
  
  // Create a new childContext object with the new layer
  final PaintingContext childContext = 
    createChildContext(childLayer, childPaintBounds ?? estimatedBounds);
  // The painter is a callback for drawing the child node; we need to pass the new childContext object to it
  painter(childContext, offset);
  // After drawing, obtain the drawing product and save it to PictureLayer.picture
  childContext.stopRecordingIfNeeded();
}

So, we simply need to create a TransformLayer, specify the required rotation transformation, and call pushLayer directly:

// Create a handle holding TransformLayer.
final LayerHandle<TransformLayer> _transformLayer = LayerHandle<TransformLayer>();

void _paintWithNewLayer(PaintingContext context, Offset offset, Matrix4 transform) {
    // Create a TransformLayer and save it in the handle
    _transformLayer.layer = _transformLayer.layer ?? TransformLayer();
    _transformLayer.layer!.transform = transform;
    
    context.pushLayer(
      _transformLayer.layer!,
      _paintChild, // Callback for drawing the child node; the child will be drawn on the new layer
      offset,
      childPaintBounds: MatrixUtils.inverseTransformRect(
        transform,
        offset & size,
      ),
    );
}

// Callback for drawing the child node 
void _paintChild(PaintingContext context, Offset offset) {
   context.paintChild(child!, offset);
}

Next, we need to check in the paint method whether the child node is a repaint boundary node. If it is, we should go through layer composition; if not, we go through layer compositing:

@Override
void paint(PaintingContext context, Offset offset) {
    if (child != null) {
      final Matrix4 transform =
          Matrix4.translationValues(offset.dx, offset.dy, 0.0)
            ..multiply(_paintTransform!)
            ..translate(-offset.dx, -offset.dy);
      
      if (child!.isRepaintBoundary) { // Add condition
        _paintWithNewLayer(context, offset, transform);
      } else {
        _paint(context, offset, transform);
      }
    } else {
      _transformLayer.layer = null;
    }
}

To make the code clearer, we encapsulate the drawing logic when the child is not null in a pushTransform function:

TransformLayer? pushTransform(
    PaintingContext context,
    bool needsCompositing,
    Offset offset,
    Matrix4 transform,
    PaintingContextCallback painter, {
    TransformLayer? oldLayer,
  }) {
    
    final Matrix4 effectiveTransform =
        Matrix4.translationValues(offset.dx, offset.dy, 0.0)
          ..multiply(transform)
          ..translate(-offset.dx, -offset.dy);
    
    if (needsCompositing) {
      final TransformLayer layer = oldLayer ?? TransformLayer();
      layer.transform = effectiveTransform;
      context.pushLayer(
        layer,
        painter,
        offset,
        childPaintBounds: MatrixUtils.inverseTransformRect(
          effectiveTransform,
          context.estimatedBounds,
        ),
      );
      return layer;
    } else {
      context.canvas
        ..save()
        ..transform(effectiveTransform.storage);
      painter(context, offset);
      context.canvas.restore();
      return null;
    }
}

Now, modify the paint implementation to directly call the pushTransform method:

@Override
void paint(PaintingContext context, Offset offset) {
  if (child != null) {
    pushTransform(
      context,
      child!.isRepaintBoundary,
      offset,
      _paintTransform!,
      _paintChild,
      oldLayer: _transformLayer.layer,
    );
  } else {
    _transformLayer.layer = null;
  }
}

Isn’t it much clearer now? Let’s rerun the example, and the effect is the same as in Figure 1: "A" has been successfully rotated!

Flutter (97): Drawing (IV) Compositing

It’s worth noting that the PaintingContext already provides a wrapped pushTransform method, which we can use directly:

@Override
void paint(PaintingContext context, Offset offset) {
  if (child != null) {
    context.pushTransform(
      child!.isRepaintBoundary,
      offset,
      _paintTransform!,
      _paintChild,
      oldLayer: _transformLayer.layer,
    );
  } else {
    _transformLayer.layer = null;
  }
}

In fact, the PaintingContext has encapsulated corresponding methods for the standard composition of layers with transformation features. Additionally, Flutter has predefined components that have the corresponding transformation capabilities. Here’s a reference table:

Layer NameCorresponding PaintingContext MethodWidget
ClipPathLayerpushClipPathClipPath
OpacityLayerpushOpacityOpacity
ClipRRectLayerpushClipRRectClipRRect
ClipRectLayerpushClipRectClipRect
TransformLayerpushTransformRotatedBox, Transform

2 When is Layer Composition Needed?

2-1. Principles of Layer Composition

From the previous example, we know that when the direct child of CustomRotatedBox is a repaint boundary node, layer composition is required in CustomRotatedBox. In fact, this is just a specific case; there are other situations where CustomRotatedBox also needs to perform layer composition. So, is there a general principle for when layer composition is necessary? The answer is yes!

Let’s consider the fundamental reason for needing layer composition in CustomRotatedBox. If all descendant nodes of CustomRotatedBox share the same PictureLayer, then everything is fine. However, once a descendant node creates a new PictureLayer, the drawing will no longer be associated with the previous PictureLayer because the drawings on different PictureLayers are isolated from each other and cannot influence one another. Therefore, to ensure that transformations apply to all descendant nodes' corresponding PictureLayers, we need to add all descendant nodes to the same ContainerLayer, which necessitates layer composition in CustomRotatedBox.

In summary, a universal principle emerges: When descendant nodes will add new drawing layers to the layer tree, the parent transformation component needs to perform layer composition. Let’s verify this:

Now, let's modify the previous example by adding a Center parent component to the RepaintBoundary:

@override
Widget build(BuildContext context) {
  return Center(
    child: CustomRotatedBox(
      child: Center( // Newly added
        child: RepaintBoundary(
          child: Text(
            "A",
            textScaleFactor: 5,
          ),
        ),
      ),
    ),
  );
}

Because CustomRotatedBox only checks if its direct child’s child!.isRepaintBoundary is true to perform layer composition, and now its direct child is Center, this check will be false, so layer composition will not occur. However, according to our earlier conclusion, since RepaintBoundary is a descendant of CustomRotatedBox and will add a new layer to the layer tree, layer composition should occur. In this example, layer composition should happen but doesn’t, so we expect that "A" cannot be rotated. After running it, we find the result is the same as in Figure 2:

Flutter (97): Drawing (IV) Compositing

Indeed, "A" has not been rotated! It seems that our CustomRotatedBox still needs further modification. The solution to this problem is not difficult: when checking whether to perform layer composition, we need to traverse the entire subtree to see if any repaint boundary nodes exist. If so, we should compose; otherwise, we do not. To accomplish this, we define a new method, needCompositing(), to check for repaint boundary nodes in the subtree:

// Recursively check if there are repaint boundaries in the subtree
needCompositing() {
  bool result = false;
  _visit(RenderObject child) {
    if (child.isRepaintBoundary) {
      result = true;
      return;
    } else {
      // Recursively check
      child.visitChildren(_visit);
    }
  }
  // Traverse child nodes
  visitChildren(_visit);
  return result;
}

Next, we need to modify the paint implementation:

@Override
void paint(PaintingContext context, Offset offset) {
  if (child != null) {
    context.pushTransform(
      needCompositing(), // Check if there are repaint boundary nodes in the subtree
      offset,
      _paintTransform!,
      _paintChild,
      oldLayer: _transformLayer.layer,
    );
  } else {
    _transformLayer.layer = null;
  }
}

Now, let's run the demo again, and the result should match that of Figure 1:

Flutter (97): Drawing (IV) Compositing

Once again, it has been successfully rotated! But there is still an issue, so let’s continue.

2-2. alwaysNeedsCompositing

Let’s consider this scenario: if there are no repaint boundary nodes among the descendants of CustomRotatedBox, but there are descendants adding new layers to the layer tree. In this case, according to our previous conclusions, CustomRotatedBox should also perform layer composition, but it actually does not. We understand the problem, but it’s not easy to solve because when traversing the descendant nodes in CustomRotatedBox, we cannot know whether non-repaint boundary nodes are adding new layers to the layer tree. What can we do? Flutter solves this issue through a convention:

In RenderObject, a boolean property alwaysNeedsCompositing is defined.

Convention: In custom components, if a component's isRepaintBoundary is false and it will add new layers to the layer tree during drawing, it should set alwaysNeedsCompositing to true.

Developers should adhere to this guideline when creating custom components. Based on this convention, we can modify the conditions in CustomRotatedBox when recursively checking the subtree to:

child.isRepaintBoundary || child.alwaysNeedsCompositing

Ultimately, our needCompositing implementation is as follows:

// Recursively check if there are repaint boundaries in the subtree
needCompositing() {
    bool result = false;
    _visit(RenderObject child) {
      // Modify the check condition
      if (child.isRepaintBoundary || child.alwaysNeedsCompositing) {
        result = true;
        return;
      } else {
        child.visitChildren(_visit);
      }
    }
    visitChildren(_visit);
    return result;
}

Note: This requires that non-repaint boundary components must set their alwaysNeedsCompositing value to true when adding layers to the layer tree.

Now let’s take a look at the implementation of the Opacity component in Flutter.