Improved Painting

When I wrote about my first Smart Mobile Studio program I mentioned that it failed on iPad (and would probably fail on iPhone too but I couldn’t test that) because the Paint code took too long and the iOS simply aborted it. As far as I know there are two ways to fix this in JavaScript – you can either move the calculation to a background thread using a Web worker mechanism or split the Paint into multiple parts where each part only calculates and paints small part of the screen and then returns control to the browser. For the time being I went with the latter solution but that will not prevent me from testing the Web worker approach at some time.

Staggered Paint

The new Paint is significantly more complicated than the old one.

procedure TW3MandelbrotCanvas.Paint;
var
  firstY    : integer;
  imageLine : TW3ImageData;
  iterations: TIterationArray;
  start     : float;
  y         : integer;
begin
  if not Visible then 
    Exit;
  if not FInitialized then
    MakeInitialWorld;
  imageLine := Canvas.CreateImageData(FCoord.View.Width + 1, 1);
  start := Now;
  firstY := FResumePaint;
  if firstY < 0 then begin
    ClearView;
    firstY := FCoord.View.Top;
  end;
  FResumePaint := -1;
  for y := firstY to FCoord.View.Bottom do begin
    iterations := FEngine.GetLine(FCoord.World.Left, FCoord.World.Right,
      FCoord.MapToWorldY(y), FCoord.View.Width + 1);
    MapIterationsToImageLine(iterations, imageLine);
    Canvas.PutImageData(imageLine, FCoord.View.Left, UpDown(y));
    if HasElapsedMs(start, CPaintTimeout_ms) and (y < FCoord.View.Bottom) then
    begin
      FResumePaint := y + 1;
      if not assigned(FTimer) then
        FTimer := TW3EventRepeater.Create(PaintTimer, 1);
      Exit;
    end;
  end;
  FTimer.Free;
  FTimer := nil;
end;

After few initial checks and optional initialization, the code allocates one high off-screen image buffer (Canvas.CreateImageData). If this is a fresh paint (and not a continuation of a previous work), view is cleared (ClearView) and the paint loop starts from the top of the view (FCoord.View.Top). Otherwise, paint loop starts from the first unpainted line (FResumePaint).

The code then loops from the starting line to the end of the view (for loop). In each iteration it gets one line of data from the Mandelbrot engine (FEngine.GetLine), converts the data to pixels (MapIterationsToImageLine) and writes the data to the screen (Canvas.PutImageData). Then it checks the time. If more than CPaintTimeout_ms (one second in the current code) has elapsed, the code stores next line to be painted in the FResumePaint field and sets the timer which will call the Paint method as soon as possible.

function TW3MandelbrotCanvas.PaintTimer(sender: TObject): boolean;
begin
  Paint;
end;

Then the story continues until the last line is painted and timer is destroyed. After that, new paint will be requested by the main form when the user action calls for it.

Painting Embellishments

Last time I mentioned painting lines and rectangles on the canvas to indicate zoom and pan operations.

Again, there are two ways to achieve nice smooth effect that is pleasing for the eye. As the user moves the line around, the rectangle/line must also move. The problem is not painting the new rectangle/line but restoring the Mandelbrot set image where the rectangle/line was painted previously.

One option is to keep Mandelbrot set image in the big off-screen bitmap and then always paint Mandelbrot set first (directly from this off-screen bitmap) and paint rectangle or line over it.

Another possibility is to store the bitmap under the rectangle/line into an off-screen buffer before it is painted and then restore it when the image must be restored. This is the way my program handles the job.

I’ll just show few examples of the code here. GrabArrow stores everything under the line (and that may be a large area as we can’t really store diagonal part of the image (or rather, it could be done but would be quite complicated) but must store a rectangle that contains the line.

procedure TEmbelishCanvas.GrabArrow(pFrom, pTo: TPoint);
var
  rect: TRect;
begin
  FArrow.Active := true;
  FArrow.PFrom := pFrom;
  FArrow.PTo := pTo;
  rect := FixOrientation(w3_Rect(pFrom.X, pFrom.Y, pTo.X, pTo.Y));
  FArrow.Image := FCanvas_ref.GetImageData(rect.Left - 1, rect.Top - 1,
    rect.Right - rect.Left + 3, rect.Bottom - rect.Top + 3);
end;

[FixOrientation is a helper function that “orients” the rectangle (swaps corners if necessary) so that Left is not larger then Right and Top is not larger than Bottom.]

GrabMarquee stores the area under the rectangle and does it in an optimized manner – it only grabs four small rectangles lying directly below the rectangle sides.

procedure TEmbelishCanvas.GrabMarquee(rect: TRect);
var
  width, height: integer;
begin
  width := rect.Right - rect.Left;
  height := rect.Bottom - rect.Top;
  FMarquee.Active := true;
  FMarquee.Rect := rect;
  FMarquee.Top := FCanvas_ref.GetImageData(
      rect.Left - 1, rect.Top - 1, width + 2, 2);
  FMarquee.Bottom := FCanvas_ref.GetImageData(
      rect.Left - 1, rect.Bottom - 1, width + 2, 2);
  FMarquee.Left := FCanvas_ref.GetImageData(
      rect.Left - 1, rect.Top - 1, 2, height + 2);
  FMarquee.Right := FCanvas_ref.GetImageData(
      rect.Right - 1, rect.Top - 1, 2, height);
end;

Hide does the reverse operation – restores the original bitmap.

procedure TEmbelishCanvas.Hide;
var
  rect: TRect;
begin
  if FArrow.Active then begin
    rect := FixOrientation(w3_Rect(FArrow.PFrom.X, FArrow.PFrom.Y,
        FArrow.PTo.X, FArrow.PTo.Y));
    FCanvas_ref.PutImageData(FArrow.Image, rect.Left - 1, rect.Top - 1);
    FArrow.Active := false;
  end;
  if FMarquee.Active then begin
    FCanvas_ref.PutImageData(
        FMarquee.Top, FMarquee.Rect.Left - 1, FMarquee.Rect.Top - 1);
    FCanvas_ref.PutImageData(
        FMarquee.Bottom, FMarquee.Rect.Left - 1, FMarquee.Rect.Bottom - 1);
    FCanvas_ref.PutImageData(
        FMarquee.Left, FMarquee.Rect.Left - 1, FMarquee.Rect.Top - 1);
    FCanvas_ref.PutImageData(
        FMarquee.Right, FMarquee.Rect.Right - 1, FMarquee.Rect.Top - 1);
    FMarquee.Active := false;
  end;
end;

The remaining code contains two paint methods (one draws a line, another a rectangle) and two public members ShowArrow and ShowMarquee which are called from the mouse handling events.

procedure TEmbelishCanvas.PaintArrow(pFrom, pTo: TPoint);
begin
  FCanvas_ref.StrokeStyle := '#FFFFFF';
  FCanvas_ref.LineWidth := 1;
  FCanvas_ref.BeginPath;
  FCanvas_ref.LineF(pFrom.X, pFrom.Y, pTo.X, pTo.Y);
  FCanvas_ref.Stroke;
  FCanvas_ref.ClosePath;
end;

procedure TEmbelishCanvas.PaintMarquee(rect: TRect);
begin
  FCanvas_ref.StrokeStyle := '#FFFFFF';
  FCanvas_ref.LineWidth := 1;
  FCanvas_ref.StrokeRectF(rect.Left - 0.5, rect.Top - 0.5, rect.Right - rect.Left + 1, rect.Bottom - rect.Top + 1);
end;

procedure TEmbelishCanvas.ShowArrow(pFrom, pTo: TPoint);
begin
  Hide;
  GrabArrow(pFrom, pTo);
  PaintArrow(pFrom, pTo);
end;

procedure TEmbelishCanvas.ShowMarquee(rect: TRect);
begin
  Hide;
  if (rect.Right = rect.Left) or (rect.Top = rect.Bottom) then
    Exit;
  GrabMarquee(rect);
  PaintMarquee(rect);
end;

Image Cache

If you check the program in the desktop browser (Chrome and Safari work best) you’ll see that it also displays the current position and zoom factor in the top left corner (just two TW3Label components) and when you zoom into the image you’ll see the blue arrow buttons lighting up.

Those two buttons enable you to quickly browse through the last 10 images. Each image is stored in a off-screen buffer when it is calculated and clicking the arrow button just puts an image from this cache on the screen so this is blindingly fast. The mechanics of the implementation are exactly the same as for the embellishment drawing – GetImageData is used to grab pixels and PutImageData puts them back.

You’ll also see two disabled buttons (“L” and “T”) which are just a placeholders for future code and a Help button (“?”) which will display a short help.

I’ll return to the Help button implementation (it actually displays a new form) and to speed buttons later. Next time you can expect something way more interesting – I’ll teach the program to recognize touch.