Image Processing with C# Lambda Expressions
For this blog, I’m going to cover the implementation of how to implement dotImage image processing commands based around C# lambda expressions.
To get started, I’m going to come up with a simplified model of image processing. A lambda command is a command that when executed maps a given pixel, represented by its color and coordinates to a new color. Note that this is a woefully incomplete definition of an image command, as it can’t do things like geometric transformations, data reductions, and so on. It can do a large variety of simple pixel-based calculations and that’s good enough for now.
Given that definition, we can define a delegate like this:
public delegate Color PixelDelegate(Color sourceColor, int x, int y);
This is exactly the definition above – a function that operates on a color from a given coordinate and returns a new color.
Lambda expressions compile into delegates, so I can write an identity pixel delegate like this:
(Color c, int x, int y) => c;
This returns the same color for every pixel provided.
From here, image that we have an object name LambdaCommand that inherits from Atalasoft.Imaging.ImageCommand includes a method that looks like this:
public ImageResults Apply(AtalaImage image, PixelDelegate pd) { /* ... */ }
We can do the following to alter an image. Given the source image below, the following transformations are trivial.
LambdaCommand command = new LambdaCommand();
AtalaImage finalImage = command.Apply(sourceImage,
(Color c, int x, int y) => Color.FromArgb(c.A, 255-c.R, 255-c.G, 255-c.B)).Image;
Color Invert Command
AtalaImage finalImage = command.Apply(image,
(Color c, int x, int y) => Color.FromArgb(c.A, c.B, c.G, c.R)).Image;
Swap Red and Blue Channels
AtalaImage finalImage = _command.Apply(image,
(Color c, int x, int y) =>
{
int luma = (int)(c.R * 0.3 + c.G * 0.59 + c.B * 0.11);
return Color.FromArgb(c.A, luma, luma, luma);
}).Image;
Convert to Gray
AtalaImage finalImage = _command.Apply(image,
(Color c, int x, int y) =>
{
int luma = (int)(c.R * 0.3 + c.G * 0.59 + c.B * 0.11);
return luma > new byte[] { 0, 192, 48, 240, 128, 64, 176, 112,
32, 224, 16, 208, 160, 96, 144, 80 }
[(x % 4) + ((y % 4) * 4)] ? Color.White : Color.Black;
}).Image;
Simple Ordered Dithering
This is really an example of an inside-out transformation. It’s the process of taking a task, hiding the overhead and taking the actual work and pulling it from the inside to the outside. We like this kind of code because it makes client work so much easier. Here is the implementation of LambdaCommand in its entirety:
using System;
using System.Drawing;
using Atalasoft.Imaging;
using Atalasoft.Imaging.Memory;
using Atalasoft.Imaging.ImageProcessing;
namespace LambdaImageProcessing
{
public delegate Color PixelDelegate(Color sourceColor, int x, int y);
public class LambdaCommand : ImageCommand
{
private PixelDelegate _pixelDelegate;
// Keep it simple - operate on continuous color formats only
private PixelFormat[] _supportedPixelFormats = new PixelFormat[] {
PixelFormat.Pixel24bppBgr,
PixelFormat.Pixel32bppBgr,
PixelFormat.Pixel32bppBgra
};
// constructors
public LambdaCommand()
: base()
{
// identity command
_pixelDelegate = (Color c, int x, int y) => c;
}
public LambdaCommand(PixelDelegate pixelDelegate)
: base()
{
if (pixelDelegate == null)
throw new ArgumentNullException("pixelDelegate");
_pixelDelegate = pixelDelegate;
}
// properties
public PixelDelegate PixelDelegate
{
get { return _pixelDelegate; }
set
{
if (value == null) throw new ArgumentNullException("value");
_pixelDelegate = value;
}
}
public override PixelFormat[] SupportedPixelFormats
{
get { return _supportedPixelFormats; }
}
// overload Apply to have a PixelDelegate passed it - this is just sugar
public ImageResults Apply(AtalaImage image, PixelDelegate pd)
{
if (pd == null)
throw new ArgumentNullException("pd");
PixelDelegate oldDelegate = _pixelDelegate;
try
{
_pixelDelegate = pd;
return Apply(image);
}
finally
{
_pixelDelegate = oldDelegate;
}
}
protected override void VerifyProperties(AtalaImage image) { }
protected override AtalaImage PerformActualCommand(AtalaImage source, AtalaImage dest, System.Drawing.Rectangle imageArea, ref ImageResults results)
{
// grab memory for source and dest
PixelMemory sourcePM = source.PixelMemory;
PixelMemory destPM = dest.PixelMemory;
// find out how many bytes there are per pixel
int bytesPerPixel = PixelFormatUtilities.BitsPerPixel(source.PixelFormat) / 8;
// do we need to worry about alpha?
bool hasAlpha = PixelFormatUtilities.HasAlpha(source.PixelFormat);
byte[] sourceBytes = new byte[source.RowStride];
// get pixel accessors for the image
using (PixelAccessor sourcePA = sourcePM.AcquirePixelAccessor(), destPA = destPM.AcquirePixelAccessor())
{
// loop over image
for (int y = imageArea.Top; y < imageArea.Bottom; y++)
{
// grab the source scanline, read only
sourcePA.GetReadOnlyScanline(y, sourceBytes);
// acquire the dest scanline
byte[] destBytes = destPA.AcquireScanline(y);
// get the start index
int index = imageArea.Left * bytesPerPixel;
for (int x = imageArea.Left; x < imageArea.Right; x++)
{
// get color byte values for pixel
byte b = sourceBytes[index];
byte g = sourceBytes[index + 1];
byte r = sourceBytes[index + 2];
byte a = hasAlpha ? sourceBytes[index + 3] : (byte)255;
// covert to Color object
Color sourceColor = ColorFromBytes(b, g, r, a);
// call the delegate
Color destColor = _pixelDelegate(sourceColor, x, y);
// extract the bytes
BytesFromColor(destColor, out b, out g, out r, out a);
// write to dest pixel
destBytes[index] = b;
destBytes[index + 1] = g;
destBytes[index + 2] = r;
if (hasAlpha)
destBytes[index + 3] = a;
// advance index
index += bytesPerPixel;
}
}
destPA.ReleaseScanline();
}
return null;
}
private static Color ColorFromBytes(byte b, byte g, byte r, byte a)
{
Color c = Color.FromArgb((int)a, (int)r, (int)g, (int)b);
return c;
}
private static void BytesFromColor(Color c, out byte b, out byte g, out byte r, out byte a)
{
b = (byte)c.B;
g = (byte)c.G;
r = (byte)c.R;
a = (byte)c.A;
}
}
}
You are probably wondering what the cost is of doing this work. After all, I’m doing some fairly heavy lifting with a great number of pixels. In my testing, I was working with a source image that was 2048x1360. On my development machine, which is 2.4G processor, the swap red/blue command takes .719 seconds running in release. By comparison, if I do the general SwapChannelsCommand which is written in unmanaged code, it takes .312 seconds. Now, this comparison isn’t entirely fair – the SwapChannelsCommand does more work on a per pixel basis as it operates in 8 bit per channel and 16 bit per channels and can do any manner of reassignment, not just red/blue. Nonetheless, this is still a nice bit of performance.
The bad news is that this is really not the right approach. You can do a lot with this command, but you can’t do transforms (flip, rotate, etc). You can’t do things that require sampling more than one pixel (blur, resample, etc).
The reason is that this code is moving source and dest lock step and there is no way to work with it. The way around this is to change the approach. Instead of looping of the source, instead, the code should loop over the destination and provide to the delegate a struct that contains the source image, the dest x and y and an assignable color. It will be up to the lambda expression to read from the source image and set the color appropriately.
Yet, it’s still more complicated. Many operations change the PixelFormat of the image. It’s not so simple to express this in the lambda alone – the command would need a property to set the target pixel format. But it’s even worse – some commands don’t know what the target pixel format should be until very late in the process, so this will clearly not work. Further, some commands change the size of the image too (crop, resample, rotate), which makes it again, much harder. It also requires the lambda expression to do random access of the source image instead of letting the command do sequential access. Sequential access using pixel accessors is about four times faster than calling GetPixelColor() on each pixel.
In order to get the full gamut of image processing, we have to pay a much greater price in the lambda expressions, and that’s sad.