Background

An AtalaImage object (or any System.Drawing.Image / System.Drawing.Bitmap type image requires a contiguous block of memory Width(pixels) * Height(pixels) * BitDepth.

When working with very large images (large height/width and higher bit depth, it's easy to find yourself with images that require a very large uninterrupted block...

In a 32 bit process, regardless of how much memory the system has in total, you only get access to 2 GiB total .. and that 2GiB is not at all guaranteed to be one big block to start.

This is "Memory fragmentation" and it's not unlike disk fragmentation..

We've likened it to "playing a game of Tetris with your memory.. but poorly"

So, when you have a use case such as "I have this very large hig bit depth image I need to down-convert to 1 bit per pixel but it's blowing out my memory when I attempt to get the full size original image as an AtalaImage".. or

The typical way to binarize an image is to just read it in as an AtalaImage then run one of our threshold commands on it to generate a new binarzied output image

AtalaImage img = new AtalaImage(streamOrFileNameHere, frameIndex, null);
AtalaImage binarizedImage = threshold.Apply(img).Image;
img.Dispose();

but the issue is that when you try and call this part:

AtalaImage img = new AtalaImage(streamOrFileNameHere, frameIndex, null);

it throws a System.OutOfMemoryException

The Solution

To fix this, we can attempt to process in "chunks" the output image still needs a full contiguous block of memory height * width * pixel depth but since Pixel Depth is 1, this is often small enough to work

The general workflow is:

1. Get the size of the original image
2. Use that size to calculate the quarters/tiles (topLeft, topRight, bottomLeft, bottomRight)
3. Make a single empty output image the size of the original
4. Loop through each tile, rendering only that one tile in full color..
1. Binarize that tile
2. Dispose the full color tile
3. Overlay that binary tile on the final image in position
4. Dispose the tile image
5. Save the final output image

For step 2, we need a quick utility method:

private Rectangle[] GetQUadTileRects(Size size)
{
    System.Diagnostics.Debug.WriteLine("\nGetQUadTileRects(" + size.ToString() + ")...");
    Rectangle[] quads = new Rectangle[4];

    // we rely on integer rounding here.. 
    int quadWidth = size.Width / 2;
    int quadHeight = size.Height / 2;

    // TopLeft is easy
    quads[0] = new Rectangle(0, 0, quadWidth, quadHeight);
    // TopRight nees to offset the width by the width of previous
    quads[1] = new Rectangle(quadWidth, 0, size.Width - quadWidth, quadHeight);
    // BottomLeft offsets Y by quadHeight from previous
    quads[2] = new Rectangle(0, quadHeight, quadWidth, size.Height - quadHeight);
    // BottomRight offsets X by quadwidth from previous and Y by quadheight from previous
    quads[3] = new Rectangle(quadWidth, quadHeight, size.Width - quadWidth, size.Height - quadHeight);

    System.Diagnostics.Debug.WriteLine("SANITY CHECK...");
    System.Diagnostics.Debug.WriteLine("  IMG Size: " + size.ToString());
    System.Diagnostics.Debug.WriteLine("  CalcSize: Width: " + (quads[0].Width + quads[1].Width).ToString() + " Height: " + (quads[0].Height + quads[2].Height).ToString());
    System.Diagnostics.Debug.WriteLine("  quads:");
    System.Diagnostics.Debug.WriteLine("    TopLeft: " + quads[0].ToString());
    System.Diagnostics.Debug.WriteLine("    TopRight: " + quads[1].ToString());
    System.Diagnostics.Debug.WriteLine("    BottomLeft: " + quads[2].ToString());
    System.Diagnostics.Debug.WriteLine("    BottomRight: " + quads[3].ToString());

    System.Diagnostics.Debug.WriteLine("GetQUadTileRects() COMPLETE\n");

    return quads;
}

/// In theory we could save some processing if we passed in the decoder but we wanted this method to be as standalone as possible
private void RenderBitonalTile(Stream inStream, AtalaImage finalImg, Rectangle currentTile, int frameIndex, BinarizeCommand binarize)
{
    inStream.Seek(0, SeekOrigin.Begin);

    OverlayCommand overlay = new OverlayCommand();
    AtalaImage currentTileImg = null;

    ImageDecoder detectedDecoder = RegisteredDecoders.GetDecoder(inStream);
    inStream.Seek(0, SeekOrigin.Begin);

    AtalaImage rawTile = null;
    if (detectedDecoder.GetType() == typeof(TiffDecoder))
    {
        TiffDecoder decoder = detectedDecoder as TiffDecoder;
        rawTile = decoder.ReadRegion(inStream, currentTile, frameIndex, null);
    }
    else if (detectedDecoder.GetType() == typeof(PdfDecoder))
    {
        PdfDecoder decoder = detectedDecoder as PdfDecoder;
        rawTile = decoder.ReadRegion(inStream, currentTile, frameIndex, null);
    }
    else
    {
        // NOTE if ytou want to do JBig2 files we can ... but they're already binary so kind of not useful
        //else if (detectedDecoder.GetType() == typeof(Jb2Decoder))
        //{
        //    Jp2Decoder decoder = detectedDecoder as Jp2Decoder;
        //    rawTile = decoder.ReadRegion(inStream, currentTile, frameIndex, null);
        //}

        //throw new Exception("This Only Works for TIFF or PDF images");
        throw new Atalasoft.Imaging.Codec.ImageReadException("This feature requires ReadRegion method in decoder: it can onlly Work for TIFF, PDF or Jbig2 images");
    }

    currentTileImg = binarize.Apply(rawTile).Image;
    rawTile.Dispose(); // getging rid of this asap

    overlay.TopImage = currentTileImg;
    overlay.Position = currentTile.Location;
    overlay.Apply(finalImg);

    currentTileImg.Dispose(); // the moment we have the final image updated we don't need the tile so clear its memory
    inStream.Seek(0, SeekOrigin.Begin); // be kind, rewind
}

Now, we just need to put this all together...

int targetFrameIndex = 0;

using (FileStream inStream  = new FileStream(inFile, FileMode.Open, FileAccess.Read, FileShare.Read))
{
    ImageInfo info = RegisteredDecoders.GetImageInfo(inStream, targetFrameIndex);
    inStream.Seek(0, SeekOrigin.Begin);

    Size imgSize = info.Size;

    Rectangle[] tiles = GetQUadTileRects(imgSize);

    AtalaImage finalImg = new AtalaImage(imgSize.Width, imgSize.Height, PixelFormat.Pixel1bppIndexed, Color.White);
	
	// We could have just binarized internally, but it would be nice to be able to control this from the outside
    BinarizeCommand binarizeCommand = new BinarizeCommand(BinarizeMethod.GlobalThreshold);

    System.Diagnostics.Debug.WriteLine("\nPROCESSING TOP LEFT IMAGE: " + tiles[0].ToString());
    RenderBitonalTile(inStream, finalImg, tiles[0], targetFrameIndex, binarizeCommand);
    // if you want to watch your progress, uncomment these saves
    //finalImg.Save("0_topLeftTile.png", new PngEncoder(), null);

    System.Diagnostics.Debug.WriteLine("\nPROCESSING TOP RIGHT IMAGE: " + tiles[1].ToString());
    RenderBitonalTile(inStream, finalImg, tiles[1], targetFrameIndex, binarizeCommand);
    // if you want to watch your progress, uncomment these saves
    //finalImg.Save("1_TopTwoTiles.png", new PngEncoder(), null);

    System.Diagnostics.Debug.WriteLine("\nPROCESSING BOTTOM LEFT IMAGE: " + tiles[2].ToString());
    RenderBitonalTile(inStream, finalImg, tiles[2], targetFrameIndex, binarizeCommand);
    // if you want to watch your progress, uncomment these saves
    //finalImg.Save("2_firstThreeTiles.png", new PngEncoder(), null);

    System.Diagnostics.Debug.WriteLine("\nPROCESSING BOTTOM RIGHT IMAGE: " + tiles[3].ToString());
    RenderBitonalTile(inStream, finalImg, tiles[3], targetFrameIndex, binarizeCommand);
    // if you want to watch your progress, uncomment these saves
    //finalImg.Save("3_allFourTiles.png", new PngEncoder(), null);

    finalImg.Save(outFile, new TiffEncoder(), null);
    finalImg.Dispose();
}

Now this is set up to just to a single frame.. but what if you want to do this to every page of a mutipage tiff or PDF?

Well, the answer is to put this code into the LowLevelAcquire of a custom ImageSource derived from our RandomAccessImageSource.

Attached to this KB article you'll find: BitonalTilingStreamImageSource.cs.zip

It's an image source that does all the lifting described here for you

Why didn't we just give you this to start? we were hoping to explain the why so you'd be comfortable and not just trust the "wizard behind the curtain"

To use it you would do something like this:

string inFile = "path\\to\\in.tif";
string outFile = "path\\to\\out.pdf;

using (FileStream inStream = new FileStream(inFile, FileMode.Open, FileAccess.Read, FileShare.Read))
{
    using (Atalasoft.Examples.BitonalTilingStreamImageSource btsis = new Atalasoft.Examples.BitonalTilingStreamImageSource(inStream), new BinarizeCommand(BinarizeMethod.GlobalThreshold), false)
    {
        using (FileStream outStream  = new FileStream(outFile, FileMode.Create))
        {
            PdfEncoder enc = new PdfEncoder();
            enc.SizeMode = PdfPageSizeMode.FitToPage;
            // NOTE: to make this output a TIFF instead, just use
            //TiffEncoder enc = new TiffEncoder();
            
            enc.Save(outStream, btsis, null);
        }
    }
}