Automatic cropping of large amounts of images
Cropping images is easy, one would think… just fire up your favorite photo editor, such as Photoshop, Gimp or any other other from a myriad of choices…
But… it becomes much harder if you need to do this for a large number of images, let’s say some tens of thousands.
Then Photoshopping becomes a task for the rest of your life, perhaps interesting for the the first few hundred of photo’s,
but quite boring after a while. Time for some software to come to the rescue.
At Double Dig IT we encountered this challenge
while we were batch converting a large batch of Tiff’s from the
Medicinsk Museum (Medical Museum) in Copenhagen to JPEG format.
A straight forward job; just read every Tiff one by one in memory and then write it away as JPEG.
While we were doing this, we noticed that the images consisted of photo’s that were glued on an inventory card of the museum. These were old paper inventory cards that where scanned by a third party. The inventory cards had text area’s in ‘boxes’ and had a number of different background colors, some where blue, others green and a whole bunch of them where plain white.
These images were intended to be used in the new Sara system, a Denmark-wide national Collection Management System, powered by Axiell Collections. In this web and cloud-based solution the data from more than 120 Danish museums is managed in a single system.
All the data is already in the Collection Management System, so wouldn’t it be good to crop all these images and get rid of the background and text to keep only the ‘real’ image. After some research on the Web we could not quickly locate a tool to help us with this, so we decided to test some code ourselves:
The methodology would be quite simple: detect the background color of an inventory card by sampling the rows of the image, approximately 10-20 pixels from the borders were sampled and an average was computed. Then a filter was applied to divide pixels into two categories: background and non-background. A threshold was applied, which used the Euclidian distance from every pixel in the RGB space to to the detected background color. The result is a ‘mask’ that can be used to detect the edges of the ‘real’ image.
The ‘glued-on’ images all shared the property that they were all rectangular, by horizontally and vertical ‘scanning’ the mask we could determine horizontal and vertical lines on the ‘mother’ image. A line would consist of at least 50% more or less consecutive pixels of the background color – although we could play with the threshold parameters.
The next step would be to detect the largest rectangle that was enclosed by these lines. A complicating factor were the text boxes in the original image, these where also horizontal and vertical lines. The solution proved to be simply setting a threshold for the width of the horizontal and vertical lines. This prevented the lines that formed the text boxes to be accidentally be detected as image edges.
We wrote a ‘cropper’ class in the C# programming language and tested this first in a Windows user interface. This allowed us to see the original, the background detection and the cropped result side by side. After tuning some of these threshold variables we used the same class in a console program to process the whole batch.
The end result was quite satisfactory, we cropped some 20.000 images in a day or so.
Lessons Learned
The standard methods to manipulate pixels in C# are quite slow. The reason for this is the memory management of C#. To overcome this problem, we used a ‘Fastbitmap’ class, which locks the memory area where the image is located while it is being processed.
Alternative approach… the results of this cropping exercise were good, but partly this was because the original scans where alike, and that the background color detection with horizontal and vertical lines worked well for this type of image. An alternative approach could be to use a neural network to do the edge detection. The advantage of this method would be that there is less programming to do than our ‘statistical’ method, no tweaking of thresholds required, but then one would have to ‘train’ the neural network instead. It would be an interesting exercise to compare both methods, in accuracy, speed and ease of use.
Source Code
– Demo program, to demonstrate these principles, we wrote a small demo program. The source code of this program can be found on Git: https://github.com/bertdd/autocrop
References
– Museion, the home page of the Medical Museum in Copenhagen: www.museion.ku.dk
– Fast bitmaps, info on the principle of fast bitmaps in C#: http://csharpexamples.com/fast-image-processing-c/ & https://www.codeproject.com/Tips/240428/Work-with-Bitmaps-Faster-in-Csharp-3
– Axiell Collections, info on Axiell Collections: https://alm.axiell.com/collections-management-solutions/axiell-collections/