Breakdown Of The Process
Stage 1: Data Acquisition
This involves the user acquiring and organising the images which will be used as the input for generating the dataset.
Stage 2: Image Processing
Produces a binary mask version of the images which separates the foreground from the background.
Stage 3: Creating Annotations
Obtaining the necessary data to generate the JSON file containing the annotation data following the Common Objects in Context (COCO) format.
Stage 4: Training An AI Model
Combining the original images with the annotation data to train an instance segmentation model.
Image Processing
Calculations And Training
This process starts by picking out a random image and using it to calculate the minimum and maximum pixels which should be present in the foreground. This calculation takes into account the image’s width and height, just using one image is enough because all the images should have the same background and are therefore of equal dimensions.
minimum pixels = (image width × image height) × (L/100) maximum pixels = (image width × image height) × (H/100)
Here L represents the lower threshold of white pixels which should be present in the binary mask, while H represents the upper threshold of white pixels which should be present in the binary mask.
Seperating The Foreground
Following this, the images are all iterated through to train the OpenCV background separator, which functions by comparing the pixels between images. When a pixel is found to be present across different images then that pixel is considered a part of the background.
Cleaning Up The Binary Mask
The next phase involves checking that the binary mask does not exceed the maximum white pixel count. This is in place as a high count may indicate that the subject in the image was too close to the camera, resulting in a blurred image which is unsuitable. To ensure that an image with a different background was not accidentally included the program checks the left and right borders. The amount of white pixel on the left and right borders were counted and checked against this formula:
maximum white border pixels = (image height × 2) × (X/100)
Here X represents the percentage of white pixels which should not be exceeded. Once the binary mask had passed all these checks, it was saved to a separate folder along with a copy of the original image in another folder. Once all the images had been iterated through, the folder containing the binary masks and the folder containing their corresponding original images were zipped and downloaded.
This process can take some time to execute and Google Colab has a time limit in which it will disconnect a program from the virtual environment should there be no human input after a while. To not lose progress I implemented a checkpoint system which after a certain number of images will generate a temporary text file containing the data and upload it onto Google Drive. The program will look for this file during start-up and continue from it, this checkpoint file is deleted once the program has generated the complete JSON file. It should be noted that there are different image detection approaches and each one demands different data, for this project the annotation data is catering for instance segmentation detection.