Image Reader Design Process
From SoftwarePractice.org
Contents |
Design Process
We plan to create a system that can identify a barcode on an image, extract an array of that barcode then read the barcode and display an output.
We are using Matlab as the main design tool, and an image processing toolbox from Matlab to process the image. The image recognition assignment was initially broken down into smaller manageable tasks.
1. The GUI
2. Recognising the image (barcode)
3. Barcode 39 specifications
4. General Coding for recognition of the bar code and the output.
Stage 1
Each barcode is to be comprised of 5 bars and 4 spaces. Each bar or space may be narrow or a wide in width. Hence from a coding point of view, the group agreed that perhaps the most effective way to represent a wide or narrow bar or space was with a binary pattern.
That is:
A narrow bar would be represented by a (1) A wide bar would be represented with a (11) A narrow space would be represented by a (0) And a wide space would be represented by a (00)
Now the output would be a one dimensional array with a pattern of 0’s and 1’s. This array can be manipulated by being multiplied by 2, and thus the resultant array would be an array of numerical values, and can be used to perform a search for the corresponding characters in the bar code. The search will be faster since Matlab will deal better with single value inputs as opposed to a multiple of binary strings.
Also established was the possibility of creating a minimum and maximum for the resultant array, to provide a foundation for error checking. That is, provide a minimum and maximum threshold for the length of the array with regards to the minimum and maximum amount of elements in the smallest and largest bar code.
Stage 2
By employing the image processing toolbox there are number of things that can be done to an image that is related to our assignment, namely
1. Reading the uploaded image
2. the possibility of De-blurring the image if it is necessary or if possible at all.
3. Converting image to correct type using that is, jpg or gif and the like.
4. Adjusting the image size, so that larger images will automatically be reduced to a standard size that can be displayed on the GUI.
5. Getting rid of "Noise" by clearing clumps of pixels less than a certain value, so that the bar code is the only image being read, and any background writing or un-wanted characters are deleted from the image processing.
6. Additional noise reduction techniques by adopting the function “regionprops”. This function has the ability to identify straight lines in an image (Eccentricity property = 1). Also the area of the size of the bars can be limited; i.e a max and min area for size of pixels groups can be defined.
7. Once all the “noise” has been eliminated the regionprops Bounding Box property can give us the x upper left coordinate, y upper left coordinate, width and height of each bar. With this function it can also calculate if the barcode is rotated, by calculating the gradient between the first bar and the last bar. Hence if the gradient is not zero it will calculate the angle and rotate the barcode so that it is straight.
8. The image is then cropped thus leading to the ability of outputting a binary array of “0s” (for white space) and “1s” (for bars). This can then be used to identify narrow and wide bars and spaces.
A problem was encountered, what if not all the noise was cleared, so that the barcode was not the only object left in the image? Well this comes down to refining the noise reduction techniques and also setting specifications on the input image, so that the barcode in the image is a certain size, if it is rotated more than 90 degrees the resultant barcode will be upside down so we will have a rotate image 180 degree option.
Also at this stage, the skeleton of the Matlab code was being formed in pseudo code. From a flow chart that was created. Research of the presence of a checksum in the bar code was completed. The mathematics and specifications were documented. However, the question of how does one know of the presence of a checksum, by just physically looking at a bar code still remains. Also the pseudo code of the checksum calculation will be completed by the next stage.
Discussion of what the group wanted to see in the GUI included so far:
1. A box that showed the original image with background noise. 2. and a box that displayed the final output. 3. more extensive research was need to discover the limitations on what the GUI could provide.
Stage 3
It has been discovered that the checksum of Bar Code 39 cannot be automatically detected by any scanner or image reader. The instrument that is reading the barcode must be initially set to be able to read and recognise the checksum digit. With regards to implementation of the checksum into our project, the question of not all barcodes containing a checksum is still apparent. Thus the image recognition code cannot always call upon the checksum m-file when reading a barcode. The group has agreed to implement a checkbox or something similar on the GUI to indicate the presence of a checksum by the user. This is also dependent on the limitation of the Matlab’s GUIDE toolbox as well.
An update on the algorithm for reading the bar code and lower level code was produced for testing and integration. The code initially uses a training process to allow for recognition of barcode 39. Thus, the elements of the stars or asterisk which are the start and stop characters for code39 bar codes, were used for training. The code aims to seek and recognise the “……..” that is characteristic of an asterisk. Also the ratio of the widths between narrow and wide elements will also be captured and compared, in order to distinguish between narrow and wide bars and spaces, and thus return a binary array. As mentioned in the specifications section, each element in a code 39 barcode may have a narrow to wide ratio of 1:2, 1:2.2 and 1:3.
Progress has been achieved with image processing techniques that can be used to deblur an image and how to extract binary array of image. To deblur an image is quite difficult and it predominately depends on how badly "blurred" the image is initially. It has been discovered that the deblurring process can be executed with a matlab image processing function called blinddeconvolution that uses a distortion operator also known as the Point Spread Function (PSF). Essentially the PSF is the inverse Fourier Transform of the Optical Transfer function. That is that to deblur an image, the image needs to be deconvolved with the PSF.
The main issues with the the debur process is that we do not know the cause of the blurring, and consequently we do not know what the distortion operator is. This is the reason we are using the blind deconvolution function as it uses an initial gaussian distributed PSF (best representation of lens distortion). This will be an option for the user, however, no guarantees can be made to the user that our processor will be able to read the resultant barcode. The group has agreed that a perfect deblured image is beyond the scope and expectation of the Image Processing assignment, and we will attempt our best to come close to a perfect image.
Stage 4
The GUI will now offer a browse button to input a barcode on jpeg form to be processed. The image will be displayed and cropped to fit into the window. There also will an advanced option to deblur the image if it is needed. Also there will be another advanced option to rotate the image if it is sideways or the like. There will also be a second window that will display the line display form the image possessor. And a smaller window will display the single array that will be sent for processing.
The barcode recognition code is almost complete. The initial training stage or the “train” function will take the input of the one dimensional array produced the image processing functions. It will then find the first dark element of the barcode and take the widths of the first nine elements. Then the variables for the width of a narrow bar/space and the width of a wide bar/space will be determined by assuming the fist character is a "*".
Next, the "preProcess" function takes the input of the one dimensional array and an array of variables from the train function. It will then read the remainder of the one dimensional barcode array generating a simplified array with one array element for a narrow barcode element and 2 array elements for a wide barcode element. This will make sure that whole character being sent to the next function which is recogniseBarcode. This function will process each character and refer to a lookup table to output the correct character for the barcode.
At this stage all the separate pieces of code and error codes will have to be integrated.
Stage 5
The code has been completely integrated and is running smoothly. The error codes have also been defined as:
0 = Success
1 = No barcode found in image (image processing function)
2 = No barcode found in output (train function)
4 = Thin/Wide ratio incorrect (train function)
6 = No star found in training (train function)
8 = Missing elements from the training process (train function)
12 = Incorrect number of elements from preprocessor (preprocess function)
22 = White space test failed (recognise barcode function)
24 = Number of elements incorrect (recognise barcode function)
26 = No trailing star found (recognise barcode function)
28 = Checksum Error (recognise barcode function)
29 = Character/s not found in lookup table (recognise barcode function)
The code has been tested with a few barcodes. These error messages are helping us identify exactly where certain barcodes will fail and allow us then to analyse that section of code to see if we can improve performance. We are on track and are in track.
Stage 6
There were issues with the final graph being displayed properly in the GUI. Initially, the image selected in the GUI to be processed by the code was displayed as a jpeg. However, the selected image is then processed, and one of the outputs is the graph that is not a jpeg, and hence to the naked eye, it is a little hard to distinguish. It was rectified with the function “rgb_img”, so that the graph would be displayed as a proper jpeg file rather than the dark code output graph.
Above all, the group is still integrating separate pieces of code, in particular our “recognise_barcode” function, and we are cleaning it up of any bugs!
We tested out the GUI with varying jpegs of barcodes. Only some of the barcodes read accurately, some were perhaps too blurred to be read and outputted errors such as “no start character”, this was particularly the case with the checksum bar code samples. We could not decide if it was the selected jpegs or the actual code. So we will try to test in isolating, and re-test with clearer scanned barcodes to attempt to prevent any further un-expected errors.
Also when reading some of the bar codes, the GUI would output the correct reading, however there would be a "+" term in the middle of each character. For example a barcode reading "code 39" would output "c+o+d+e 39", as seen in the figure below. We thought this was strange, however, we were also just glad that our entire Matlab Image recognition was functioning.
We also outputted a jpeg in our image output box on our GUI when an error occurred. This was just some room to have fun while exploring the extent of Matlab guide. The only issue was that we could not get rid of the error jpeg once a new bar code was selected for image reading.
Stage 7
The checksum barcodes were tested an error is always returned. Usually the errors that frequent occurred were “no trailing character” or “no start character”. Which meant that the Matlab code was being stopped close to the beginning in the training stage, and not processing the calculation for the checksum at all.
A new font called “free 3 of 9” was uploaded onto our OS, which allowed us to create bar codes straight from Microsoft word. These samples were used to attempt to test more checksum bar codes. They were saved as jpegs and then ran in our GUI, however with no avail. After repeated checking the variances in the code, the noise reduction factor and the checksum calculations we could not understand where the errors were occurring for the checksum bar codes.
This issue was not solved until we again, created a bar code with a checksum and saved it at a higher resolution which was close of 40KB. This bar code was successfully read this time. Which prompted us to realise that the previous bar codes we were testing were all below 20KB, which must have been too small for accurate processing.
We are currently also considering extending our project scope beyond code 39, to contain the full ASCII code 39 version for extra project marks. Since there is not much else we can add to our current image reading system for bonus marks.
Below are some successful screen shots of our Woking GUI, both with expected errors and result outputs.
This first figure shows that checksum is working as the input barcode has a checksum in it (*BAAARF101%*) and our function has correctly calculated the checksum (i.e. the % is the checksum and it has been correctly accounted for).
This second figure shows the deblur function and the limitation that yes it will deblur the image, however, the image may be too blurred to begin with so it will return in error stating that no barcode has been found.
Stage 8
We realised that the reason why some of our bar code readings were outputting a "+" character in the GUI output was due to the fact that the program used to create the barcode was using "code 39 full-ASCII", however our image recognition code was only for "code 39". (refer to stage 6)
Hence when our code was to process a small case letter, a "+" character would be appended to the end of it. This was the reason why "+" was added to the letters and not the numbers and the space characters in some of the test bar codes used. Which seemed to be the source of much confusion at the time.
We also realised that following error message kept appearing "Incorrect number of elements from preprocessor". Now this made us look at the preprocess function and this is the function which will classify the narrow bars and the wide bars. We realised that the source of this problem was the variance value as this was originally set to 0.4*thin. This would then be used to set the limits of a wide and a thin, i.e. (thin - variance <= thin <= thin + variance) and (wide-variance <= wide <= wide + variance). Now as you can see there is a gap between high thin and low wide and this meant that some bars were not being classified and thus the reason for the error. Now we manually calculated the wides and thins for some barcodes that were producing this error and they did turn out to have some bars in this unidentified space. Then we decided to set the variance to 1, so that a thin would be (0 <= thin <= 2*thin) and a wide (0<=wide<=2*wide). Now it would classify a thin first and then anything that was not a thin would be a wide. This dramatically improved performance and we were able to read more barcodes then ever before. We were still worried about this cross over thresholds and the final variance that we came up with is detailed in our Image Processing to Barcode Reader page.
To return to main page of Team B click here Bar code reading from image
