In my application I am using Zxing library for decoding barcodes. "Motorola Xoom" and "Samsung " are the target devices. The company for which I am developing this application uses Code 39 barcodes for their products.
Zxing decodes short barcodes fine, but when I try to decode lengthy "Code 39" barcodes it keeps on trying but produces no result. For image clearance I increased the scanning rectangle area which proved successful for Samsung but for Motorola it is not. Is there any way by which I can make it work for Motorola? Any feedback will be highly appreciated.
Often the problem is a difference in minimum focal distance. That is, if the Motorola device can't focus as closely, then widening the rectangle may make the user hold the barcode so close as to be too close to focus. I would look at this first.
Otherwise you're looking at improving the image processing for this case. The challenge is that the app does simple thresholding, which works well in common cases. It falls down when you have dense 1D barcodes whose bar width nears 1 pixel. Because each pixel is either black or white you lose proportionally a lot of detail about exactly where the bars are.
If that's really the issue you could look at rewriting your app to use a full-resolution capture from the camera, instead of preview. In normal cases, more resolution doesn't help; in these cases it might. You would not be able to have a continuous-scan app this way.
I am one of the Barcode Scanner devs, and maintain a (for-pay) enhanced version called Barcode Scanner+. It has a different image processing algorithm that finds boundaries at sub-pixel resolution, which works better for codes like these. You may want to see how it does -- and if that works well, at least that tells you the kind of approach that works better. I can't send you that code but can describe what it does, if you want to investigate that sort of image processing.
Related
I am using com.google.mlkit:barcode-scanning:17.0.2 to detect QR codes in the pictures.
After getting URI from the gallery I create InputImage and then process this image with BarcodeScanner to find QR codes. When I select a photo of QR codes on paper code is found. But when I take a photo of the QR code on the monitor screen code is never found. What I should do to be able to detect a QR code in a photo of a monitor screen?
(When I use the same scanner with CameraX to do live QR code detection it finds code on the monitor screen)
val image = InputImage.fromFilePath(context, uri)
val scanOptions =
BarcodeScannerOptions.Builder()
.setBarcodeFormats(
Barcode.FORMAT_QR_CODE,
)
.build()
val scanner = BarcodeScanning.getClient(scanOptions)
scanner.process(image)
.addOnSuccessListener {
val code = it.getOrNull(0)?.rawValue
if (code == null) {
// code NOT found
} else {
// code was found
}
}
Example of QR code on paper which is found
Example of QR code on the monitor screen which is NOT found
Chances are that you're fighting against Moiré effect. Depending on the QR detection algorithm, the high frequencies introduced by the Moiré effect can throw the detector off its track. Frustratingly, it is often the better QRcode detectors that are defeated by Moiré patterns.
A good workaround is:
take the picture at the highest resolution you can
perform a blurring of the picture
increase contrast to the max, if possible
(optionally) run a sigma thresholding, or just rewrite all pixels with a luma component below 32 to 0, all those above 224 to 255.
Another way of doing approximately the same operation is
take the picture at the highest resolution you can
increase contrast to the max, if possible
downsample the picture to a resolution which is way lower
The second method gives worse results, but usually can be implemented with device primitives.
Another source of problems with monitors (not in your picture as far as I can see) is the refresh rate. Sometimes, you'll find that the QR code is actually an overexposed QRcode in the upper half of the picture and an underexposed QRcode in the bottom half of the picture. Neither are recognized. This effect is due to the monitor's refresh rate and strategy and is not easy to solve - you can try lowering the monitor's luminosity to increase exposure time, until it exceeds 1/50th or 1/25th of a second, or take the picture from farther away and use digital zooming. Modern monitors have higher refresh rates and actually refresh at more than their own dwell time, so this should not happen; with old analog monitors however it will happen every time.
A third, crazy way
This was discovered half by chance, but it works really well even on cheap hardware provided the QR SDK or library supplies some small extra frills.
Take a video of about 1 second length at the highest frame rate you can get (25 fps?).
From the middle (e.g. 13th) frame, extract the three QR "waypoints" - there might be a low-level function in your SDK called "containsQRCode()" that does this. If it returns true, the waypoints were found and their coordinates are returned to allow performing scaling/estimates. It might return a confidence figure ("this picture seems to contain a QR code with probability X%"). These are the APIs used by apps to show a frame or red dots around candidate QR codes. If your SDK doesn't have these APIs, sorry... you're out of luck.
Get the frames immediately before and after (12th and 14th), then the 11th and 15th, and so on. If any of these returns a valid QR code, you're home free.
If the QR code is found (even if not correctly decoded) in enough frames, but the waypoint coordinates vary much, the hand is not steady - say so to the user.
If you have enough frames with coordinates that vary little, you can center and align on those, and average the frames. Then run the real QRCode recognition on the resulting image. This gets rid of 100% of the Moiré effect, and also drastically reduces monitor dwell noise with next to no information loss. The results are way better than the resolution change, which isn't easy to perform on (some) devices that reset the camera upon resolution change.
This worked on a $19 ESP32 IoT device operating in a noisy, vibration-rich environment (it acquires QR codes from a camera image of carton boxes on a moving transport ribbon).
I noticed a strange issue with Google's ML Kit Barcode scanner. I try to scan a QR code which has a green background, and it does not recognize the QR code at all. However if I crop the same QR code and remove the green background then it scans perfectly well. I don't put any code examples of my implementation here as the official example has the exact same issue.
Here is my image. I don't even know how to research this problem as I totally don't understand what green background can do.
Well, after spending some time on trying to solve this problem with various image processing techniques etc. I found out that the solution is rather simple and was always there right in front of me.
So while building the image analyzer, there is a configuration function to set Target Resolution setTargetResolution(#NonNull Size resolution), which if not set explicitly is defaulting to 640x480, which is probably ok for general use cases related to image analyzer (otherwise I wonder why Google should pick this resolution as a default). And it's also OK for normal QR codes, but for problematic QRs like this it seems to mess thing up, so ML kit needs a higher resolution images for processing.
So just changing this default 640x480 to 1920x1440 immediately solved the issue, and the QR code without borders started to be scanned immediately and with very good performance. I tried other, smaller resolutions as well, tried on different high and low end devices with good and bad cameras, and came to this resolution, which seems to perform the best.
So currently me Image Analyzer builder looks like this, and it works just fine
private val analyzerUseCase = ImageAnalysis.Builder()
.setTargetResolution(Size(1440, 1920))
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
.build()
It seems I've found myself in the deep weeds of the Google Vision API for barcode scanning. Perhaps my mind is a bit fried after looking at all sorts of alternative libraries (ZBar, ZXing, and even some for-cost third party implementations), but I'm having some difficulty finding any information on where I can implement some sort of scan region limiting.
The use case is a pretty simple one: if I'm a user pointing my phone at a box with multiple barcodes of the same type (think shipping labels here), I want to explicitly point some little viewfinder or alignment straight-edge on the screen at exactly the thing I'm trying to capture, without having to worry about anything outside that area of interest giving me some scan results I don't want.
The above case is handled in most other Android libraries I've seen, taking in either a Rect with relative or absolute coordinates, and this is also a part of iOS' AVCapture metadata results system (it uses a relative CGRect, but really the same concept).
I've dug pretty deep into the sample app for the barcode-reader
here, but the implementation is a tad opaque to get anything but the high level implementation details down.
It seems an ugly patch to, on successful detection of a barcode anywhere within the camera's preview frame, to simple no-op on barcodes outside of an area of interest, since the device is still working hard to compute those frames.
Am I missing something very simple and obvious on this one? Any ideas on a way to implement this cleanly, otherwise?
Many thanks for your time in reading through this!
The API currently does not have an option to limit the detection area. But you could crop the preview image before it gets passed into the barcode detector. See here for an outline of how to wrap a detector with your own class:
Mobile Vision API - concatenate new detector object to continue frame processing
You'd implement the "detect" method to take the frame received from the camera, create a cropped version of the frame, and pass that through to the underlying detector.
Maybe a comparison of pictures best illustrate the problem.
This is the original picture:
Using Android SDK, I managed to take this photo from my Android phone:
You may see that, there are lots of gray strips on the photo.
Although the main shapes are there, for I'm processing these photos on an image recognition project, these gray stripe completely ruined the results.
It (Edit: does not ) seems that the built-in photo app would automatically eliminate them, but I don't know how to do it manually in my app. Seems that this is caused by display having a different refresh rate.
What you're seeing is happening because cameras have a small advantage over the human eye when taking the photo.
The refresh rate for most displays is 50Hz or 60Hz, which is too fast for our eyes to notice.
However, a camera sensor takes the image much faster than the human eye, and can see the scan lines created by the refreshing of the image on the display. You can work around this by using a longer exposure time, closer to the human eyes' speed, but you may not be able to control that on most Android devices.
I suggest you use your operating system's inbuilt screenshot utility instead.
I write an application for Motorola Xoom tablet with Android 3.1 for my master thesis that can scan multiple QR Codes in real time with it's camera and that displays additional information in the display over recognised QR Codes.
The recognition is done with the ZXing android app (http://code.google.com/p/zxing/), I basically just changed the code of the ZXing app so that it can recognise multiple QR Codes at the same time and can do this scan continually, without freezing after a successful scan like the original app does. So my app is basically the ZXing app with continous scanning of multiple QR Codes.
But I'm facing a problem:
The recognition rate of QR Codes with the built in camera is not
very good. The ZXing app uses the pictures that it gets from the
camera preview. But these pictures do not have a very good quality.
Is there any possibility to make the camera preview making better
quality pictures?
P.S. I also tried to make real snapshots with camera.takePicture()
to get a better quality, but it takes too long to take the picture
so the real time experience for the user is lost.
Any help is highly appreciated!
Thanks.
Well, the question would be... why is the image quality that bad? Do the image have low resolution? Is the preview out of focus? I've worked with the ZXing Android app before and I know that it has a mechanism to keep the camera auto focusing the live scene.
If the auto focus mechanism is undergoing, then you are possibly decoding some images that might be out of focus. Rationaly, it would make sense to decode only when the camera is in focus, but that would delay the decoding process, since it would have to wait for the focusing to do the image processing phase. However, I wouldn't be too much worried about this for several reasons: 1) the auto focus is very quick, so there will be very few blurry images (if there are any at all), 2) the camera keeps focus for a sufficient amount of time that would allow for a couple decodings, 3) QRCodes typically do not require perfect images to be detected and decoded - they were designed that way.
If this is a problem for you, then disable the continous auto-focus and set the parameter to anything that suits you.
If the problem comes from low resolution frames, well increase it..., but QRCodes were also designed to be identified even in small resolutions. Also, keep in mind the increasing the resolution will also increase decoding time...