I've been experimenting with barcodedetector and associated Vision APIs for a week or so and - unfortunately - the conclusion is that they are unreliable. I've tested them with ~10 devices: Samsung and Nexus tablets/phones with Android 4.4 to 6.01.
The common problem was that I could not decode certain QR codes - other apps based on zxing library as well as iPhone had no problems decoding them. The problematic QR codes had one of the following 'flaws': somewhat distorted timing (due to scaling), no quiet zone, or their bitmap data had a bottom/left origin (i.e. image was mirrored).
Also troubling is that the associated APIs (e.g. CameraSource) appear to be buggy. Fortunately, the barcode reader sample comes with open source version so it can be fixed if necessary. The most obvious bug is the way camera resolutions are matched to view sizes without taking into account current orientation. You can clearly see the side effects in the sample - camera preview never fills the containing activity area, especially in the landscape orientation.
Has anyone run into similar problems? Are there any workarounds for the show stoppers - i.e. decoding qr codes?
Thanks.
Related
I noticed a strange issue with Google's ML Kit Barcode scanner. I try to scan a QR code which has a green background, and it does not recognize the QR code at all. However if I crop the same QR code and remove the green background then it scans perfectly well. I don't put any code examples of my implementation here as the official example has the exact same issue.
Here is my image. I don't even know how to research this problem as I totally don't understand what green background can do.
Well, after spending some time on trying to solve this problem with various image processing techniques etc. I found out that the solution is rather simple and was always there right in front of me.
So while building the image analyzer, there is a configuration function to set Target Resolution setTargetResolution(#NonNull Size resolution), which if not set explicitly is defaulting to 640x480, which is probably ok for general use cases related to image analyzer (otherwise I wonder why Google should pick this resolution as a default). And it's also OK for normal QR codes, but for problematic QRs like this it seems to mess thing up, so ML kit needs a higher resolution images for processing.
So just changing this default 640x480 to 1920x1440 immediately solved the issue, and the QR code without borders started to be scanned immediately and with very good performance. I tried other, smaller resolutions as well, tried on different high and low end devices with good and bad cameras, and came to this resolution, which seems to perform the best.
So currently me Image Analyzer builder looks like this, and it works just fine
private val analyzerUseCase = ImageAnalysis.Builder()
.setTargetResolution(Size(1440, 1920))
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
.build()
We are developing our own Android-based hardware and we wish to use Vuforia (developed via Unity3D) for certain applications. However, we are having problems making Vuforia work well with our current camera orientation settings.
On our hardware, when the camera is placed horizontally - everything works fine. That is, when the camera is parallel to the placement of the display. However, we need to place the camera vertically, or in other words, with a 90 degree difference to the placement of the display. These are all hardware settings. Our kernel is programmed according to such settings and every other program that utilises the camera works compatibly with everything, including our IMU sensors. However, apps developed with Vuforia behave completely odd when the camera is placed vertically.
We assume the problem to be related to Vuforia's algorithms of processing raw camera data however we are not sure. Moreover, we do not know how to fix the situation. For further details, I can list:
-When "Enable Video Background" is on, the projected image is distorted and no video feed is available. The AR projection appears on a black background with distorted dimensions.
-When "Enable Video Background" is on and the device is rotated, the black background is replaced by flickering solid colors.
-When "Enable Video Background" is off, the AR projection has normal dimensions (no distortion) however it is tracked with wrong axis settings. For example, when the target moves left in real world, the projection moves up.
-When "Enable Video Background" is off and the device is rotated, the AR projection is larger compared to its appearance when the device is in it's default state.
I will be glad to provide any more information you need.
Thank you very much, have a nice day.
PS: We have found out that applications that use the camera as a main purpose (Camera apps, Barcode Scanners, etc) work fine while apps for which camera usage is an extra quality (such as some games) have the same problem as Vuforia. This make me think that apps who access the camera directly work fine whereas those who use Android API and classes fail for some reason.
First understand that every platform deals with cameras differently and that beyond this different android phone manufacturers deal with these differently as well. In my testing WITHOUT vuforia I had to transform the plane I cast the video feed onto 0,-90,90 for android/iphone and -270,-90,90 for the windows surface tablet. Past this the iPhone rear camera was mirrored, the android front camera was mirrored as well as the surface front camera. That is easy to account for, but an annoying issue is that the Google Pixel and Samsung front cameras were mirrored across the y (as were ALL iOS on the back camera), but the Nexus 6p was mirrored across the x. What I am getting at here is that there are a LOT of devices to account for with android so try more than just that one device. Vuforia so far has dealt with my pixel and 4 of my iOS devices just fine.
As for how to fix your problem:
Go into your player settings for unity and look at the orientation. There are a few options here and my application only uses portrait so I force portrait and it seems to work fine (none of the problems I had to account for with the above mentioned scenario). Vuforia previously did NOT support auto rotation so you need to make sure you have the latest version since it sounds like that is what you need. If the auto rotate is set and it is not working right you may have to account for that specific device (don't do this for all devices until after you test those devices). To account for that device use an if (or construct a case statement if you have multiple instances of this problem with different devices) and then reflect or translate as needed. Cross platform development systems (like unity) doesn't always get everything perfect since there is basically no standard. In these cases you have to directly account for them by creating a method and a case statement within that so you can cleanly and modularly manipulate all necessary devices. It is a pain, but it beats developing for all devices separately.
One more thing is make sure you check out the vuforia configuration file as it has some settings such as camera mirror and direction settings on there. These seem to be public settings so you should also be able to script to these in your case statement in the event you need to use "Flip Horizontally" for one phone, but not another.
In my AS3 Flex Mobile application for Android, I am using camera and it is being automatically rotated 90 degrees before I even done any video rotation by myself, it seems like it's a known bug in AIR. But I was wondering if anyone found a solution since it's really pretty important feature for mobile application developer.
I've tried to do some rotation manually in my code, but it is only fixes the view on my display, but still sends the wrong video to the receiver.
If any code is required I will add the snippets
Please let me know.
As you mentioned, this is a known bug with AIR. It is not consistent, either. On some devices, it is in the correct orientation but in some (and all iOS devices, I believe, though I haven't fully tested that), it is rotated as you are seeing. For example, it was always oriented correctly on my Nexus 4 and on my Nexus 5, but a friends Moto X is rotated incorrectly.
Unfortunately, I don't believe there is anything you can do short of having the user do a calibration (i.e. overlay a straight line and tell them to place it horizontally and click a button) and rotating the camera display and any images you take with the display.
That being said, if you are using the camera to take photos, I highly recommend using CameraUI instead, which is the native implementation.
I've faced the same issue today but i'm developping in Java, not with AIR so i don't know if it the same, for me the solution was to add this line before starting the recording.
mMediaRecorder.setOrientationHint(90);
I write an application for Motorola Xoom tablet with Android 3.1 for my master thesis that can scan multiple QR Codes in real time with it's camera and that displays additional information in the display over recognised QR Codes.
The recognition is done with the ZXing android app (http://code.google.com/p/zxing/), I basically just changed the code of the ZXing app so that it can recognise multiple QR Codes at the same time and can do this scan continually, without freezing after a successful scan like the original app does. So my app is basically the ZXing app with continous scanning of multiple QR Codes.
But I'm facing a problem:
The recognition rate of QR Codes with the built in camera is not
very good. The ZXing app uses the pictures that it gets from the
camera preview. But these pictures do not have a very good quality.
Is there any possibility to make the camera preview making better
quality pictures?
P.S. I also tried to make real snapshots with camera.takePicture()
to get a better quality, but it takes too long to take the picture
so the real time experience for the user is lost.
Any help is highly appreciated!
Thanks.
Well, the question would be... why is the image quality that bad? Do the image have low resolution? Is the preview out of focus? I've worked with the ZXing Android app before and I know that it has a mechanism to keep the camera auto focusing the live scene.
If the auto focus mechanism is undergoing, then you are possibly decoding some images that might be out of focus. Rationaly, it would make sense to decode only when the camera is in focus, but that would delay the decoding process, since it would have to wait for the focusing to do the image processing phase. However, I wouldn't be too much worried about this for several reasons: 1) the auto focus is very quick, so there will be very few blurry images (if there are any at all), 2) the camera keeps focus for a sufficient amount of time that would allow for a couple decodings, 3) QRCodes typically do not require perfect images to be detected and decoded - they were designed that way.
If this is a problem for you, then disable the continous auto-focus and set the parameter to anything that suits you.
If the problem comes from low resolution frames, well increase it..., but QRCodes were also designed to be identified even in small resolutions. Also, keep in mind the increasing the resolution will also increase decoding time...
In my application I am using Zxing library for decoding barcodes. "Motorola Xoom" and "Samsung " are the target devices. The company for which I am developing this application uses Code 39 barcodes for their products.
Zxing decodes short barcodes fine, but when I try to decode lengthy "Code 39" barcodes it keeps on trying but produces no result. For image clearance I increased the scanning rectangle area which proved successful for Samsung but for Motorola it is not. Is there any way by which I can make it work for Motorola? Any feedback will be highly appreciated.
Often the problem is a difference in minimum focal distance. That is, if the Motorola device can't focus as closely, then widening the rectangle may make the user hold the barcode so close as to be too close to focus. I would look at this first.
Otherwise you're looking at improving the image processing for this case. The challenge is that the app does simple thresholding, which works well in common cases. It falls down when you have dense 1D barcodes whose bar width nears 1 pixel. Because each pixel is either black or white you lose proportionally a lot of detail about exactly where the bars are.
If that's really the issue you could look at rewriting your app to use a full-resolution capture from the camera, instead of preview. In normal cases, more resolution doesn't help; in these cases it might. You would not be able to have a continuous-scan app this way.
I am one of the Barcode Scanner devs, and maintain a (for-pay) enhanced version called Barcode Scanner+. It has a different image processing algorithm that finds boundaries at sub-pixel resolution, which works better for codes like these. You may want to see how it does -- and if that works well, at least that tells you the kind of approach that works better. I can't send you that code but can describe what it does, if you want to investigate that sort of image processing.