I want to use QR code to get the smart phone's location (either UTM or Lat/Lon). Reading this article, it looks like it is possible to get the position of the smart phone. In addition, I want to render some 3D models on the camera screen. Is it possible? Actually I have no clue from where should I start.
Can anyone help me out regarding this?
Thanks.
If you read that article carefully, all it suggests to get the location of the phone, is to simply encode the lat/lon in the QR code itself. This will only work if the location of the displayed QR codes are fixed (e.g. a sticker on a wall rather than printed on a flyer).
Is it possible to render 3D models on a camera screen? Sure. It wouldn't be the default camera app, you'd have to make your own. It would involve a fair bit of math if you wanted to position the 3D model relative to the QR code. You'd probably try to build planes based on the sides of the squares.
Related
Before I make a giant word dump my effective question is this:
Can I supply some extra information / heuristics to ARCore to refine it's idea of what the pose of a detected augment image is? Or can I use the pose of other trackable objects to refine the the pose of a detected augment image?
For more info, here is some background information on my workflow:
My AR app revolves around overlaying various 3D CAD models on top of their real-world machine equivalents. The user interaction goes like this:
The user will adhere a QR code (sized .2 meters by .2 meters) to a predetermined location on the associated machine (location is specific to the type of machine).
The user will then load up the app, point the camera at the QR code and the app will pass the camera image to a QR code reading library and use the payload (an id for a specific machine) to retrieve the associated CAD Model & metadata.
Once the QR code is detected I can use the QR code reading library to construct a pristine image of the QR code and pass this image to ARCore so that it can detect it in 3D space from the camera.
Once the QR code is detected in 3D space, I attach an anchor and I use the knowledge of where the QR code should be placed on the given model (also retrieved from my database using the payload info) to determine a basis for my CAD model.
Information can be overlayed using the CAD model to show various operations / interactions.
Now I've got all this working pretty well but I've run into some issues where the model is never quite positioned exactly to the real-world equivalent and requires some manual positional adjustment after the fact to get things just right. I have some ideas for how to resolve this but I don't know how feasible any of these ideas are:
ArTrackable_acquireNewAnchor allows you to specify multiple anchors per trackable with different poses. I assume this will refine the tracking the object but I'm not clear as to how to use this API. I'm currently just passing the pose generated from ArAugmentedImage_getCenterPose so I don't know what other poses I would pass.
If I promote my QR Code anchor to a cloud anchor after detection will that aid in detecting / refining the qr pose detection in the future?
If I try and match other features detected by ARCore (like planes) to known topology in the real environment (like floors / walls) could better approximate the position of the QR code image or provide some heuristic to ARCore so that it can?
Instead of using a single QR code image what if I use a set of images (one QR code, and two other static images) that are slightly offset from each other. If we know how far apart these images are in the real world we can use this information to correct for the error in ARCore's estimation of where they are.
Sorry for the giant word dump, but I figured the more info the better. Any other ideas outside the framing of my question are also appreciated.
I'm building an Android app that has to identify, in realtime, a mark/pattern which will be on the four corners of a visiting card. I'm using a preview stream of the rear camera of the phone as input.
I want to overlay a small circle on the screen where the mark is present. This is similar to how reference dots will be shown on screen by a QR reader at the corner points of the QR code preview.
I'm aware about how to get the frames from camera using native Android SDK, but I have no clue about the processing which needs to be done and optimization for real time detection. I tried messing around with OpenCV and there seems to be a bit of lag in its preview frames.
So I'm trying to write a native algorithm usint raw pixel values from the frame. Is this advisable? The mark/pattern will always be the same in my case. Please guide me with the algorithm to use to find the pattern.
The below image shows my pattern along with some details (ratios) about the same (same as the one used in QR, but I'm having it at 4 corners instead of 3)
I think one approach is to find black and white pixels in the ratio mentioned below to detect the mark and find coordinates of its center, but I have no idea how to code it in Android. I looking forward for an optimized approach for real-time recognition and display.
Any help is much appreciated! Thanks
Detecting patterns on four corners of a visiting card:
Assuming background is white, you can simply try this method.
Needs to be done and optimization for real time detection:
Yes, you need OpenCV
Here is an example of real-time marker detection on Google Glass using OpenCV
In this example, image showing in tablet has delay (blutooth), Google Glass preview is much faster than that of tablet. But, still have lag.
I need to scan a special object within my android application.
I thought about using OpenCV but it is scanning all objects inside the view of the camera. I only need the camera to regognize a rectangular piece of paper.
How can i do that?
My first thought was: How do barcode scanners work? They are able to regognize the barcode area and automatically take a picture when the barcode is inside a predefined area of the screen and when its sharp. I guess it must be possible to transfer that to my problem (tell me if im wrong).
So step by step:
Open custom camera application
Scan objects inside the view of the camera
Recognize the rectangular piece of paper
If paper is inside a predefined area and sharp -> take a picture
I would combine this with audio. If the camera recognized the paper make some noice like a peep or something and the more the object is fitting the predefined area the faster the peep sound is played. That would make taking pictures for blind people possible.
Hope someone got ideas on that.
OpenCV is an image processing framework/library. It does not "scan all objects inside the view of the camera". By itself it does nothing and yet it gives the use of a number of useful functions, many of which could be used for your specified application.
If the image is not cluttered and nothing is on the paper, I would look into using edge detection (i.e. Canny or similar) or even colour blobs (even though colour is never a good idea, if your application is always for white uncovered paper, it should work robustly).
OpenCV does add some overhead, but it would allow you to quickly use functions for a simple solution.
I am looking for some kind of auto trim/crop functionality in android.
Which detects a object in captured image and creates a square box around object for
cropping. I have found face detection apis in android, but my problem is captured images are documents/pages not human faces so how can I detected documents or any other object from captured picture.
I am thinking of any algorithms for object detection or some color detection. Is there any apis or libraries available for it.
I have tried following link but not found any desired output.
Find and Crop relevant image area automatically (Java / Android)
https://github.com/biokys/cropimage
Any small hint would also help me alot. Please help. Thanks in advance
That depends on what you intend to capture and crop, but there are many ways to achieve this. Like littleimp suggested, you should use OpenCv for the effect.
I suggest you use edge-detection algorithms, such as Sobel, and perform image transformation on it with, for example, a Threshold function that will turn the image into a binary one (only black and white). Afterwards, you can search the image for the geometric shape you want, using what's suggested here. Filter the object you want by calculating the detected geometric figure's area and ratio.
It would help a lot to know what you're trying to detect in an image. Those methods I described were the ones I used for my specific case, which was developing an algorithm to detect and crop the license plate from a given vehicle image. It works close to perfect and it was all done by using OpenCV.
If you have anything else you'd like to know, don't hesitate to ask. I'm watching this post :)
Use OpenCV for android.
You can use the Watershed (Imgproc.watershed) function to segment the image into foreground and background. Then you can crop around the foreground (which will be the document).
The watershed algorithm needs some markers pre-defining the regions. You can for example assume the document to be in the middle of the image, so create a marked region in the middle of the image to get the watershed algorithm started.
i just need some guide on how to detect a marker and make an output text.. for ex: a marker with an image of a dog , when detected, i have an output text "DOG" in a textfield .. can someone help me with my idea? oh, btw which one is more effective to use nyartoolkit or andar for my idea?thanks:) need help..!
What you're looking for isn't augmented reality, it's object recognition. AR is chiefly concerned with presenting data overlaid on the the real world, so computation is devoted each frame to determining the position relative to the camera of the object. If you don't intent to use this data, AR libraries may be an inefficient. That said...
AR marker tracking libraries usually find markers by prominent features like corners, and can distinguish markers by binary patters encoded inside the marker, or in the marker's borders. If you're happy with having the "dog" part encoded in the border of a marker, there are libraries you can use like Qualcomm's AR development kit. This library, and Metaio's Unifeye mobile can also do natural feature tracking on pre-defined images. If you're happy with being able to recognize one specific image or images of dogs that you have defined in advance, either of these should be ok. You might have to manipulate your dog images to get good features they can identify and track. Natural objects can be problematic.
General object recognition (being able to recognize a picture of any dog, not known beforehand) is still a research topic. There are approaches, but they're mostly very computationally intensive, and most mobile solutions involve offloading the serious computation to a server. Recognition of simple outline sketches however is more tractable, there's a great paper called "Shape recognition and pose estimation for mobile augmented reality" (I can't find a copy online, but the IEEE link is here) that uses contours to identify objects - this is light enough to run on a mobile (and it's pure genius).