I am working on augmented image example in arcore where I am able to detect the image but the image is not getting tracked and the object is not getting placed.I am referring augmented image example from codelabs. I have changed the image (hand made image), whose arcoreimg score in 100 and also done following changes to the code. It's getting detected continuously but not tracked.
config.setUpdateMode(Config.UpdateMode.LATEST_CAMERA_IMAGE);
config.setFocusMode(Config.FocusMode.AUTO);
For successive detection and tracking of Augmented Images in ARCore follow these basic rules:
In ARCore 1.15+, if your image doesn't move (like a poster on a wall), you should attach a global anchor to the image to increase the tracking's stability.
The physical image has to occupy 1/4 of the camera feed.
The smallest image resolution should be 300 x 300 pixels.
You must track your image under appropriate lighting conditions. Barely-lit room is not good environment for AR user experience.
It's much better to specify an expected physical size of a tracked image. Additional metadata improves tracking performance, especially for large physical images (more than 75 cm in size).
When ARCore detected a desired image with no expected physical size specified, its tracking state will be automatically paused. For user it means that ARCore has recognised the image, but hasn't gathered enough data to estimate its location in 3D space. Do not use the image's pose and size estimates until the image's tracking state is tracking.
Augmented Images support .png and .jpeg. However, avoid heavy compression for .jpeg.
Use images with a high contrast content, it's no matter whether they are color or black-and-white.
Avoid images with repetitive patterns (like Polka dot) and sparse features.
Andy's answer is correct, but maybe insufficiently specific. I had this issue as well and as soon as I added an expected width in meters, it started working almost immediately.
Instead of augmentedImageDatabase.addImage(DEFAULT_IMAGE_NAME, augmentedImageBitmap);
Use augmentedImageDatabase.addImage(DEFAULT_IMAGE_NAME, augmentedImageBitmap, <width in meters>);
Then it'll start tracking almost as soon as it is detected and you won't have to deal with this paused shenanigans. Worked great for me with a 7cm image with a 95 score. It even works great with an image with a score of 40. 40 score image with a set width works better than 100 score image without a set width.
Related
I am working on an app where user needs to click an image of a document and upload and server will do the further processing of doing OCR and extracting specific data from it. OCR and data extraction completely depends on the image quality. So what is the best way, i can instruct user to take a near to perfect image of document at run-time.
Factors i am considering.
1) Light available while capturing image
(Can't use the light sensor because it is present on front side of phone whereas document will be on bottom side)
2) Document angle.
(Tried edge detection using openCV but not properly works on white document with light color background surface. Any Edge detection tutorial for run-time document detection over custom camera surface will also be a great help)
3) Distance from document.
Processing the image after capturing (Transforming the document to a rectangle using openCV) is not always giving a great result with OCR because of character distortion.
So please suggest some way to instruct the user to take a better image of straight document from a perfect distance and better lightening conditions. And also after capturing best way to judge the image quality.
If proportions of document is known, you can draw appropriate inner (for min document size) and outer (for max document size) bounding rectangles on preview (as shown on pict) and control that user positioned document within outer and over inner bounding rect. That is also helps to control right document angle. Also You can control tilt and orientation of smartphone as described in answers for this questions.
Then process preview image (or even its small copy) on-the-fly with lightweight quality estimation algorithms and inform user about low quality and its reason (e.g. low brightness, you can estimate brightness of outer rectangle region, for example, as described in this question) and, may be, block "take snapshot" control.
You can use classifier (Machine Learning techniques) to reject low quality images using various features which can differentiate between low and good quality or images captured from long distance by detecting the document in the image.
I am trying to do detection of LSB Steganography using real-time camera on mobile phone. So far i havent had much luck with detecting the LSB Steganography, whether on printed material or on the PC Screen.
I tried using OpenCV and do the conversion of each frame to RBG, and then read the bits from each pixel, but that never detects the steganography.
I also tried using the Camera functionality, and check onFrame whether pixel by pixel the starting string is recognized or not, so i can read the actual hidden data in the remaining pixels.
This provided few times positive result, but then the reading of the data was impossible.
Any suggestions how to approach this?
Little bit more information on the hidden data:
1. It is all over the image, and i know the algorithm works, since if i just read the exact image through Bitmap in the app, the steganography is detected and decoded, but when i try to use the camera no such luck.
2. It is in a grid, 8x5 pixels all over the image, so it is not that it is only on 1 specific area of the image, and it can not be detected in the camera view.
I can post some code as well if needed.
Thanks.
You still haven't clarified on the specifics of how you do it, but I assume you do some flavour of the following:
embed a secret in a digital image,
print this stego image or have it displayed on a pc, and
take a photograph of that and detect the embedded secret.
For all practical purposes, this can't work. LSB pixel embedding steganography is a very fragile technique. You require a perfect copy of the stego pixels images for extraction to work. Even a simple digital manipulation is enough to destroy your secret. Scaling, cropping and rotation are to name a few. Then you have to worry about the angle you take the photo and the ambient light. And we're not even touching upon the colours that are shown on a pc monitor or the printed photo.
The only reason you get positives for the starting sequence is because you use a short one and you're bound to be lucky. Assuming the photographed stego image results in random deviations for each pixel from its true value, you'll still get lucky sometimes. Imagine the first pixel had the value 250 and after photographed it's 248. Well, the LSB in both cases is still 0.
On top of that, some sequences are more likely to come up. In most photos neighbouring pixels are correlated, because the colour gradient is smooth. This means that if the top left of a photo is dark and the top right is bright, the colour will change slowly. For example, the first 4 pixels have the value 10, then the next few have 11, and so on. In terms of LSBs, you have the pattern 00001111 and as I've just explained, that's likely to come up fairly frequently regardless of what image you photograph out there.
The the iOS Mail app used to have a handy feature (I believe they removed it with the advent of Mail Drop) that would give you the option to select a scaled version of an image attachment. The great thing about this feature was that it would actually calculate the file size of each of the scaled images; shown below:
The server I am uploading images to has a small file size limit (10 MB), and I would like to emulate this functionality in order to prevent uploads that exceed this limit.
With the assumption that the image is not actually scaled down three times in order to determine the file size of each of the scaled images; how would I go about doing this?
I have not been able to find any information regarding some type of formula to calculate the file size of a scaled down image based on the size of the original image.
Given the delay i've always seen when I pop that dialog on older devices, which is a non-trivial delay, I challenge your assumption that Apple isn't just doing the dirty-deed here -- ie. writing those JPGs directly to memory/disk and reading the size, rather than calculating this... ie. let data = UIImageJPEGRepresentation(image, 0.6)!
The trick is likely that people are falling down the "it needs to be at least 80% quality to be real!" hole. That's only true if you have a terrible JPG/media library. In reality, if you're writing a reasonably complex UIImage to memory/disk and you don't need transparency, then 60% is plenty.
My question is how to efficiently display large custom maps in an offline Phonegap application, allowing them to be panned and zoomed smoothly while still supporting older mobile devices?
I’m developing a mobile application that involves using geolocation to navigate walking routes in remote areas where it’s likely the user won’t have a signal and therefore an internet connection. It’s important that the app works well with Android 2.2+ (so SVG is not an option) as well as iOS4+.
I’ve drawn custom vector maps using Adobe Illustrator at resolutions appropriate to each route, the average being about 2000x2000 pixels and the largest of which so far results in an image 4000x2400 pixels.
I’ve chosen to go with Phonegap/JQM rather than native simply because I come from a web programming background and it seemed the fastest way to get a user interface up and running without needing to delve into native code too much, although I’ve written a couple of Phonegap plugins using native code for the purposes of power and screen management.
The application needs to allow the user to pan around the map (by dragging) and zoom in/out (by pinching) between about 25% to 200% of the original image size.
Most of the testing I’ve done has been on an HTC Desire running Android 2.3.3 and an HTC Wildfire running Android 2.2 since these are likely to be some of the lowest spec devices the app is going to have to run on.
I’ve tried out various approaches to display the map (detailed below), but so far each has proved unfit for purpose either because the memory usage of the app is too great, the storage space required makes the app too large to download or the CPU usage is too intensive causing lag when panning/zooming.
Any suggestions much appreciated. Thanks in advance.
Approaches I’ve tried:
1. Display map as raster PNG using tag
This was the first approach I tried. Exporting the 4000x2400 pixel image from Illustrator as a 128 colour PNG-8 resulted in a 746Kb file. I panned the image by absolutely positioning it relative to the viewport and zoomed the image by scaling the width/height attributes of the tag.
The problem with this approach was that even at a 1:1 zoom level, the Android application used 60Mb of RAM for the image and zooming in to 200% caused this to increase 120Mb, causing the app to crash on the HTC Wildfire.
2. Display portions of raster PNG using HTML5 canvas
To avoid the problem of zooming-in causing a proportional increase in memory usage, I tried loading the image via JS then copying the portion of the image to be displayed to a canvas the size of the viewport, something like:
var canvas = $(‘canvas#mycanvas’);
canvas.width = $(window).width;
canvas.height = $(window).height;
...
var img = new Image();
img.src = “map.png”;
...
var context = canvas[0].getContext("2d");
context.drawImage(img, x, y, w, h, 0, 0, canvas.width, canvas.height);
where x,y is the top-left corner within the source image defined by panning
and w,h is the area size within the source image determined by zoom level
The problem here was that large map images were somehow losing quality while in memory (I can only assume there’s some upper memory limit which is resulting in dithering), causing the maps to look distorted in the app: see here for an example screenshot
3. Display map as vector using HTML5 canvas
A bit of Googling led me to discover ai2canvas, an Illustrator plugin that enables you to export artwork as vectors displayed in an HTML5 canvas. The result of the export is an html file containing a chunk of JS which represents all the paths in illustrator as bezier curves. Exporting my 4000x2400 map resulted in a 550Kb html file containing the vector paths.
In my test app, I rendered the entire map to an in-memory canvas (not attached to the DOM) of 4000x2400 pixels, then copied the relevant portions of it to a viewport-sized canvas using context.drawImage() with the in-memory canvas as the source.
On the HTC Wildfire, although the initial render of all the bezier curves to the in-memory canvas took around 2000ms, copying between canvases was fast enough to allow smooth panning and zooming. The problem was when I looked at the memory usage of the app, it was using 120Mb for the in-memory canvas once all the vectors had rendered.
I tried a second approach using the vector map; instead of rendering all the vectors to a large in-memory canvas, I made the app calculate which vector paths were visible within the viewport at the current pan position/zoom level during each drag/pinch event and only draw the visible vectors to the viewport-sized canvas. While this reduced the required memory usage to 10Mb, the CPU cycles required to perform these calculations on every drag/pinch cycle made the app lag so much on the old android phones it was unusable.
4. Display map using offline tiling
Using map tiler, I created PNG tiles for my maps at zoom levels from 25% to 100%. In my test app, I was then able to lazy load the tiles on demand reducing memory usage and producing a smooth pan/zoom experience even on the HTC Wildfire. I thought I’d found the solution until I looked at the size of the APK produced: for my 4000x2400 map, map tiler produced 4Mb of tile images. Most of my maps are around 2000x2000 pixels, resulting in about 2Mb of tiles. The code of my proper application plus the Phonegap overhead is another 2Mb.
My intention is to release a series of apps available on the Android/Apple markets, each with a set of around 10 maps, but with tiling each map weighs in at between 1-4Mb so the resulting app becomes a very large download.
In case this is of interest to anyone else, I solved this by using map tiling in the end, using a tool called pnqnq to create 8-bit PNGs constrained to 256 colours. The resulting set of tiles for my 4000x2000 map was about 800K in size as opposed to 4Mb for PNG-24, which was an acceptable size for assets in my Android and iOS applications.
My current project is about android image processing.But if my phone camera is about 1-2 megapixel, will it affect the result of preprocessing like grayscale and binarization?
Your phone camera won't affect any pre-processing you perform in that your pre-processing code will act just the same regardless of the number of megapixels in your camera. Garbage in, garbage out still does apply. If you start with a low quality, poor contrast, blurred picture, you aren't going to be able to turn it in to something fantastic you want to hang on your wall. Additionally, as Mizuki alluded to in his comment, a 1-2 megapixel phone image is far higher resolution than the average image used on the internet, and these can be binarised and greyscaled just fine.
As for the two methods of preprocessing you mentioned in your question:
Binarization
This just converts an image into a two colour version. Normally black and white, though other colours are possible. The number of pixels in the image doesn't matter for this, other than it taking longer if it has more pixels to process. Low quality mobile phone cameras can sometimes produce low contrast photos and this may make it harder for the binarization algorithm to correctly determine the threshold at which pixels should be displayed in either colour.
Greyscale
Converting an image to greyscale is done by manipulating the colours of each pixel so, again, the number of pixels should only increase the preprocessing time, not change the result.