Android Shape Recognition on Screen - android

I want to recognize shapes like a circle,triangle and rectangle which is drawn on screen.My main aim is a user draws a shape on screen and I need a code to recognize this shape.How should i approach this problem?

What you are trying to achieve can be quite tricky, but I happened to implement something similar a while ago, and here is the approach that I used:
stick to black & white drawings
have a smallish database of (black & white) drawings (50 or so) with a fixed resolution, let's say 256x256 (you can store them in sqlite as binary blobs if you wish). Make sure that you use decently thick lines for these drawings (10 px should be OK, or something about twice as thick as the user's input drawing). Also, the drawings should be normalized, meaning that they must have at least one of their dimensions as large as the image itself.
extract the shape drawn by the user and process it:
a) if it has an aspect ratio close to a square, then simply crop the white space around it and enlarge it such that it has the same size as your database images
b) Otherwise, it will most likely have one dimension about two times larger than the other one, in which case you crop the white space, rotate it to have the height as it's biggest dimension, enlarge it to 256x128 and then add on both sides 64 px of white space.
you'll have to compare your drawing with each of your database images pixel by pixel and determine the amount of black pixels which overlap for each database image. Then you sort these numbers and you'll get the best match. Even if the best match has less than 20% overlapping pixels, the results are usually good.
Because some shapes can be considered the same, even if they are rotated (imagine various ways to place a triangle in an image: one tip pointing up, or down, or towards one side etc), you'll probably want to rotate your input drawing around 12 - 24 times (by 15 - 30 degrees at each step) and compare each rotation to every image in your database. Given that this step will most likely require a lot of processing power, you might consider storing all the rotations of your initial database drawings in the database, as different pictures, thus making the database bigger, but saving you the effort of rotating the input image, which is costly.
Given that the above algorithm is a bit of a resource hog, you might consider having a server somewhere, which can do the actual comparisons, especially if you want to add many images to your database. Since I already implemented this algorithm for a demo application, I can already tell you that you're going to have to do a lot of pixel operations. Also, rotating images with the Android SDK can be annoying, because it changes the image dimensions...
If you are feeling adventurous, here are a couple of papers describing state of the art algorithms for tackling this problem: "Shape contexts enable efficient retrieval of similar shapes" by Greg Mori, Serge Belongie and Jitendra Malik (2001) and "Shape Matching: Similarity Measures and Algorithms" by Remco C. Veltkamp (2001). The maths might be a bit heavy, though.

You should look into GestureOverlayView.
A good tutorial is: http://www.vogella.com/articles/AndroidGestures/article.html

Related

scaling images in libgdx only once

In my android game, I am using images of fixed resolution lets say 256x256. Now for different device screens, I am rendering them by calculating their width and height as appropriate for that device.
Assume that on galaxy note2 I calculated width=128 and height=128... similarly for different devices, width and height will vary.
This is how I created texture..
....
imageTexture = new Texture(...);
....
in render()..
....
spriteBatch.draw(imageTexture,x,y,width,height);
....
So, every time when I call draw() method, does libgdx/opengl scale image from 256x256 to 128x128, which I think, yes!
Is there any way to tell opengl/libgdx to calculate all scaling only once ?
I have no idea how images were rendered? loaded into memory? scaled etc ?
How does Sprite in libgdx work? I tried understanding the code of Sprite and looks to me like they are also getting image width and height and then scale it every time, even though they have setScale() method.
First rule of optimizing: get some numbers. Premature optimization is the root of many problems. That said, there are still some good rules of thumb to know.
The texture data will be uploaded by libgdx/OpenGL to the GPU when you invoke new Texture. When you actually draw the texture with spriteBatch.draw instructions are uploaded to the GPU by OpenGL that tell the hardware to use your existing texture and to fit it to the bounds. The draw call just uploads coordinates (the corners of the box that defines the Sprite) and a pointer to the texture. The actual texture data is not uploaded.
So, in practice your image is "scaled" on every frame. However, this is not that bad, as this is exactly what GPUs are designed to do very, very well. You only really need to worry about uploading so many textures that the GPU has trouble keeping track of them all, you do not need to worry much about scaling the textures beforehand.
The costs of scaling and transforming the four corners of the sprite are relatively trivial next to the costs of sending the data to the GPU and the cost of refreshing the screen, so they probably are not worth worrying about too much. The "batch" in SpriteBatch is all about "batching up" (or gathering together) a lot of coordinates to send up to the GPU at once, as roughly, each call out to the GPU can be expensive. So, its always good to do as much work within a single batch's begin/end as you can.
Again, though, modern machines are stupidly fast, and you should be able to do whatever is easiest to get your app running first. Then once you have something working correctly, you can figure out which parts are actually slow and fix those. The parts that are "inefficient" but are not actually measurably impacting your application can be left alone.

How to break the image into shattered shapes in android

Am trying to break image in shattered pieces, but am unable to catch the logic, please give me way how to achieve.
I hope the below image can give my idea, what I want, Breaking the bitmap into a shattered pieces like triangle or any shape. later i will shuffle those bitmap shapes and giving puzzle to enduser rearrange them in order.
OK, if you want to rearrange the pieces (like in a jigsaw) then each triangle/polygon will have to appear in a rectangular bitmap with a transparent background, because that's how drawing bitmaps works in Java/Android (and most other environments).
There is a way to do this sort of masking in Android, its called porter-duff compositing. The Android documentation is poor to non-existent, but there are many articles on its use in Java.
Basically you create a rectangular transparent bitmap just large enough to hold your cut-out. Then you draw onto this bitmap a filled triangle (with transparency non-zero) representing the cut-out. It can be any colour you like. Then draw the cutout on top of the source image at the correct location using the Porter-Duff mode which copies the transparency data but not the RGB data. You will be left with your cutout against a transparent background.
This is much easier if you make the cutout bitmap the same size as the source image. I would recommend getting this working first. The downsides of this are twofold. Firstly you will be moving around large bitmaps to move around small cutouts, so the UI will be slower. Secondly you will use a lot of memory for bitmaps, and on some versions of Android you may well run out of memory.
But once you have it working for bitmaps the same size as the source image, it should be pretty straightforward to change it to work for smaller bitmaps. Most of your "mucking about" will be in finding and using the correct Porter-Duff mode. As there are only 16 of them, its no great effort to try them all and see what they do. And they may suggest other puzzle ideas.
I note your cutout sections are all polygons. With only a tiny amount of extra complexity, you could make them any shape you like, including looking like regular jigsaw pieces. To do this, use the Path class to define the shapes used for cutouts. The Path class works fine with Porter-Duff compositing, allowing cutouts of almost any shape you can imagine. I use this extensively in one of my apps.
I am not sure what puzzle game you are trying to make, but if there is no special requirements of the shattered pieces,
only the total number of them which can span the whole rectangle, you may try doing the following steps,
the idea is basically by knowing that n non-intersecting lines with two end points lie on any of the 4 edges of the rectangle, n+1 disjoint areas is formed.
Create an array and store the line information
For n times, you randomly pick two end points which lie on those 4 edges of the rectangle
2a. Try to join these two points: start from either end point, if you get an intersection with another line you drew before, stop at the intersection, otherwise stop at the other end point
You will get n+1 disjoint areas with n lines drawn
You may constrain your lines choosing if you have some special requirements of the areas.
For implementation details, you may want to have a look of dot product and euler's theorem

Android - is there a possibility to make infinite canvas?

Currently I am doing app allowing user to draw. Simple think, just extend Canvas class and most of the thing is done.
That was my initial thinking and idea. But as the canvas is rather small because this is only what user see on the screen there is not much possible space to draw. Going through documentation I found translate() method allowing me to move canvas. What I did find out is when I move it, there is some kind of blank space just as you would move piece of paper. I understand that this is totally normal, as I've said before - canvas is only "the screen".
My question is - is there a possibility to make something like infinite canvas so you can make a huge painting and move everything around?
Before this question I was thinking about two things how something like this can be done:
Move all objects on canvas simultaneously - bad idea, because if you have a lot of them then the speed of moving is very bad.
Do something similar as it is done in ListView when you move it (or better see on the screen) only views that are on the screen together with one before and one after are loaded to memory and rest is uploaded dynamically when needed. I think this is the best option to achieve this goal.
EDIT:
Question/answer given by Kai showed me that it is worth to edit my question to clarify some of the things.
Basic assumptions about what can be done by user:
User is given opportunity to draw only circles and rectangles with some (around 80%) having drawable (bitmap) on them on canvas.
I assume that on all screens there will be maximum 500-800 rectangles or circles.
First of all thinking about infinity I was thinking about quite big number of screens - at least 30 on zoom 1x in each side. I just need to give my users bigger freedom in what they are doing.
On this screen everything can be done as on normal - draw, scale (TouchListener, ScaleListener, DoubleTapListener). When talking about scaling, there is another thing that has to be concerned and connected with "infinity" of canvas. When user is zooming out then screens, or more precise objects on the invisible "neighbours" should appear with proper scaling as you would zoom out camera in real life.
The other thing that I've just realised is possibility of drawing at small zoom level - that is on two or three screens and then zooming in - I suppose it should cut and recalculate it as a smaller part.
I would like to support devices at least from API 10 and not only high-end.
The question about time is the most crucial. I want everything to be as smooth as possible, so user wouldn't know that new canvas is being created each time.
I think it really depends on a number of things:
The complexity of this "infinite canvas": how "infinite" would it really be, what operations can be done on it, etc
The devices that you want to support
The amount of time/resource you wish to spend on it
If there are really not that many objects/commands to be drawn and you don't plan to support older/lower end phones, then you can get away with just draw everything. The gfx system would do the checking and only draws what would actually be shown, so you only waste some time to send commands pass JNI boundary to the gfx system and the associated rect check.
If you decided that you needs a more efficient method, you can store all the gfx objects' positions in 4 tree structures, so when you search the upper-left/upper-right/lower-left/lower-right "window" that the screen should show, it'll fast to find the gfx objects that intersects this window and then only draw those.
[Edit]
First of all thinking about infinity I was thinking about quite big
number of screens - at least 30 on zoom 1x in each side. I just need
to give my users bigger freedom in what they are doing.
If you just story the relative position of canvas objects, there's practically no limit on the size of your canvas, but may have to provide a button to take users to some point on canvas that they are familiar lest they got themselves lost.
When talking about scaling, there is another thing that has to be
concerned and connected with "infinity" of canvas. When user is
zooming out then screens, or more precise objects on the invisible
"neighbours" should appear with proper scaling as you would zoom out
camera in real life.
If you store canvas objects in a "virtual space", and using a "translation factor" to translate objects from virtual space to screen space then things like zoom-in/out would be quite trivial, something like
screenObj.left=obj.left*transFactor-offsetX;
screenObj.right=obj.right*transFactor-offsetX;
screenObj.top=obj.top*transFactor-offsetY;
screenObj.bottom=obj.bottom*transFactor-offsetY;
//draw screenObj
As an example here's a screenshot of my movie-booking app:
The lower window shows all the seats of a movie theater, and the upper window is a zoomed-in view of the same theater. They are implemented as two instances of the same SurfaceView class, besides user input handling, the only difference is that the upper one applies the above-mentioned "translation factor".
I assume that on all screens there will be maximum 500-800 rectangles
or circles.
It is actually not too bad. Reading your edit, I think a potentially bigger issue would be if an user adds a large number of objects to the same portion of your canvas. Then it wouldn't matter if you only draw the objects that are actually shown and nothing else - you'd still get bad FPS since the GPU's fill-rate is saturated.
So there are actually two potential sources of issues:
Too many draw commands (if drawing everything on canvas instead of just drawing visible ones)
Too many large objects in the same part of the screen (eats up GPU fill-rate)
The two issues requires very different strategy (1st one using tree structures to sort objects, 2nd one using dynamically generated Bitmap cache). Since how users use your app are likely to different than how you envisioned it to be, I would strongly recommend implementing the functions without the above optimizations, try to get as many people as possible to do testing, and then apply optimizations to each of the bottlenecks you encounter until the satisfactory performance is achieved.
[Edit 2]
Actually with just 500~800 objects, you can just calculate the position of all the objects, and then check to see if they are visible on screen, you don't even really need to use some fancy data structures like a tree with its own overheads.

Best Strategy for Storing Handwriting

I am writing a mobile app (Android) that will allow the user to 'write' to a canvas using a single-touch device with 1 pixel accuracy. The app will be running on a tablet device that will be approximately standard 8 1/2" x 11" size. My strategy is to store the 'text' as vector data, in that each stroke of the input device will essentially be a vector consisting of a start point, and end point, and some number of intermediate points that help to define the shape of the vector (generated by the touchscreen/OS on touch movement). This should allow me to keep track of the order that the strokes were put down (to support undo, etc) and be flexible enough to allow this text to be re-sized, etc like any other vector graphic.
However, doing some very rough back of the envelope calculations, with a highly accurate input device and a large screen such that you can emulate on a one for one basis the standard paper notepad, that means you will have ~1,700 strokes per full page of text. Figuring, worst-case, that each stroke could be composed of up to ~20-30 individual points (a point for every pixel or so of the stroke), that means ~50,000 data points per page... WAY too big for SQLite/Android to handle with any expectation of reliability when a page is being reloaded and the vector strokes are being recreated (I have to imagine that pulling 50,000+ results from the SQLite db will exceed the CursorWindow limit of 1Mb)
I know I could break up the data retrieval into multiple queries, or could modify the stroke data such that I only add an intermediate point to help define the stroke vector shape if it is more than X pixels from a start, finish or other intermediate pixel, but I am wondering if I need to rethink this strategy from the ground up...
Any suggestions on how to tackle this problem in a more efficient way?
Thanks!
Paul
Is there any reason of using vector data in the first place? Without knowing your other requirements, it seems to me that you just need to store the data in raster / bitmap and compress it with regular compression methods such as PNG / zip / djvu (or if performance suffers, simple things like run-length-encoding / RLE).
EDIT: sorry I didn't read the question clearly. However if you only need things like "undo" and "resize", you can take a snapshot of the bitmap for every stroke (of course you only need to take a snapshot of the regions that change).
Also it might be possible to take a hybrid approach where you display a snapshot bitmap first while waiting for the (real) vector images to load.
Furthermore, I am not familiar about the android cursor limit, but SQL queries can always be rewritten to split the result in pieces (via LIMIT... OFFSET).
Solution I am using for now, although I would be open to any further suggestion!
Create a canvas View that can both convert SVG paths to Android paths, and can intercept motion events, converting them to android Paths while also storing them as SVG paths.
Display the Android Paths to the screen in onDraw()
Write the SVG Paths out to an .svg file
Paul

Android scrolling background of bitmap tiles

I'm tried to determine the "best" way to scroll a background comprised of tiled Bitmaps on an Android SurfaceView. I've actually been successful in doing so, but wanted to determine if there is a more efficient technique, or if my technique might not work on all Android phones.
Basically, I create a new, mutable Bitmap to be slightly larger than the dimensions of my SurfaceView. Specifically, my Bitmap accomodates an extra line of tiles on the top, bottom, left, and right. I create a canvas around my new bitmap, and draw my bitmap tiles to it. Then, I can scroll up to a tile in any direction simply by drawing a "Surfaceview-sized" subset of my background Bitmap to the SurfaceHolder's canvas.
My questions are:
Is there a better bit blit technique than drawing a background bitmap to the canvas of my SurfaceHolder?
What is the best course of action when I scroll to the edge of my background bitmap, and wish to shift the map one tile length?
As I see it, my options are to:
a. Redraw all the tiles in my background individually, shifted a tile length in one direction. (This strikes me as being inefficient, as it would entail many small Bitmap draws).
b. Simply make the background bitmap so large that it will encompass the entire scrolling world. (This could require an extremely large bitmap, yet it would only need to be created once.)
c. Copy the background bitmap, draw it onto itself but shifted a tile length in the direction we are scrolling, and draw the newly revealed row or column of tiles with a few individual bitmap draws. (Here I am making the assumption that one large bitmap draw is more efficient than multiple small ones covering the same expanse.)
Thank you for reading all this, and I would be most grateful for any advice.
I originally used a similar technique to you in my 'Box Fox' platformer game and RTS, but found it caused quite noticeable delays if you scroll enough that the bitmap needs to be redrawn.
My current method these games is similar to your Option C. I draw my tiled map layers onto a grid of big bitmaps (about 7x7) taking up an area larger than the screen. When the user scrolls onto the edge of this grid, I shift all the bitmaps in the grid over (moving the end bitmaps to the front), change the offset of grid, and then just redraw the new edge.
I'm not quite sure which is faster with software rendering (your Option C or my current method). I think my method maybe faster if you ever change to OpenGL rendering as you wouldn't have to upload as much texture data to the graphics card as the user scrolls.
I wouldn't recommend Option A because, as you suggest, the hundreds small bitmap draws for a tiled map kills performance, and it gets pretty bad with larger screens. Option B may not even be possible with many devices, as it's quite easy to get a 'bitmap size exceeds VM budget' error as the heap space limit is set quite low on many phones.
Also if you don't need transparency on your map/background try to use RGB_565 bitmaps, as it's quite a lot faster to draw in software, and uses up less memory.
By the way, I get capped at 60fps on both my phone and 10" tablet in my RTS with the method above, rendered in software, and can scroll across the map smoothly. So you can definitely get some decent speed out of the android software renderer. I have a 2D OpenGL wrapper built for my game but haven't yet needed to switch to it.
My solution in a mapping app relies on a 2 level cache, first tile objects are created with a bitmap and a position, these are either stored on disk or in a Vector (synching is important for me, multithreaded HTTP comms all over the place).
When I need to draw the background I detect the visible area and get a list of all the tiles I need (this is heavily optimised as it gets called so often) then either pull the tiles from memory or load from disk. I get very reasonable performance even on slightly older phones and nice smooth scrolling with no hiccups.
As a caveat, I allow tiles not to be ready and swap them with a loading image, I don't know if this would work for you, but if you have all the tiles loaded in the APK you should be fine.
I think one efficent way to do this would be to use canvas.translate.
On the first drawing the entire canvas would have to be filled with tiles. New android phones can do this easily and quickly.
When the backround is scrolled I would use canvas.translate(scrollX, scrollY), then I would draw individualy one by one tile to fill the gaps, BUT, I would use
canvas.drawBitmap(tileImage[i], fromRect, toRect, null) which would only draw the parts of the tiles that are needed to be shown, by setting fromRect and toRect to correspond to scrollX and scrollY.
So all would be done by mathematics and no new bitmaps would be created for the background - save some memory.
EDIT:
However there is a problem using canvas.translate with surfaceView, because it is double buffered and canvas.translate will translate only one buffer but not the second one at the same time, so this alternating of buffers would have to be taken into account when depending on surfaceView to preserve the drawn image.
I am using your original method to draw a perspective scrolling background. I came up with this idea entirely by accident a few days ago while messing around with an easy technique to do a perspective scrolling star field simulation. The app can be found here: Aurora2D.apk
Just tilt your device or shake it to make the background scroll (excuse the 2 bouncing sprites - they are there to help me with an efficient method to display trails). Please let me know if you find a better way to do it, since I have coded several different methods over the years and this one seems to be superior. Simply mail me if you want to compare code.

Categories

Resources