I'm working on a novelty detection project (in Android) using background subtraction. I would like to do simple classification after an object is detected (e.g. Person, Dog, Car, Motorbike). I have read some literature on how to do this using e.g. Bag Of Words, Cascade classifier, but I'm still not sure how to approach the task. Essentially, I want to store the novelty as an image and then label it into some sort of class - like Person, Dog, etc..or even match against a specific person, for example. The number of classes doesn't have to be too large - 2 or 3 even. So my questions are:
1) What's the best/simplest approach to do this? Cascade/Bag of Words/other methods?
2) How can I transport this classifier to my Android app, so that after I store the novelty as an image, I can just pass it to the classifier, which then applies a label. I know Cascade classifiers have associated xml files, but is this the only way?
Sorry if the questions seem trivial, but I've had a difficult time finding literature online.
Related
I've tried several ways to create android app to classify (with Tensorflow) that image on camera view IS document of my type OR NOT.
I've decided to create my custom model for this classification task.
First I try to use CustomVision helper, I've created 'model.pb' file, train it on 100 correct images of my document and 50 images on this document with mistakes (I know that its very small number of images, but that's all i have at the moment). On the output I have 'model.pb' and 'labels'('correct' and 'invalid') files. I put it in android example (custom vision sample) and it work very sadly: app always says that all I'm seeing in the camera screen (peoples, desks, windows, nature...) - is my CORRECT document label. Only sometimes, if I catch document with wrong stamps in the camera screen I've got INVALID label.
So i've decided to use more complex model and simple re-train it.
I've used this tutorial to get model and train it (Tensorflow for Poets codelab). But the situation is the same: all in camera view detecting such as 'correct' and sometimes (when i put camera on document with wrong angle or not full document) - 'invalid'
SO MY QUESTION IS:
What I'm doing in concept way wrong? Maybe I train models in wrong way? Or maybe tensorflow models couldn't be used to goal os detection documents on screen?
My requirement is to scan a fixed object. After recognizing that, I want to highlight the object and to display corresponding pre-feeded parameters accordingly, like height, width, circumference, etc.
This all, I want to do, without internet, using camera only.
Please, let me know if any solution / suggestion for this.
I have seen CraftAR SDK. It seems working as per my requirement, in order to recognize object, but it uses its server for storing images, which I don't want. As I want the static image, to be stored in app itself.
Try using the TensorFlow Object Detection API. Link: TensorFlow Object Detection API
And you can then customize your overall app behaviour accordingly, managing all your requirements (like for eg. showing a pop up with all the details of the object that's being detected after receiving some kind of callback when using the Tensoflow Object Detection API after the object has been detected successfully) as well as I believe that you can customise the TensorFlow object detection scenario part as per your need (Here, I am talking about the UI related part specifically in case of how you want your app to detect the object graphically).
To answer about the details like how it works offline and the resulting overall APK size etc. can be better understood from the links given below:
1] Step by Step TensorFlow Object Detection API Tutorial — Part 1: Selecting a Model
2] How to train your own Object Detector with TensorFlow’s Object Detector API
As an overview, for detecting the objects offline you have to be limited (just to reduce your APK size) with your own set of data/objects (as you have mentioned that you have got a fixed object for detection, that's good) and then you have to train (can be trained locally as well as on cloud) this set of objects using a SSD-Mobilenet model. Then you will have your own trained model (in simpler words) of those set of objects which will give you a retrained_graph.pb file (this goes into your assets folder for your android project) which is the final outcome that includes the trick (in simpler words) to detect and classify the camera frames in real time thereby displaying the results (or object details) as per the info (or the set of data/objects) provided; without the need of any sort of internet connection. For instance, TF Detect can track objects (from 80 categories) in the camera preview in real-time.
For further reference follow these links:
1] Google Inception Model
2] Tensorflow Object Detection API Models
3] Speed/Accuracy Trade-offs for Modern Convolutional Object Detectors
Also you can optimize (or compress) the retrained_graph.pb to optimized_graph.pb as this is the only major thing that would increase your APK file size. Long ago, when I tried detecting 5 different objects (using TF Classify), each object's folder was having about 650 photographs and the overall size of all the 5 folders (together) was about 230 mb and my retrained_graph.pb size was only 5.5 mb (which can further be optimized to optimized_graph.pb reducing its size even more).
For to start learning it from the beginner's level I would suggest you to once go through these codelab links and understand the working of these two projects as I too did so.
1] TensorFlow For Poets
2] TensorFlow For Poets 2: Optimize for Mobile
Wishing you good luck.
The below link to TensorFlow GitHub (Master) includes almost everything:
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android
I am writing an Android game in Kotlin where the board changes according to a specific pattern - which pattern, depends on the level the user currently plays.
I need a way to use many different patterns (20, 30 at max) in my code, deciding which one of them to use at runtime.
I thought of encoding each pattern as a string, putting all these strings in a file and at runtime loading it and parsing the desired one. However, the patterns aren't so simple, so parsing will be a complicated process. It also seems as an over-complication.
My best idea right now is writing a class for each distinct pattern (and a common parent abstract class to be used by the calling entities). Each class will have a single "apply" method that applies that specific pattern on the board.
However, it means dozens of classes (I could put them in a different folder so they don't make the main code folder too crowded), and a big switch case which maps the pattern id (received from the level manager) to the specific implementation. I'm pretty sure I don't want that.
Any better ideas? Thanks in advance.
It depends on what may happen in future. Will there be more patterns? Are they have parts which are common? Will there be other rules, say if you have used pattern 1 then you cannot go to pattern 1 again next time and so on?
Either way you will have a dozen of patterns. Storing them as a string, object, file, model or whatever is totally up to you but you will not escape from having them. What i mean here is having 1 file with 20 patterns is not very different from having 20 files each with 1 pattern. So no, having many classes is not something that is wrong in that case.
Put them in a folder and create a facade. That way all you need to do to use them is call the facade and get apply method based on some criteria(pattern id):
The facade itself can either map id to behavior or directly checks if there is file(class) named that way in the folder. If there is one it will get instance of it and call apply method on it as a result. You can store the instance so:
pros:
can be changed at any moment and behavior will change
there will be only 1 way to present pattern ( no option to get 2 patterns trying to draw them self's )
cons:
if some patterns at some point need to have one more method you need to change all of the classes and the facade
That way you don't need switch and there is no need to change the facade itself if new pattern shows up - just create new file with its own implementation and you are ready to go.
The question is not that ideal for SO, however, I found it really interesting. Hence I am putting some of my ideas if I had to design such game.
This is a level based game which incurs different board patterns in different levels. So it is really important that how you are designing your patterns to be translated into the board. Your pattern may have some generic keywords which can be translated into a program to create a board part by part. Let us look into an example, for making the idea clearer.
Suppose, you are building a line of pipes. Each part is being connected with the already constructed pipe. There can be a lot of differently shaped pipes in your hand. So while building a pattern, you just name each shape. For example, left-round-vertical-up, right-round-vertical-up, straight-horizontal, straight-vertical etc. You have a Factory class which knows the implementation of each of your shape. Now its pretty simple to store the whole pattern in your local database table. Which will be translated into your board in runtime, based on the logic your Factory class has.
A sample row in your database table may look like the following.
id level_number level_passed pattern_desc
-- ------------ ------------ ------------
1 1 1 left-round-vertical-up, straight-horizontal, straight-vertical, right-round-vertical-up
2 1 0 straight-horizontal, straight-vertical, right-round-vertical-up
So when you have the data in the structure above in your database and you know all the keywords of your pipe segments to be translated in your board, its simpler for you to maintain the code and to add new patterns via API call.
In your current structure, its difficult to update your patterns without any application update. However, in the proposed architecture, you can easily add new patterns in different levels using a simple API call from your server. The application knows how to parse the pattern and can show them accordingly. Then your job is to call an API to get the newly introduced patterns from your server and insert them into your table for storing patterns with appropriate values.
The factory implementation might require many classes, which indicates each shape of your pipe segment. However, you are not writing classes for each of your patterns which is quite a lot and difficult to manage further.
Hope that helps!
I'm developing an android app in which we're going to display some mathematics and physics question with multiple-choice answers.
The text of the questions are ready on a Microsoft Office Word .docx format.
The text of the questions usually contains formulas and equations and we want to save them in a SQLite database and access to it on android app.
The real problem is how to manage the displaying part, since to the best of my knowledge, we should manage such complex text with html tags.
There are actually about at least 2000 of these questions and we're after an optimized solutions for the problem.
and I'm a tough guy ! :) just give me some keywords and I'll go get it done.
So, please share your experiences and suggestions.
An easy solution, mentioned in the comments, would be to use MathJAX.
An alternative, if you only have about 2000 formulas, would be to set up TeX on your local machine, generate the formulas, and convert into individual png images.
You could also use HTML+Unicode directly if the formulas are simple.
Background:
My CSS360 group is attempting to create an Android application that will include an auto-complete search feature. The data that we'll be searching consists of around 7000 entries, and will be stored in a SQLite database on the phone itself. The most obvious approach would be to do a linear search of the database following every character that the user types, and then return a list of suggestions which are potential alphabetic extensions of the user's query. However, this seems like it would be pretty inefficient, and we've been looking for better alternatives. In another one of my classes today, my instructor briefly discussed the trie data structure, and mentioned that it's often used to store entire dictionaries. Entries into a trie can be retrieved in logarithmic time (as opposed to linear time for a regular old array), so this seems like a great tool for us to use! Unfortunately, we're in waaaay over our heads on this project already, and none of us really have a clue how to make this happen. All any of us have ever coded to date are basic console applications to teach us programming basics. We're all attempting to learn the Android platform in a week's time by watching YouTube videos, and differing the database stuff to the one guy in our group who has any SQL experience whatsoever. We could seriously use some pointers!
Questions:
When creating a trie, is it possible to have the entire structure pre-populated? IE: generate a line of code for every node used, so that the entire structure will already be in memory when the program starts? My thinking here is that this will save us the overhead of having to regenerate the entire trie from the database every time the program starts. If so, is there an easy way to get these thousands of lines of code into our program? IE: Some sort of script which converts the database files into a giant text file of java commands which can be copied and pasted into Eclipse?
Will there be a considerable amount of overhead if we search the database directly instead of using some sort of internal list or data structure? Should we be copying the names out of the database and searching them inside the program for our auto-complete function?
If this proves too technically difficult for us, and we have to resort to a regular linear search, will the performance be noticeably affected?
Our current plans are to run the auto-complete function each time the user enters a character, and then wait for the function to return before allowing them to continue typing. The only programs any of us have written so far function synchronously like this. What would we need to know to make this function asynchronously? Considering our novice abilities, and the requirements that we're already having to meet, would this be too technically challenging for us?
sqlite should be able to serve this auto-complete functionality reasonably well. I'd recommend using their internal indexes over re-implementing the wheel. If you need to do the latter, then sqlite is probably not going to help you after you've done that work.
If you want substring searching, then full text search is probably your best bet.
If you only want to complete the beginning of the word, then just using their vanilla indexes should be more than enough. If performance is a problem, then just wait until they type three characters before doing the query. Set a limit on your results for snappy responses.