Saving ORB feature vectors using OpenCV4Android (java API) - android

I have a training set of images, for each of which I've detected and computed their feature vectors (using ORB feature descriptors and extractors. The questions is: since I need to save those features to reutilise them for matching against test images (using SVM classifier); what is the best way to store the feature vectors, locally on the Android device?
The feature vectors to be saved are of variable size per image, and are thus those with non-maximal sizes are padded with zeros to unify all vectors' sizes. The maximum size currently is 500 rows x 32 cols; thus 16k features.
here are the options I could reach so far;
I've heard of OpenCV's FileStorage, but when going through the java documentation, I noticed a save method for HOG features (not ORB). Furthermore, I'm not sure if saving features using OpenCV's file storage options would be most optimal memory-wise for Android phones, given that the xml file would be too large to load.
My current choice is to opt for a sqlLite database, having a table with two cols; id and feature (as frequently suggested online); to tabulate all the 16k features in sqlLite. That seems rather phone-storage intensive, but it's the most reasonable solution I can find.
Is there a common method to handling feature vectors on Android phones? Does it encompass any of the above methods; if not can you please offer some guidelines on how to implement such a storage solution?
Thank you.

In my opinion the most universal way to store the keypoints is to first convert them to a data-interchange format like JSON.
After you are able to do that conversion you have a lot of flexibility to store it. JSON is easily converted to a String and/or sent through a network connection.
With OpenCV C++ you are able to store data as YAML, but that is not available for Android yet.
To parse JSON in Java you can use this easy to use library Google GSON.
And here is my first attempt to do exactly that:
public static String keypointsToJson(MatOfKeyPoint mat){
if(mat!=null && !mat.empty()){
Gson gson = new Gson();
JsonArray jsonArr = new JsonArray();
KeyPoint[] array = mat.toArray();
for(int i=0; i<array.length; i++){
KeyPoint kp = array[i];
JsonObject obj = new JsonObject();
obj.addProperty("class_id", kp.class_id);
obj.addProperty("x", kp.pt.x);
obj.addProperty("y", kp.pt.y);
obj.addProperty("size", kp.size);
obj.addProperty("angle", kp.angle);
obj.addProperty("octave", kp.octave);
obj.addProperty("response", kp.response);
jsonArr.add(obj);
}
String json = gson.toJson(jsonArr);
return json;
}
return "{}";
}
public static MatOfKeyPoint keypointsFromJson(String json){
MatOfKeyPoint result = new MatOfKeyPoint();
JsonParser parser = new JsonParser();
JsonArray jsonArr = parser.parse(json).getAsJsonArray();
int size = jsonArr.size();
KeyPoint[] kpArray = new KeyPoint[size];
for(int i=0; i<size; i++){
KeyPoint kp = new KeyPoint();
JsonObject obj = (JsonObject) jsonArr.get(i);
Point point = new Point(
obj.get("x").getAsDouble(),
obj.get("y").getAsDouble()
);
kp.pt = point;
kp.class_id = obj.get("class_id").getAsInt();
kp.size = obj.get("size").getAsFloat();
kp.angle = obj.get("angle").getAsFloat();
kp.octave = obj.get("octave").getAsInt();
kp.response = obj.get("response").getAsFloat();
kpArray[i] = kp;
}
result.fromArray(kpArray);
return result;
}

I would suggest storing the feature vectors as images to have a simple and compact representation. You could even use non-destructive compression such as png to minimize file size.

I see that you have considered using the Android SQLite database:
My current choice is to opt for a sqlLite database, having a table with two cols; id and feature (as frequently suggested online); to tabulate all the 16k features in sqlLite. That seems rather phone-storage intensive, but it's the most reasonable solution I can find.
There is a way to save and retrieve MatOfKeyPoint to SQLite database with reasonable efficiency.
Using the database has the advantage of not needing to request write external storage permission from the user (although that permission might be needed for some other of your apps functions).
There is a complete Android solution, with Java code which can be found in this StackOverflow Answer.
The following is a description of what's going on in the code from that answer...
MatOfKeyPoint to byte[] and some attributes
To save to the database, you need to save to a byte[] object. Using the MatOfKeyPoint.get() method, you can get a populated float[]. Then using ByteBuffer.putFloat(), you can loop through all of your floats, finally getting a populated byte[] by using ByteBuffer.array().
You also need to save some attirbutes MatOfKeyPoint.rows(), MatOfKeyPoint.cols(), and MatOfKeyPoint.type() to your database, along with the blob of byte[].
Database blob (byte[]) to MatOfKeyPoint
To reconstitute your MatOfKeyPoint object from the database, first you make a float[] out of your blob. Use ByteBuffer.wrap(blob), then run ByteBuffer.asFloatBuffer(), and finally FloatBuffer.get() with a properly sized new float[].
Now that you have yourFloatArray you run MatOfKeyPoint.create(rows, cols, type), those three things coming from the database record. Finally you run MatOfKeyPoint.put(0, 0, yourFloatArray).

Related

I have a database on Parse.com where the values are strings. Is there a way I can split this one attribute into multiple?

I'm creating an app on android that helps users find an apartment around campus by letting you choose the complex, the building inside of that complex, and finally the apartment number. These are done in listviews first starting with a list of the complexes. After this, it opens up a list of the buildings etc.
All of the information is stored in Parse. I have a table called 'Parent' That has all of the complex information stored, and then a 'Children' table that stores the complex, building, a building color (if it has one) and an apartment number.
The app is already created on IOS by a friend of mine and now he wants me to make the android app. When the database was created, it was created in a very messy way. In the 'Children' table, there is a childKey that stores the complex, building, color and apartment number as a string separated only by spaces.
I'm looking for the simplest way to create multiple attributes from this one childKey. Here is an example of what I'm talking about.
Parent Table:
apartmentName = Campus West
apartmentKey = CW
Children Table:
childKey = CW T 10337 ('Parent apartmentKey', 'building', 'apartment#')
latitude = 12.345
longitude = 67.890
I want the Children Table to have:
apartmentKey = CW
building = T
apartment# = 10337
latitude = 12.345
longitude = 67.890
Any ideas?
Could you simply parse through the string data and create a JSONObject that would allow you to grab any specific data you might need for each "Children"?
Parse childKey String
String data = "CW T 10337 ('Parent apartmentKey', 'building', 'apartment#')";
String[] dataList = data.split(" "); // Make String array based on space
Create JSONObject
JSONObject childrenData = new JSONObject();
childrenData.put("apartment_key", dataList[0]);
childrenData.put("building", dataList[1]);
childrenData.put("apartment_no", dataList[2]);
// Not sure what the data in the () is as you do not state its use/where to save it
// Now add the longitude and latitude
childrenData.put("latitude", mLatitude);
childrenData.put("longitude", mLongitude);
You could then pass the JSONObject along and read from it
Read JSONObject
String apartmentKey = childrenData.getString("apartment_key");
String building = childrenData.getString("building");
int apartmentNumber = childrenData.getInt("apartment_no");
Or you could save the JSON object to the device and grab it later on if need be.
This might not be EXACTLY how you tend to implement it but I could see JSONObject being useful as it allows you to store multiple attributes via a key and a value. As long that you are consistent in the database childKey field then this should be fine. If not you could set up some additional logic to catch any special cases.
EDIT:
In case you haven't heard of/used JSONObjects before here is a quick layout of a JSON object:
{
"data": {
"array": [{"value1", 1}, {"value2", 2}]
"string": "value"
"int": 1
}
}
The first part of a JSON object is the key. In "int": 1, the "int" is the key and its value is 1.
Here's a good link about JSON

use stored JSON instead of Core Data

There are lots of tutorials out there describing how to fetch JSON objects from the web and map them to Core Data.
I'm currently working on an iOS (later: Android as well) app which loads json objects from web and displays them to the user. In my opinion all this mapping from and to Core Data is an overhead in this case, it would be much easier to save the JSON objects directly and use them as "cache" in the app. Are there libraries/documented ways how to achieve fetching json objects, save them locally and fetch them with a predefined identifier?
I would love to fetch e.g. 10 objects, show them to the user and save the data locally. The next time the user is on that list the local data is shown and in the background the json-file is fetched again to be up-to-date. I guess this is a common use case but I didn't find any tutorials/frameworks enabling exactly this.
You can simply use NSURLCache to cache http responses instead saving JSONs
http://nshipster.com/nsurlcache/
There are many ways to implement this. You can implement cache using either file storage or database depending on the complexity as well as quantity of your data. If you're using files, you just need to store JSON response and load it whenever activity/fragment is crated. What I have done sometimes is store the JSON response in the form of string in a file, and then retrieve it on activity/fragment load. Here's an example of reading and writing string files:
Writing files:
FileOutputStream outputStream = context.openFileOutput("myfilename",Context.MODE_PRIVATE);
String stringToBeSaved = myJSONObject.toString();
outputStream.write(stringToBeSaved.getBytes());
Reading from files
FileInputStream inputStream= context.openFileInput("myfilename");
int c;
String temp="";
while( (c = inputStream.read()) != -1){
temp = temp + Character.toString((char)c);
You can convert this string to JSONObject using :
JSONObject jsonObject = new JSONObject(temp);
Or you can use the string according to your needs.

Out of Memory while parsing data in JSON android?

Hi I am parsing data from JSON. While parsing data I am getting OUT OF MEMORY error and dalvik-heap. How to over come from this problem. and I search in net i didn't found any solution please help me.
Thanks in advance
03-06 08:19:00.618: E/dalvikvm-heap(1415): Out of memory on a 2506782-byte allocation.
JSONObject jsonObj1 = new JSONObject(jsonStr1);
ArrayList<String> matter1= new ArrayList<String>();
String stat = jsonObj1.getString(TAG_stat);
String suc=jsonObj1.getString(TAG_success);
// Getting JSON Array node
matter = jsonObj1.getJSONArray(TAG_mdata);
// looping through All Contacts
dbdata =new ArrayList<String>();
for (int i = 0; i < matter.length(); i++) {
JSONObject j = matter.getJSONObject(i);
String ddesc= j.getString(TAG_ddesc);
String spd= j.getString(TAG_sped);
String gzid=j.getString(TAG_geozid);
String dev = j.getString(TAG_devid);
matter1.add(stat);
matter1.add(suc);
Log.v("contact", ""+matter1);
dbdata.add(ddesc);
dbdata.add(spd);
dbdata.add(gzid);
dbdata.add(dev);
dbdata1.add(dbdata);
One strange thing about this is that the crash only occurs every 2nd or 3rd time the app is run, leaving me to believe that the memory consumed by the app is not being garbage collected each time the app closes.
That is certainly possible, and if it is the case then it probably due to a memory leak that can be traced back to something that your application is doing. I think you should focus your initial efforts into investigating this aspect ... rather than loading the file in chunks. (I am not familiar with the Android tool-chain, but I am sure it includes memory usage profilers or memory dump analysers.)
EDIT
In response to your followup comment, the fact that it works 2 times in 3 suggests that your app ought to work roughly as-is. Admittedly, you don't have much leeway if the input file gets bigger.
A couple of ideas though:
Rather than reading the file into a String and running the JSON parser on the String, use a parser that can read directly from a stream. Your current solution needs space for two complete copies of the data in memory while you are doing the parsing.
If the file gets much bigger, you may need to think of a design that doesn't create a complete in-memory representation of the data.
I'm not sure that it is a good idea to read a JSON file in "chunks". This could present problems for parsing the JSON ... depending on exactly what you mean by reading in chunks.
Use Gson library. Will optimize all your process.

Read JSON from /assets folder into an ArrayList in Android?

First, I have researched this question a lot. I learned how to read from my text file in the assets folder. The problem is that I don't want to end up with a string because the file is actually an arraylist written using json and inside that arraylist are objects. I need to access one object only. Here's what I mean:
I have a Book class, which has an int called chapter, an int called title, and an int called pageNum. I created several Book objects and added them to an ArrayList. Then I used the following code to write my ArrayList to a text file (in a regular java project):
ArrayList<Book> aList = new ArrayList<Book>();
aList.add(book1);
//etc...add more Book objects here...
File aFile = new File("books.txt");
FileOutputStream aFileStream = new FileOutputStream(aFile);
JSONOutputStream jsonOut = new JSONOutputStream(aFileStream);
jsonOut.writeObject(aList);
jsonOut.close();
That code creates a text file which I then put into the /assets folder in my Android project because I want it included with the app. In a non-Android java project I could simply use the following code to repopulate an ArrayList so that I could parse Book obects from specific indexes:
File bFile = new File("books.txt");
FileInputStream bFileStream = new FileInputStream(bFile);
JSONInputStream jsonIn = new JSONInputStream(bFileStream);
Arraylist<Book> bList = (ArrayList<Book>) jsonIn.readObject();
Book aBook = bList.get(253); //some arbitrary index
The json code I'm using comes from quickconnectfamily.org. You have to add a file called qc_json.jar to the build path of your project. http://www.quickconnectfamily.org/qcjson/more.html
The problem in Android is when I read the file using InputStream, I can only get the entire file into a string, the code above doesn't work in Android. I can't wrap JSONInputStreams around an InputStream, only around a FileInputStream. But it seems I am unable to use
FileInputStream.
So what I need is a way to create an ArrayList rather than a string in my Android app.
Without giving away too much about my app, the app basically generates a random number and creates a Book object from that index in the ArrayList. Then the user is quizzed with info from that specific book. Sounds silly, but the real app is much cooler.
I'm open to solutions, alternative methods of storing objects in a text file, etc. Please don't simply post criticism about my grammar, syntax, or application idea. I'm pretty new to app development and I couldn't care less about personal opinions. If anyone wants to see
more code I can upload it, but it doesn't seem necessary at this point. Thanks.
I figured out the solution. I connected my Evo 4G and tested the app and it worked. Here's what I did:
I used the following method to read from the file, which is what I was doing before:
InputStream is = appContext.getAssets().open("books.txt");
int size = is.available();
buffer = new byte[size];
is.read(buffer);
is.close();
String bufferString = new String(buffer);
When you do this, you end up with a String of the entire file. This is what I was able to do before, but what I wanted was a way to convert the String to an ArrayList of Book objects. Here's how I accomplished this:
//convert string to JSONArray
jsonArray = new JSONArray(bufferString);
//parse an Object from a random index in the JSONArray
JSONObject anObject = jsonArray.getJSONObject(randomNum);
Book aBook = new Book();
aBook.setTitle((String) anObject.get("title"));
//you can continue to set the different attributes of the Book object using key/value pairs from the Book class (e.g. setPageNum, setChapter, etc).
I don't know, maybe that was obvious to some people, but I really couldn't find any examples that did this. In a different question someone mentioned using the json library native to Android org.json and so I tried that and it worked.

Elegant way to store numerical data such as game moves

I am currently storing game data such as a sequence of moves in a multi-dimensional array, that is defined as:
private final int[][] gameMoves = {{1},{2},{0},{3},{1,4},{1,4,5},{2,4}}
The array is of course much bigger.
I feel that there should be a more elegant way to do this, possibly storing the data separately from the code using either an XML file or the SQLite database and then retrieving it in a while-loop.
If using an XML file, what would be the format and what would be the method of retrieval?
If using an SQLite database, what would be the way to get the data inside the database before executing the source code?
Any examples would be much appreciated.
If this is data that never gets changed, there isn't really any reason to persist it, especially if it gets read from frequently, since you'll be taking a performance hit in the reading thereof. If anything, perhaps a separate package containing game data objects each providing 'gameMoves', might be a winner.
Or, if you're tied to the separation from a conceptual and/or requirement standpoint, JSON might be a good way to go, as it's more directly tied to multidimensional arrays than a XML file might be, and certainly easier to work with for that than a DB.
for example, your JSON file might be:
[1, 2, 3, [45, 6, 7], 8, [9, 10]]
You could then use the JSON library to parse it with minimal fuss:
JSONArray data = new JSONArray(textFromFile);
for(i = 0; i < data.length; ++i) {
//Read data
}
See http://developer.android.com/reference/org/json/JSONArray.html
If those numbers represent moves you should use constants.
const int MOVE_WAIT= 0;
const int MOVE_LEFT= 1;
const int MOVE_RIGHT = 2;
const int MOVE_UP = 3;
const int MOVE_DOWN = 4;
const int[] SHOOORYUKEN = {MOVE_RIGHT, MOVE_DOWN, MOVE_RIGHT};
int[][] gameMoves = {{MOVE_LEFT},{MOVE_RIGHT},{MOVE_WAIT},{MOVE_UP},SHOOORYUKEN}
etc...

Categories

Resources