Read JSON from /assets folder into an ArrayList in Android? - android

First, I have researched this question a lot. I learned how to read from my text file in the assets folder. The problem is that I don't want to end up with a string because the file is actually an arraylist written using json and inside that arraylist are objects. I need to access one object only. Here's what I mean:
I have a Book class, which has an int called chapter, an int called title, and an int called pageNum. I created several Book objects and added them to an ArrayList. Then I used the following code to write my ArrayList to a text file (in a regular java project):
ArrayList<Book> aList = new ArrayList<Book>();
aList.add(book1);
//etc...add more Book objects here...
File aFile = new File("books.txt");
FileOutputStream aFileStream = new FileOutputStream(aFile);
JSONOutputStream jsonOut = new JSONOutputStream(aFileStream);
jsonOut.writeObject(aList);
jsonOut.close();
That code creates a text file which I then put into the /assets folder in my Android project because I want it included with the app. In a non-Android java project I could simply use the following code to repopulate an ArrayList so that I could parse Book obects from specific indexes:
File bFile = new File("books.txt");
FileInputStream bFileStream = new FileInputStream(bFile);
JSONInputStream jsonIn = new JSONInputStream(bFileStream);
Arraylist<Book> bList = (ArrayList<Book>) jsonIn.readObject();
Book aBook = bList.get(253); //some arbitrary index
The json code I'm using comes from quickconnectfamily.org. You have to add a file called qc_json.jar to the build path of your project. http://www.quickconnectfamily.org/qcjson/more.html
The problem in Android is when I read the file using InputStream, I can only get the entire file into a string, the code above doesn't work in Android. I can't wrap JSONInputStreams around an InputStream, only around a FileInputStream. But it seems I am unable to use
FileInputStream.
So what I need is a way to create an ArrayList rather than a string in my Android app.
Without giving away too much about my app, the app basically generates a random number and creates a Book object from that index in the ArrayList. Then the user is quizzed with info from that specific book. Sounds silly, but the real app is much cooler.
I'm open to solutions, alternative methods of storing objects in a text file, etc. Please don't simply post criticism about my grammar, syntax, or application idea. I'm pretty new to app development and I couldn't care less about personal opinions. If anyone wants to see
more code I can upload it, but it doesn't seem necessary at this point. Thanks.

I figured out the solution. I connected my Evo 4G and tested the app and it worked. Here's what I did:
I used the following method to read from the file, which is what I was doing before:
InputStream is = appContext.getAssets().open("books.txt");
int size = is.available();
buffer = new byte[size];
is.read(buffer);
is.close();
String bufferString = new String(buffer);
When you do this, you end up with a String of the entire file. This is what I was able to do before, but what I wanted was a way to convert the String to an ArrayList of Book objects. Here's how I accomplished this:
//convert string to JSONArray
jsonArray = new JSONArray(bufferString);
//parse an Object from a random index in the JSONArray
JSONObject anObject = jsonArray.getJSONObject(randomNum);
Book aBook = new Book();
aBook.setTitle((String) anObject.get("title"));
//you can continue to set the different attributes of the Book object using key/value pairs from the Book class (e.g. setPageNum, setChapter, etc).
I don't know, maybe that was obvious to some people, but I really couldn't find any examples that did this. In a different question someone mentioned using the json library native to Android org.json and so I tried that and it worked.

Related

use stored JSON instead of Core Data

There are lots of tutorials out there describing how to fetch JSON objects from the web and map them to Core Data.
I'm currently working on an iOS (later: Android as well) app which loads json objects from web and displays them to the user. In my opinion all this mapping from and to Core Data is an overhead in this case, it would be much easier to save the JSON objects directly and use them as "cache" in the app. Are there libraries/documented ways how to achieve fetching json objects, save them locally and fetch them with a predefined identifier?
I would love to fetch e.g. 10 objects, show them to the user and save the data locally. The next time the user is on that list the local data is shown and in the background the json-file is fetched again to be up-to-date. I guess this is a common use case but I didn't find any tutorials/frameworks enabling exactly this.
You can simply use NSURLCache to cache http responses instead saving JSONs
http://nshipster.com/nsurlcache/
There are many ways to implement this. You can implement cache using either file storage or database depending on the complexity as well as quantity of your data. If you're using files, you just need to store JSON response and load it whenever activity/fragment is crated. What I have done sometimes is store the JSON response in the form of string in a file, and then retrieve it on activity/fragment load. Here's an example of reading and writing string files:
Writing files:
FileOutputStream outputStream = context.openFileOutput("myfilename",Context.MODE_PRIVATE);
String stringToBeSaved = myJSONObject.toString();
outputStream.write(stringToBeSaved.getBytes());
Reading from files
FileInputStream inputStream= context.openFileInput("myfilename");
int c;
String temp="";
while( (c = inputStream.read()) != -1){
temp = temp + Character.toString((char)c);
You can convert this string to JSONObject using :
JSONObject jsonObject = new JSONObject(temp);
Or you can use the string according to your needs.

What is the best way to store parsed json data

i would like to ask, how to store json data. I have a JSON file, which i parse using JSON Library. Now i got the data from a file. But i want to store them and show them later again.
The question is, whats the best way to store data? And is it even worth to store them?
I'm thinking about sql database, because its simple and most used.
Official android docs have few examples, so far i searched but if u have better guide, let me know.
Thank you! :)
EDIT1:
Ok, i have json file with data, which i can add to my app using RAW resources. Those data wont change, its a list of recipes, i dont have to download it. I can read the data like this:
InputStream is = mContext.getResources().openRawResource(R.raw.package_01);
Writer writer = new StringWriter();
char[] buffer = new char[1024];
try {
Reader reader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
int n;
while ((n = reader.read(buffer)) != -1) {
writer.write(buffer, 0, n);
}
is.close();
//catchblock
.....
}
and then i can parse the data trought JSONLibrary like this:
try {
//representing []JSON
JSONArray jsonArray = new JSONArray(writer.toString());
if(jsonArray != null){...}
...}
Im sending a HashMap to ListView, which includes name and id. And if the user clicks the ListView/GridView item, there is new Activity started, which shows all parsed data. I need to get match those parsed data with the id.
There are about 200 recipes in the file. The data are parsed on start of the Activity, while Splashscreen is displayed. I´m not sure, if its good idea to parse the data everytime when app starts.
So, is it effitient to parse data everytime the app starts? And if yes, how to keep the parsed data "together"? Should i use HashMap?
I hope i clarified my question :) Thanks for help.
EDIT2:
So after i knew what to do, i tried the suggested solution to use HashMap. Problem was there i got Failed Binder Exception. I have images encoded in Base64, that means i have a very long String, example here. That is a lot of data, there is limit:
The Binder transaction buffer has a limited fixed size, currently 1Mb, which is shared by all transactions in progress for the process.
I´ve tried to save it to
Set<String> titles = new HashSet<String>();
and then to SharedPreferences but the data for each recipe gets mixed.
**So, here it comes again, should i save the data to SQLite database or is there another effective option i can use? **
Thank you!
It really depends on a number of things like: How much data do you have now, how much will you have later, how complicated is the data. You could use something as simple as an array or hashmap; or something as complex as a database. You need to consider what you are trying to do , and find the simplest solution. If you are trying to persist data, you could use shared preferences, database, and internal/external storage (options outlined here).
Without more information it's hard to say what exactly to do. Keep it simple though. If you are getting JSON from a web service, I'd use an ArrayList or HashMap to store the data, rather than persisting it. It is simpler to implement and does the job.
EDIT:
To answer your question: Yes, using a HashMap and parsing each time is fine. You only have 200 fields, and you don't have images, so the time it will take to parse is minimal. Regardless of how you store the data, there is going to some level of "parsing" done. For example, if you store the data in a database, you are going to have to still pull the data, and put it into a HashMap.

view SQLite Databases in android listview?

I am sure there is an easy solution to this one but I figured I would check with all the folks here first.
I am working on a database creation and management application for android where the user creates and manages data along the line of what PHPMyAdmin does for regular computers.
I have the section where the user creates a database and can insert tables with the appropriate styled data into the system.
The next priority is selecting which DB to enter and modify its contents. Is there a way to display the available databases, along with its table contents, in the form of a list-view for the user to enter and edit the desired data??
I know that this is a rather dull question, but this is basically the last piece of the puzzle for me to fit into this app before it is operational in a raw format. If you need any further information, or any code to examine, I will be happy to provide.
Thanks again for everyone's assistance.
Here's a good tutorial on SQLite databases and displaying contents in a ListView:
http://www.vogella.com/articles/AndroidSQLite/article.html#databasetutorial
It doesn't go over editing that much, but it's easy to see where he puts the values into the database.
thenewboston on YouTube is a good resource for Android tutorials and he goes over SQLite databases:
http://www.youtube.com/watch?v=gEg9OdufXmM
It's pretty comprehensive and slow if you already kinda know what you're doing so here is where he goes over inserting data/editing the database if you just wanna jump to that:
http://www.youtube.com/watch?v=S3Z4e7KgNdU
I know this below can be optimized but for now just create such a method to do it automatically. The method...
Creates an empty database of a random name for a second,
Saves the new database's location - getDatabasePath,
Quikly deletes the empty database,
Removes filename form the saved path to get the directory path olny,
List all files in the database path excluding '-journal' files.
And it goes like this:
ArrayList<String> arr_list_of_db_files = getDBFILES();
pivate ArrayList<String> getDBFILES()
{
ArrayList<String> arr = new ArrayList<String>;
String db_path, rand_name, str_tmp;
//ad.1-2. random file name for db
rand_name = new Random().nextInt((4000000-2000+1)+2000).toString()+".db";
db_path = openOrCreateDatabase(rand_name, MODE_PRIVATE, null).getPath();
//ad.3.
deleteDatabase(rand_name);
//ad.4.
db_path = db_path.replace("/" + rand_name, "");
//ad.5.
File [] files = new File(db_path).listFiles();
if (files == null) { return null; }
//so now we get the filenames one by one
for (int i = 0; i < files.length; i++)
{
str_tmp = files[i].getName();
if (!str_tmp.endsWith("-journal"))
{ arr.add(str_tmp); }
}
return arr;
}
Btw, I cant test the code, but I hope it is fine and some of you find it useful.
Edited: I have optimized the above code.

Saving ORB feature vectors using OpenCV4Android (java API)

I have a training set of images, for each of which I've detected and computed their feature vectors (using ORB feature descriptors and extractors. The questions is: since I need to save those features to reutilise them for matching against test images (using SVM classifier); what is the best way to store the feature vectors, locally on the Android device?
The feature vectors to be saved are of variable size per image, and are thus those with non-maximal sizes are padded with zeros to unify all vectors' sizes. The maximum size currently is 500 rows x 32 cols; thus 16k features.
here are the options I could reach so far;
I've heard of OpenCV's FileStorage, but when going through the java documentation, I noticed a save method for HOG features (not ORB). Furthermore, I'm not sure if saving features using OpenCV's file storage options would be most optimal memory-wise for Android phones, given that the xml file would be too large to load.
My current choice is to opt for a sqlLite database, having a table with two cols; id and feature (as frequently suggested online); to tabulate all the 16k features in sqlLite. That seems rather phone-storage intensive, but it's the most reasonable solution I can find.
Is there a common method to handling feature vectors on Android phones? Does it encompass any of the above methods; if not can you please offer some guidelines on how to implement such a storage solution?
Thank you.
In my opinion the most universal way to store the keypoints is to first convert them to a data-interchange format like JSON.
After you are able to do that conversion you have a lot of flexibility to store it. JSON is easily converted to a String and/or sent through a network connection.
With OpenCV C++ you are able to store data as YAML, but that is not available for Android yet.
To parse JSON in Java you can use this easy to use library Google GSON.
And here is my first attempt to do exactly that:
public static String keypointsToJson(MatOfKeyPoint mat){
if(mat!=null && !mat.empty()){
Gson gson = new Gson();
JsonArray jsonArr = new JsonArray();
KeyPoint[] array = mat.toArray();
for(int i=0; i<array.length; i++){
KeyPoint kp = array[i];
JsonObject obj = new JsonObject();
obj.addProperty("class_id", kp.class_id);
obj.addProperty("x", kp.pt.x);
obj.addProperty("y", kp.pt.y);
obj.addProperty("size", kp.size);
obj.addProperty("angle", kp.angle);
obj.addProperty("octave", kp.octave);
obj.addProperty("response", kp.response);
jsonArr.add(obj);
}
String json = gson.toJson(jsonArr);
return json;
}
return "{}";
}
public static MatOfKeyPoint keypointsFromJson(String json){
MatOfKeyPoint result = new MatOfKeyPoint();
JsonParser parser = new JsonParser();
JsonArray jsonArr = parser.parse(json).getAsJsonArray();
int size = jsonArr.size();
KeyPoint[] kpArray = new KeyPoint[size];
for(int i=0; i<size; i++){
KeyPoint kp = new KeyPoint();
JsonObject obj = (JsonObject) jsonArr.get(i);
Point point = new Point(
obj.get("x").getAsDouble(),
obj.get("y").getAsDouble()
);
kp.pt = point;
kp.class_id = obj.get("class_id").getAsInt();
kp.size = obj.get("size").getAsFloat();
kp.angle = obj.get("angle").getAsFloat();
kp.octave = obj.get("octave").getAsInt();
kp.response = obj.get("response").getAsFloat();
kpArray[i] = kp;
}
result.fromArray(kpArray);
return result;
}
I would suggest storing the feature vectors as images to have a simple and compact representation. You could even use non-destructive compression such as png to minimize file size.
I see that you have considered using the Android SQLite database:
My current choice is to opt for a sqlLite database, having a table with two cols; id and feature (as frequently suggested online); to tabulate all the 16k features in sqlLite. That seems rather phone-storage intensive, but it's the most reasonable solution I can find.
There is a way to save and retrieve MatOfKeyPoint to SQLite database with reasonable efficiency.
Using the database has the advantage of not needing to request write external storage permission from the user (although that permission might be needed for some other of your apps functions).
There is a complete Android solution, with Java code which can be found in this StackOverflow Answer.
The following is a description of what's going on in the code from that answer...
MatOfKeyPoint to byte[] and some attributes
To save to the database, you need to save to a byte[] object. Using the MatOfKeyPoint.get() method, you can get a populated float[]. Then using ByteBuffer.putFloat(), you can loop through all of your floats, finally getting a populated byte[] by using ByteBuffer.array().
You also need to save some attirbutes MatOfKeyPoint.rows(), MatOfKeyPoint.cols(), and MatOfKeyPoint.type() to your database, along with the blob of byte[].
Database blob (byte[]) to MatOfKeyPoint
To reconstitute your MatOfKeyPoint object from the database, first you make a float[] out of your blob. Use ByteBuffer.wrap(blob), then run ByteBuffer.asFloatBuffer(), and finally FloatBuffer.get() with a properly sized new float[].
Now that you have yourFloatArray you run MatOfKeyPoint.create(rows, cols, type), those three things coming from the database record. Finally you run MatOfKeyPoint.put(0, 0, yourFloatArray).

Create list in android app

I want to know a good way to create a list in my android app. I have all info in my DB and want to load data from it each time I start the app and make a list from it (id and title).
What is the best approach?
Should I make a PHP-script that responds with a JSON encoded array with all list items or should I make an XML-file that generates each time the data in the DB changes that I import to the app each time it starts? or any other good way to do it?
Since all stuff are made by XML-files in android it feels like importing a XML would be a good thing, is it? And how do I import an XML-file from a web server into the app?
// Daniel
You can use either JSON or XML.
You can use the web service approach or you can include your db with your application.
In fact, I most often choose to create a sqlite3 database of my data and include it in the assets folder, which can be copied to the app's data folder on startup.
As for copying your sqlite3 database from assets/ to the db data directory, I found these instructions helpful.
In your situation I would pick JSON over XML for all the reason's stated in the following post: http://ajaxian.com/archives/json-vs-xml-the-debate
Plus, in android, there are JSON Array's built in by default so you don't have to do any extra passing of the code.
return new JSONArray("my json string goes here...");
Since we are talking about a mobile device, I would always generate changes in your php script rather than have a full sync as this will be a lot smaller in size that a full sync. However, you will need to give your user a option to do a full re-sync if this is applicable to your app. I would use a SQLite database to store the data and only update the changes in that.
To also make the stream smaller, you can gzip compress your output from php as this can be natively read by the android device. In my app, I compress 500kb down to ~110kb before transmitting, a huge saving on performance. Here a partial example of how to read the stream:
InputStream in = null;
HttpURLConnection httpConn = null; // you will have to write your on code for this bit.
if (httpConn.getContentEncoding() != null)
{
String contentEncoding = httpConn.getContentEncoding().toString();
if (contentEncoding.contains("gzip"))
{
in = new GZIPInputStream(httpConn.getInputStream());
}
}
else
{
in = httpConn.getInputStream();
}
I hope that this all makes sense, it's been a long day programming :)
Stu

Categories

Resources