I am trying to create an Android dictionary-like application and get slow performance on retrieving the data. Currently, each dictionary entry is stored in a text file (inside Android assets), each file is named as number, so that I can use index to locate, open and read the file's content (simply read out a single line of String). When using ListView to render the output data and reading the file's content inside getView() method, it takes about 3 second to retrieve 10 entries. I just wonder if there are another approaches (using SQLLite, ???) for retrieving and rendering these entries faster. Any recommendations are appreciated.
I would suggest you to use SQLite
Advantages
You can query
You can update definitions easily
Your data is more secure while using database (If you can Encrypt
using AES or similar algorithm it, then it will become more
secure!)
Fetching results is more faster
You can easily populate the results to a ListView
You can see a complete article here
SQLite will definitely make your job easier and make the app work faster. It's also a lot easier to read data; when you're writing data to the DB make sure to use transactions to speed up multiple sequential writes. I probably wouldn't even consider using a text file except for initial data. There are many resources available online such as this tutorial.
Related
I have been looking at different ways to hold onto some predefined character data, however I am having a hard time nailing down which would be the best solution.
An example of data would be 10 strings, 5 int arrays (of size 10 each). There would be 10+ set's of this data. The application would load in the information and inflate generic "character" objects.
Possible solutions:
XML: Due to Android's structured XML requirements it can be hard to use without making a different XML file for every character, and even then it would have ID overlapping for similar named data values.
SQLite: Wouldn't be a huge database, but databases are ugly version controlled unless it is done with a create-database script which has its own downsides (such as making sure DB is up to date between builds).
Hard-coded Objects: By far my least favorite solution, using polymorphism to hardcode all of the objects. Too dirty, not nearly as dynamic as it should be.
I would like to consider things such as version controlling the files, ease of updating (due to them only being inflated, never changed by the app).
If this data is baked i would suggest to use harcoded data.
Reasons.
In those three solution you save the data in the application.
If you use XML-data, you have to consume the time while code parsing inside the code. And you have to write the code that parses your xml.
If you use SQLite, your data will be doubled because of you have to store this database in raw or assest directory, copied in the /data/data folder. Futhermore, if you use Strings and SQLite by default the data will be doubled again (due to UTF-16 encoding).
If think, if only you manage the data this is more usefull to store directly inside the code. Obviously, if you do not use tons of content:)
You might want to use the Realm framework, which is comparatively faster than SQLite and easy to implement inside in your current code.
It handles large data too and it feels like you're using only native android classes.
I am looking to use an XML file to store the data my Android app generates. With that in mind, I have two questions for the community:
Is XML the best way to store data on Android and most efficient in cases where data may be added or altered every second or less then a second.
If XML is indeed the best for the scenario described in #1, how do I go about setting it up?
1.) Is XML the best way to database data on android and most efficient in cases where data may be added or altered every second or less then a second.
Definitely not.
2.) If XML is indeed the best for the scenario described in #1, how do I go about setting it up?
If you plan to store data just locally, the best way would be SQLite which works as a local database on every device.
If you later plan to synchronize this data with a central database, you may do this asynchronously within an AsyncTask or a Thread which would run periodically, but writing each second into a XML file is a bad idea as far as performance goes.
It's probably also a bad idea synchronizing a remote database at each insert/modification/deletion operation as if you had many users you could collapse the remote database.
I think the best approach is (as previously said) having a local database where you would store that data, and implement a webservice in the remote side if needed and use it to periodically synchronize both databases.
I would use JSON over XML and I would highly consider using GSON from Google. You maybe want to consider writing directly to a database with it's own structure and use transactions and sets. Is there are reason you want to go through JSON/XML?
XML is one of the worst ideas to keep local data in Android.
Most common used is SQLite available on the Android platform, but it all depends on what data and how you want to use.
In many mobile applications you don't need the relational database for one of the following reasons:
You have no relational data (i.e. settings) => no point in making relational tables with 1 record each
You have small, and dynamically changed data (like cache for downloaded content)
You don't need to search for data (using indexes etc.)
What alternatives can be used?
Shared preferences - simple key/value storage of primitive objects
Data serialization - for your consideration - binary (native java), JSON, parcelable (can be combined with the shared preferences)
For most of my app I'm currently using the binary serialization for "local storage".
- It's fast enough (usually much faster than starting the local SQLite engine)
- It's extremely easy and quick to implement, especially when you are using it for json/xml downloaded data parsed to POJO objects. All you need to do is just put "extends serializable" and put few lines of code to serialize/deserialize whole structure
- You can use those same classes for keeping data locally and communication with backend
Of course - it all depends from the situation - if you want to keep locally log of data from some sensor, or allow others apps to use this data, have to quick filter 1k+ records, or you really like to write few hundreds lines of code SQLite will be the best option for you. But most of mobile applications has no clear reason to use the relational (and trust me - not perfect one) engine.
There are few questions related to this topic on stackoverflow, But I didn't get the proper answer. I have some doubts on performance of flat files, Is it better to use flat files instead of SQLite ? Can anybody have performance statistics ? Or example of proper way to code flat file in android.
Aside from performance benefits, here's a simple list of advantages of using SQLite rather than flat file:
You can query items as you wish -- don't need to load all of them and select which ones you need.
Deleting records is a much less painful process. No rewriting of whole files into wherever.
Updating a record is as easy as removing or creating one.
Have you ever tried doing cross-referencing lookups on a flat file? Not worth it.
To summarize, it's every advantage a Database has over a text file.
It depends on your requirement.
If your storage data size is structured-bulky in size then i suggest you for SQLite. On the other hand if the data size is just a single or few lines then flat file is best option.
What makes difference between them is, SQLite stores data in structured format, so it will be easier to find a record from multiple set of records which is very tedious process in case of flat file.
However when if you are storing blob kind of data then it is suggested to use combination of both, SQLite and file system both. i.e. store the image/sound/video data as file format and store their path in SQLite.
Also visit this accessing performance.
SQlite definitely way better in terms of performance and this gets even more important as the size of your data increases.
I've been working on a flutter app where I needed to display a filtered list of items dynamically based on typed text. I initially used a json file to store data and would read and store relevant values into a list, then filter this list as the user types.
This worked just fine with a few items so I thought I was fine until I tested with a real dataset which contained over 150,000 items. Trying to filter a list this large as a user types crashed the app. I moved to a database solution and all my problems were solved. Instant filtering and no more crashes
I'm making a game to run on android. So I want to store player names and the winner, which I will also list up in a view. What is the best way to do this, use a database or write to a file (if so, what type, xml?). I have to be able to add data after every completed game, and the size won't be so large. What would be the best solution?
IMO, using a SQLite database would be the most straightforward. You don't have to worry about the xml parsing that goes along with an xml file. Additionally, the data your storing seems to have a natural relationship that would be conducive to a SQLite schema. For more information about how to use SQLite in Android, see the data storage documentation here.
SQLite is great if you have database concerns and want that sort of data lookup.
However, if you really want to do it Simply with XML then you can in Android.
There is a web service that provides some data that my app makes use of. This data is fairly large and only changes VERY infrequently so I thought it would be nice if the app could cache it on the SD Card and only update it as needed.
Currently I'm grabbing the data (an XML file) and parsing it into an object tree using SAX. This process takes (at most) 2-3 seconds over my WIFI. However, serializing the resulting objects to the SDCard takes significantly longer (a minute or more) and deserializing it still takes longer than just download/parsing in the first place.
Does anyone have any recommendations for improving this or alternate ideas for persisting this data (other than just saving the XML file and reparsing every time)?
UPDATE: This is more than a trivial collection of records. The object-graph is actually ridiculously complex and storing it into a database would result in dozens of tables with only a single record in each one.
Android serialization is notoriously slow. I highly suggest switching to using XML or JSON (or whatever) and writing the file out that way. Since you've already got an XML parser, it may make the most sense just to cache the original XML file you downloaded and reparse it as necessary.
I have switched from Serializable to JSON file storage in an app before and the speed increase was incredible, at least one order of magnitude.
(I may be misunderstanding your question - I assume you are using Serializable for writing to the disc. If you are reproducing the XML, then I'm not sure why it is so much slower on the SD card. Also, I agree that the SQLite database makes the most sense typically, but as you've already stated it does not fit the needs of your application.)
Also unless your data is at least 100s of Kb, I would suggest just storing it in your private data storage instead of on the SD card. Keep in mind that you can't rely on the SD card being available.
I've just been writing an android application for the last week which basically does this. It fetches some (large) XML file online, and then displays part of the data in various views.
We do it by fetching and parsing the XML using SAX, and (while parsing) writing it all to a SQLite database. And then we are just querying the database each time we need to display some view of the dataset.
Works like a charm, and is fast enough for displaying a lot of data on a google map overlay, where we are querying the database on every single call to the draw method of our map overlay.
So I would definitely suggest going for a SQLite database, if the data in the XML document is easily represented in a database.
If the web service can give you just a specified number of results(something like: requestData between index 1 and 10 or give me first 25 results) try to use that (put a simple "Load more results" button or implement an auto-loading mechanism). If the web service not provide this feature then try to save your xml on sdcard and when you need the data try to parse just a specified number of results. Hope this help!
Why don't you use database? See Android Data Storage Guide