I'm making an dictionary app for android. This app uses stardict and DICT files which is are often pretty large (10MB or more) and contains just plain texts.
When users look up a word, my app will read the file randomly and return the result. I've read on some articles that reading file is expensive and slow. So I wonder if bringing all data into database is a better solution?
I would suggest putting your words into a database for the following reasons:
DB lookups on android with SQLite are "fast enough" (~1ms) for even the most impatient of users
Reading large files into memory is a dangerous practice in memory-limited environments such as android.
Trying to read entries out of a file "in place" rather than "in memory" is effectively trying to solve all the problems that SQLite already solves for you.
The only challenge presented in using a database is initializing it with the data you require. This problem can best be solved by creating the desired database in advance and then attaching it to your APK assets. There is an example here that is quite good.
Hope this helps.
Related
I have been asked to create a tiny android app.
In everyday work i code for .NET and I have no experience connected with Android, but as it is a really small app I guess it's going to be a good experience rather than something hard.
The core of the app would be a small database (probably XML, unless somebody suggest better solution) that would contain categories, names of the institutions assigned with each category and logo (not very high resolution I guess a single file would be <100kB) of the institution.
The database also would not be very big - I expect not more than 1000 records in total. The DB has to be totally offline and local, it cannot require Internet access when operating.
The model I assume would be to ship new version of the application when the database changes (which is not going to be very frequent).
What is the best way to deal with these requirements?
My first idea was to create an XML file that would contain the records and link to the image. The XML and all the images linked to it would be stored in single file (preferably zip) that would be stored in app resources. This is very good as it is going to be very easy to update the database.
The second idea that somebody suggested me would be to use SQLite and store images in BLOB. In general I have read that it isn't a good idea to store images in database directly, and I am afraid if it's going to be possible to meet all requirements mentioned above.
Mostly I have no idea how to update the database easily and attach it to new version of application.
Any suggestions?
I would be grateful for any response.
I wouldn't go about using XML to save your data and by no means zip anything.
I think your way of thinking is ok, but you're making things really complicated for yourself.
Seeing as you're used to .NET I suppose you're also pretty confident with SQL, so I'd suggest you have a look at how to use the built-in SQLite database in Android.
If you would go the XML route you'd have to serialize and de-serialize the XML file over and over again and then parse the XML. Ok you don't have a lot of data, but searching inside an XML file with at least 1000 nodes would be slow in comparison to the performance of a database.
Also upgrading an existing SQLite database is not that hard - Android has methods for that (onUpgrade coming from the SQLiteOpenHelper).
As to saving images I'm assuming that you won't fetch new pictures from the Internet, so it would be best just to store them in the drawable folder of your app (be mindful of different screensizes) and then reading them into an ImageView when needed. To figure out what image should go for what institution I would store either the image name of each image in the SQLite database or store the resource id for each image in the database - for instance R.drawable.myawesomepictureformyinstitution.
I know my answer is somewhat "superficial", but your question is also somewhat "broad" and hard to answer without me actually writing most of the code, and that's not my intention ;-)
Hope this helps - let me know if anything is unclear.
Newbie android programmer here. I've searched to try to find what I'm looking for, but haven't found it exactly. Please forgive me if this is answered already or too simplistic a question. The closest is this discussion: Android Performance : Flat file vs SQLite
I've used SQL in the past, and I know that my data would be much better served using a relational database rather than a flat file. However, there is a need to export the data every day into a CSV, and so I wrote the app to store to a small flat file csv instead. The file contains customer info like name, balance, phone, address, etc. Total customers is about 500. The file is about 30K.
1) I'm wondering what is best practice on an Android device regarding saving data to a file in terms of timing. Assuming I stick with a flat file, is it better to save the entire file every time the user enters data (once every 30 seconds or so for an hour), or have a timed saving, or have the user save the file? I can't use onPause/etc. as the app switches between activities frequently and at times when there may be no need to write.
2) Does SQLite on Android write an entire db file every time, or is it less 'disk-intensive'? IE when i write to the flat file, I have to replace the entire contents every time... does the SQLite write an entire file or is it more similar to an append, and thus less disk intensive? I understand how the DB works in the memory, but I don't know how it works in terms of disk output. Wondering about efficiency, even for a small file. If it's just silly to keep re-writing a flat file vs. a simple one-liner append to a DB file, maybe it's worth setting up a DB and then exporting to the csv.
I'd be grateful for any help.
In Android, entering data in SQLite uses more time and more lines of code than in .txt file.
Saving data in .txt and use FileReader is convenient to get the data.
What is the advantage of Using SQLite rather than File ?
Advantages of SQLite Databases over File Storage
If you have related pieces of data, regular files don't let you indicate their relationship; SQLite databases do.
SQLite lets you store data in structured manner.
SQLite has higher performance.
SQLite databases can also be queried and the data retrieval is much more robust.
The android.database and android.database.sqlite packages offer a higher-performance alternative where source compatibility is not an issue.
Android-databases created in Android are visible only to the application that created them
There is no file parsing and generating code to write and debug.
Content can be accessed and updated using powerful SQL queries, greatly reducing the complexity of the application code.
Extending the file format for new capabilities in later releases is a simple as adding new tables or new columns to existing tables.
Diverse content which might otherwise be stored as a "pile-of-files" can be encapsulated into a single disk file.
The content can be viewed using third-party tools.
The application file is portable across all operating systems, 32-bit and 64-bit and big- and little-endian architectures.
The application only has to load as much data as it needs, rather than reading the entire application file and holding a complete parse in memory. Startup time and memory consumption are reduced.
Small edits only overwrite the parts of the file that change, not the entire file, thus improving performance and reducing wear on SSD drives.
Content is updated continuously and atomically so that there is no work lost in the event of a power failure or crash.
Applications can leverage the full-text search and RTREE capabilities that are built into SQLite.
Performance problems can often be resolved using CREATE INDEX rather than redesigning, rewriting, and retesting application code.
A federation of programs, perhaps written in different programming languages, can all access the same application file with no compatibility concerns.
Multiple processes can attach to the same application file and can read and write without interfering with each another.
Cross-session undo/redo can be implemented using triggers.
In many common cases, loading content from an SQLite database is faster than loading content out of individual files. See Internal Versus External BLOBs for additional information.
Content stored in an SQLite database is more likely to be recoverable decades in the future, long after all traces of the original application have been lost. Data lives longer than code.
The main reasons which immediately spring to mind, which SQLite gives you and a simple file does not:
http://en.wikipedia.org/wiki/ACID
A standard API: SQL
Theoretically better performance (say, O(log n) rather than O(n))
No wheel reinventing
and plenty of others
Note that trying to solve any of these problems with using a flat file yourself is going to start moving into database territory – but of course you really don't want to write that sort of thing yourself.
Im working on a self-test app.
And I wondering on how to store the data, I've got over 200 questions and more is on the way.
Was thinking of storing them as XML but didnt find a way to get a random question without reading the whole string-array to a variable, which is bad for the memory.
So the correct way to go is to use a SQL-database, right?
But how do I make such a database so that it exists at boot and dont need to be made during start up?
Can't seem to find any tutorial on this subject, on how to handle questionnaires.
Here's a good tutorial on SQLite and Content Provider. It'll introduce you to using SQL databases on Android, and wrapping them into a ContentProvider.
As for how to get the data to the device - you have two options:
You pack the SQLite .db file in the application assets folder. Pros: the database is ready for consumption on the first run of the app. Cons: your .apk is too big. Updating is hard.
You download the data on the first run. Pros: your .apk is slim. updating is easy Cons: there's a delay before the user can use the app.
You ship a small .db file with the first 10 questions. Pros: Your users can start using the app immediately, while you download the rest of the questions in the background. Cons: You have to pick 10 questions you're likely to never or rarely change, or you risk your app to start with outdated data.
Create the db offline and either put it in the apk or download it.
I have an Android app that needs to work offline and requires a lot of static data.
Currently I'm using a JSON file in the /res/raw and loading it with the Jackson parser into my POJO scheme. It works really well since I have an external program that will be generating this data and once in a while when there is a change I'll just publish new version to the Market so I don't have to deal with running an update server and so on.
However, right now my JSON file is about 2.5MB with limited dataset for testing, in the end it'll be about 5-10MB.
The issue is that it already takes about 3-5 seconds to parse the file and this needs to be done every time the application is restarted.
So, what are my options here? I could put the data to a sqlite database, but that would require rewriting the external application and changing the data structure quite a bit. But then I could only query the things I need at the moment and not loading the entire thing at once.
Is there some easier/better way? Also, is there a good way to publish the app with the sqlite database? All the articles I've found talk about creating the database for user data at first startup, but this is not user data and I need it to be deployed from the Market.
JSON feels like the wrong approach for this - it's a good way to encode data to transfer, but that's pretty much it.
It'd be nice to have a bit more info on what exactly your app does, but I'm struggling to imagine a use-case where having several MB of POJOs in memory is an efficient solution. I think it'd be much better to use SQLite, and this is why:
I could put the data to a sqlite database, but that would require rewriting the external application and changing the data structure quite a bit.
You can still use your other program's JSON output, but instead of loading everything into POJOs with Jackson, you could populate the database on first app launch. This way, the app boot time is negligible if the dataset is unchanged.
If you still want to work with POJOs in the rest of your app, it'd be trivial to write a query that retrieved data from the database, and created objects in the same manner as Jackson.
But then I could only query the things I need at the moment and not loading the entire thing at once.
What're you doing that requires access to all the data at once? Searching or ordering a set of objects is always going to be slower than a SQL query to achieve the same thing.
Also, is there a good way to publish the app with the sqlite database?
You can definitely ship your app with a database, though I've not done so personally. This is a relevant question:
By Default load some data into our database sqlite
Hope that's of some help.
There's an excellent API called JExcel (just google it) that works with .xls spreadsheets. If you're not going to be doing any selecting and just loading data from a source, I like to use JExcel because it's more manageable from a desktop and produces easier-to-read code.
Not sure of any performance differences, however. Just throwing in my 2 cents :p