I am trying to build dictionary app(actually it is a modification of google SearchableDictionary sample), whose source of words and their definitions is very big, around 5MB. I tried many ways and using many formats and it still cant run properly on android. Sqlite database should be the best solution, I have built it and its size is 10MB(tried building it both before runtime and during runtime).
The main problem is the size of the definitions, but I have seen some other applications have managed to do this. It might be that there are some file size limits built into android system, but anyways if they werent it all takes so much to search and run queries in this sqlite database.
What am I doing wrong?
BTW: It HAS to be offline dictionary (download definitions max 1 time).
Problem in a nutshell:
word -definition
word2 -definition2...
Stored in a 10MB sqlite database (tried loading it from assets), not working.
With some hacks (loading it manually with eclipse DDMS tool) it is working but terribly slow.
Are you loading the database from the Assets folder? If yes, then that's your problem. There is a file size limit on what is in the assets folder (1mb I believe).
You have two options:
Split up your database into multiple 1mb files
Create a webservice. Have your application call the webservice which in turn downloads the database to your Android device. OR create a webservice API that your application uses to get data on as it needs it basis.
I have achieved by zipping the database and unzipping it into external caching direcotry (SD Card). you can look at the sample code here - http://www.android.manutech.mobi/2011/03/how-to-manage-sqlite-databases-with.html
Rename it to something like "databasename.mp3" or any media format. It won't be compressed by the package manager and therefore you can use it just like you need.
Have you tried compressing the file? If the data is just raw text I bet it'll compress to smaller than 1 MB.
1) Store a compressed version of only the words using the GZIP. (this will be very small only about 350k, and must be pre-sorted)
2) Load the list into a:
new ArrayList<String>();
3) Use a binarySearch to find the word
.binarySearch
4) When you need a definition, call an API like this with the word
http://services.aonaware.com//DictService/DictService.asmx/Define?word=aardvark
5) parse the resulting XML
10M file is just way too small to worry about. It didn't work, it might be because of the way of opening the database file. By default, Android read database file from ://data/data/PACKAGENAME/databases/DATABASENAME.
A simple solution could be: 1) compress the db file to res/raw/DATABASE.zip, 2) then unzip it to //data/data/PACKAGENAME/databases/.
You can get sqlite android demo code from here: http://www.cs.dartmouth.edu/~campbell/cs65/lecture15/lecture15.html
Related
I'm trying to implement a simple pre populated database into my assets folder of my Android application which I will then query at a later point. But the problem I'm facing is an encoding issue which I can convert the database inside of Android studio but then it implements a password using cipher encryption.
My current process of adding the database to my project
Firstly I went through the process of building a simple database and table inside of the sqlitebrowser application and made sure that the encoding within the preferences section was set to UTF-8.
http://sqlitebrowser.org - Link to the application which I'm currently using to build the database.
After I was happy with the database I then simply copy and past it into my assets folder.
I'm now presented with the image below (example)
4.My next step is that I convert it to UTF-8 as requested but I thought the sqlite browser would of already put it in this format.
5.The Code will now build the database correctly and I can see it within my data folder.
I then take a copy of the database off the virtual device, but when attempting to open it using sqlitebrowser I'm presented by a window asking for a password (Cipher Encryption). Now when I first built the database at no point did I add a password, so this makes think when Android asked to covert it did it also implement encryption?
Going back to the main question, how can I build a simple database in UTF-8 format which I can then implement into Android Studio assets folder?
At step 2 STOP ignore steps 3 on.
Instead step 3 write an application (perhaps using Android SQLiteAssetHelper, there are numerous tutorials on using SQLiteAssetHelper) to open and use the Database.
Android Studio is not designed to browse/interogate SQLite files, so basically it doesn't have a clue what the underlying data means. However from your screen shot you can see that some of the data does in fact indicate an SQLite file.
For example the following is a screen shot of opening an SQLite Database, (actually created and used in an Android App) in Notepad++ (see any similarities?) :-
e.g. It starts with SQLite format 3, the below mentions tablestudent and then goes on with CREATE TABLE students (_id......
So in all likelihood the data is fine it just needs to be opened/used by something that knows how to interpret the data in the file i.e. SQLite.
What can be seen from your screen shot, is that in all likelihood you have a column named _id.
Therefore your an answer to your question (ignoring opinions such as the best) :-
Hello Zoe what is the best application for building a simple sqlite
database which will be in UTF-8 format that I can then import into my
assets folder?
is, it looks as though you have a suitable tool already (personally I use SQlite Manager for maniuplating and interrogating SQLite Databases, can't say whether SQLite Brwowser is better or worse or the same). All you need to do now is ascertain how to do the new step 3.
regarding your statement
I then past the database into my assets folder which means 3 must be
an image from the assets folder.
That's not the case, all it is is that Android Studio has made an attempt to open the file and perhaps believes that it should be an image. I suspect you could drop in a PDF or Word document (might have to change the extension to fool Android Studio) and it may well come up with a similar response. i.e (not that I've tried this) it may be that Android Studio has not been designed to read in MS Word or PDF documents, so knows nothing about them.
Ok, tried this now, here's part of a PDF file (extension renamed to xxx, as AS opened it as a PDF with pdf extension) :-
I have an app that downloads many images that are about 50k to 100k in size. The full database can be anywhere from 50MB to 500MB.
We are currently using sqlite, but it has slow inserts. We did a test writing files and that was slow as well. It seems to be taking about 300 to 800ms per image to write - which comes out to about 250kb/sec (slower than the network speed). I did a benchmark with A1 benchmark and it says my write speed is 14MB/sec, so i'm not sure why my app is so slow?
Here are the primary requirements:
1) Fast writes
2) Fast reads/lookups via an custom integer key (this can be converted a file path for individual files)
3) Prevents user from easily accessing the images (if we save as images they can connect the external storage to their computer and browser, but we can write the files in reverse byte order)
hopefully we can still use the sqllite solution, as i like one file, but either way it seems like a storage issue?
As suggested in the comments, I think you are better off using the private memory storage for the actual images. This will have better speed then storing BLOBs in SQLite.
If you still need to keep a DB, for example for complex image searches or such, I suggest you just replace the BLOB field in your DB with a string with the actual location of the image file.
Another solution is to keep the images as app assets, but this assumes the images are always the same and can't change dynamically, and I doubt this is your use case.
I have a huge set of data that I want to insert into the sqlite database before the user is able to do anything inside my application. Right now I have all my data stored in CSV files, then I parse those files and use DatabaseUtils.InsertHelper to do a bulk insertion, but this is taking to long to complete.
I stumbled on this tutorial some time ago and I'm wondering: is it safe to distribute a pre-generated sqlite file? Can I run into problems due to different versions of SQLite on different devices?
I'm planning to support Android 2.1 and higher.
I suppose it depends on your definition of safe. It is certainly possible as long as the database conforms to the metadata table spec Android expects, which is what that tutorial you stumbled upon is showing you. You won't have to worry about version conflicts with SQLite as that is a package built into the core platform and isn't something OEMs add to or implement anything on top of.
However, if by safe you mean "protected" you would need to take special steps to ensure that your database is not externally readable if that is a concern. If you simply place the preconstructed DB into assets/ and copy it over, anyone who can properly deconstruct an APK file can view your database data. This may or may not be an issue for you.
The best approach is to populate this data in the database, keep the database in assets & then copy it to the device ... You can follow this complete sample code here.
I've seen the tutorial for <1MB databases, here, which I've run into the same problem that many people posted in the comments (tells me tables that exist don't exist). But apart from that, since it is limited to 1MB, I'm worried that if that DB ends up being larger down the road, I'd have to completely re-work how it is handled.
Which brings me to this question: how can I load a DB from the assets folder in a better manner?
If you are developing android application <3.0 android version,
So the best solutions are,
Shrink your database file size and make it smaller then 1 MB.
Or just describe table structure in database file and download larger data from web server (using web service).
Cut your database file in separate chunks, Load files bigger than 1M from assets folder
But there are some quick fixing solutions are also available,
I seen one comment in SO for same kind of question which just change the SQLite db file extension and use it,
It something changing of extension of the file in assets folder from .sqlite to ".jpg".
(Try this if its worked in your case, As I never try it)
Also look at this post about Dealing with Asset Compression in Android Apps
you will get something helpful from that.
Basically, I'm trying to store some data (~300 rows, ~10 columns) for an android app. This data will not be changed by the app, just used in calculations and stuff.
However, everything I've found online (example) talks about using a database that is created at runtime. I do not want this, the data will be the same every time the app is run.
So is there a way of using a databse like this? (Or any other way of storing data in a table-like fashion?)
Generate the SQLite database as part of your build and keep it in your app's raw resources. Since all you need is the file's path and name to open it, you can still read it fine. Your open helper will still go through onCreate() the first time unless you include the table Android uses for its own bookkeeping, but that should be okay.
Make sure you only open it for reading, and you should be good to go.
Put your custom file in the assets folder under the project root.
To get a inputstream from the file, just do:
context.getAssets().open(file);
In this way you can store your static data in conma separated or any model you want.
If you want the data constantly changing, you can create a temporary file in the SDCard, by accessing and creating a new file under some path at:
Environment.getExternalStorageDirectory()
How about storing this data in raw file under Assets or Res/raw folder. You can either dump this data on the fly in Database or read it and process it [which may be costly]. Dynamic handling may be costly, test it and compare performance.