I have an app that includes a pre-populated sqlite database, encoded in UTF-8. DB file size is 170 MB. I am using SQLiteAssetHelper to manage the database
Considering most Android Apps are less than 50MB, it is quite large. My current app size according to the Settings is 238MB.
Is there any way to make the downloadable app size smaller when user downloads it from Store? I have read that you can zip the sqlite database, but I haven't tried that yet. Will that make the app's download size smaller? Are there any other better options?
I am concerned about the app's download size, not the storage space. I do need the app to work offline, so web service is not an option.
Is there any way to make the downloadable app size smaller when user downloads it from Store?
Have a smaller database.
Will that make the app's download size smaller?
Since the APK is already a ZIP file, no, as ZIPping an entry in a ZIP file will not reduce its size further.
Are there any other better options?
You could not use SQLiteAssetHelper, and download the database on the first run of your app. Or, you could not use SQLiteAssetHelper and look into using APK expansion files. Neither of those will change the number of bytes to be downloaded, but they will reduce the overall disk space your app takes up a bit. You can use tools like command-line unzip to see how much space your database file is taking up in the APK (my one test app's database is 89% compressed and so takes up 568 bytes instead of the uncompressed 5120 bytes). This space is non-recoverable, insofar as you have no way of deleting a database from assets/. Assuming a similar compression ratio, your 170MB database will take up an extra ~17MB on disk from the copy in assets/.
You could also not have the 170MB database on the device at all, and have your app work with some Web service to access a hosted copy of the database. This won't work offline, of course.
I ended up using text files for the major lookup tables in my database. I added them in the assets folder, along with the smaller sqlite db. I kept only the small tables that I needed to do inserts/adds in the sqlite db.
And I was also able to easily encrypt the data in the file (which is not easy to do in sqlite)
I had to create indexes to index the file data, and used binary search to find by key or value.
That shortened the apk size, and it's about 30 MB now. Before it was about 76MB. and the app size was > 200 MB.
Related
I ship a read only database with my app including it in Assets/Database directory. All data is pre-populated beforehand.
After integrating with SQLCipher and encrypting the database I noticed that the APK size ballooned from 25MB to 150MB. Reason being is that the SQLite db file no longer gets compressed.
Android Studio normally will compress resources but after DB file is encrypted it seems zipping the file has no effect. I tried testing this outside with regular Zip & 7Zip and it makes no difference, zipped file is the same size as the original.
Un-encrypted database is around 130MB and when zipped takes only 18MB as most of it is text and strings zip very well. Zipping encrypted db file makes no difference in size.
Populating via server isn't a viable option as it would take forever due to amount of records. Downloading a full encrypted DB file from the server is the next option but it would still take too slow with a 140MB file.
Need to find an approach to ship encrypted DB with the app while still keeping the app size to a reasonable 20-30MB.
Any ideas?
Thanks.
Compression requires finding patterns in the data.
Encryption removes all such patterns.
You need to do the compression before the encryption.
This is not possible with SQLCipher; you'd have to ship a compressed and encrypted .sql file, and then execute it on a new, encrypted database.
I have a big database that I use on my website and I would like to use it for an android application without an Internet connection.
My database can export to .db file. How should I do to use this file for my android app.
Regards
Keep copy of your database in asset folder,
copy that database in /data/data/your.package.name/databases while you loading your application first time.
if your database is larger than 1MB Divide the database in small parts.
ex: if size = 8 MB make 8 parts of 1MB than join that parts problematically while copying.
I have an SQLite3 Database (created in a desktop sqlite application), with the android_metadata table for use in my android application.
What is the best way to create the android database using this database file?
I have tried including it in the assets folder, but got a size error when copying this to the application and wasn't sure if this was due to the asset files having a maximum size.
I have also read guides on storing the database on the sd card and accessing it.
Which function on androids sqlite helper is best to open a new database from an sqlite3 export?
I pushed the database file to data/data/com../databases/ and it created an "android.database.sqlite.SQLiteDatabaseCorruptException: disk malformed..
Not sure how to do this, appreciate any help!
You can attempt to use SQLiteAssetHelper for this, as it is designed to make it easy for developers to package a database with their app.
That being said, your database is much too big. There are plenty of 1.6 devices that will not have room for both your APK and the unpacked database.
The reason you get a size error is because android likes to compress files in the Assets folder but then can't decompress them if they are over 1 meg in size. Some file extensions do not get compressed like jpeg, gif, mp3, jet. I always name my db something like name.db.jet and then rename it to name.db when I copy it to the Database directory. The extension on the file really means nothing but naming it with an extension like .jet gets around the max size restrictions.
By the way, if you are going to include a DB that is 30mb in size, be sure to include it compressed. That will take it down to about 10mb in size.
I am working on a library app and have books stored as .sqlite files. Each book's sqlite database file is about 10MB in size. I first tried to put it in assets folder and then copying it to database/ folder but since the file is >1Mb this gives me an IOException. Then I tried to access it from raw folder but its still giving me IOException. So, what is the correct way to access such a file. Also, in future the app might need to download such files from server, so in that case where should I store such database files?
Thanks!!
Before Android 3.0 you are not allowed to open files larger than 1mb.
From the link: [P]rior to Android 2.3, any compressed asset file with an uncompressed size of over 1 MB cannot be read from the APK.
Here are some solutions:
Perhaps you want to look into shrinking the size of the databases, and that might be by making your own, or by removing some of the entries that you do not need/want.
Another solution would be to offload the databases to a server that you have access to and require the application to access the web to get the data that is necessary from this server. I can imagine a nice RESTful API to do this.
There is a workaround to this issue. The limitation of 1MB is only for SQLite files and not for other file types. Rename your SQLite file as something like "db.mp3" and then when your app starts, you can copy this file to your SD Card and use it as a normal SQLite file from SD card.
I have implemented this solution and it works perfectly fine on all Android versions.
Have the app download the database from a web server to the sdcard - this saves the waste of storing both a compressed version in the .apk and an uncompressed version outside. And it's not really any more insecure as an .apk is just a zip file anyone who really wants to can read.
I have a database of questions from an external SQlite manager. I would like to create a database on the first run of my app.
Is this possible instead of writing hundereds of lines of code?
Yes, this is possible. You can put the SQLite file in the assets folder of your Android project (or download it on demand) and copy it to the database folder of the installed app.
A tutorial explaining all the neccesary steps can be found here.
Note however that if you include the database file into your APK, it will consume twice the disk space than absolutely necessary (since it will exist twice, as asset and as usable database). Depending on the size of your database, it might be preferable to download it instead of embedding it into the APK.
Note furthermore, that prior to Android 2.3, any compressed asset file with an uncompressed size of over 1 MB cannot be read from the APK (and any asset will be compressed by default).