I have an app that uses a sqlite db of size 30 mb (in zipped the size is 13 mb).
The updates are costly in size since the sqlite db is getting downloaded every time even though there is no change in data.
I have the following solutions in mind.
Approach 1:
Download the db on the launch of the app (if the db is not already present). So, the updates will not download. Problem with this approach is user needs to wait till the download is complete. It might make the user to uninstall the app since he has to wait.
Approach 2:
Put the sqlite db in the expansion file and it will downloaded in the play store before installation itself.
I am planning to take the approach 2.
Is it fine? In the approach 2,
Will the android allows to put an expansion file even though my app
size is less than 50 mb?
Will android show warning like "This app is going to do huge download.
Use it only on Wi-Fi kind of warning?
You can add the zipped DB file in your assests folder and first time when your app is launched unzip and copy the DB file from assests folder to the application memory.Now you can use it easily.
Note-One drawback of this approach is that you won't be able to delete the DB file from assests folder because it is readable only.
Related
I am trying to download a operator specific db when the app is first time opening.
Since its operator specific, I cannot bundle this db as part of apk.
I have implemented a way to use DownloadManager to download the my_operator_specif_db.db and then reading it through SQLiteAssetHelper.
The size of the db is 23 mb. Since I am using "Accept-Encoding: gzip" in the headers, it will download zipped version of this db which is 8.3 mb. Still 8.3 mb transfer takes time to download on poor internet connection. Instead of this db, if I download a flat csv file with the same contents it comes around 13mb RAW and 3.8 mb zipped.
Effectively, I can make the download double times faster if I have a logic to load the flat csv file to an empty sqlite db.
Any suggestions on this approach?
How to load a csv file into db?
23 MB is not that much if what you're downloading is crucial for the app to function properly and you can always make it so that it is downloaded using wi-fi only(usually fast enough) That being said, you do want to keep the downloads to a minimum so what you can do is build the db from a raw text file(or .csv if that's what you have).
In my own opinion, if you're going to download something more than 1 MB, just get the gzipped db, it saves hassle. I'm saying this because downloading anything on a slow connection will be slow.
I'm writing an android app that makes use of many images(dozens). These images, like most of the other data in the app, are updated from a remote database. For the data I am going to have a local database and sync it with the remote database every time the remote db is updated.
The remote database will store the images as URLs, and my app will download the images from these urls to display them in the app. So I could just have my local database sync with the remote one and I'll have the URL of all the image files I'm using and I can re download them from the remote server every time the app is run, but this is obviously slow and wastes a lot of data.
What I want to do is, everytime the database is updated and needs to be synced, the app will sync its local db and download the new images from their URLs as usual, but then it will save the image files somewhere on the device, so next time the app is run it can just grab the images from the device.
I can't seem to find an effective way of doing this, perhaps it's because it's a bad idea to do it this way in the first place? Sharepreferences probably won't have enough room, external storage isn't available all the time, and I hear it's a bad idea to just store a big chunk of binary data(such as an image) in the local database.
What are my options here?
Start with Android Storage Options.
Further:
external storage isn't available all the time
That's true in theory, but in practice, you'll have external storage 99% of the time, especially if you're developing for newer devices. Note that "external storage" and SD Card aren't the same thing -- the terminology here is confusing. For this, use getExternalFilesDir().
Regardless, if your image storage is a reasonable amount (<100MB is reasonable IMHO), then you can just use internal storage. As of Android 3.x, this won't cause a device to run out of space like early devices did. In practice, you'll be fine 99% of the time. Here you might want to use the cache dir, returned by getCacheDir().
In one of my apps I have a large number of files that are synced with a CDN. I use the sync process and database to retrieve download URLs and MD5 hashes of the files. If the MD5 hash has changed, then I download the file again in the background. I simply store the file using the hash as the file-name, so I can easily resolve duplicate files. Periodically, I also walk the cacheDir to see whether there are any files that are no longer referenced in the database.
I'm looking to packing a large sqlite database with an android app, about 200-300MB. From what I've read, my options are putting it in the assets folder, and coping it out at runtime, resulting in duplicate data, or downloading the database from the web at runtime. I don't want to create a webserver for this app, so I was thinking of creating a seperate app that just installs the database to get around the data duplication problem.
All of those are pretty crummy, and I was wondering if there is a better solution?
First, I don't think you are allowed to put a 200-300MB database in your assets folder and distribute it as a single apk, since the hard limit for an apk file size is (as per google's documentation) 50 MB and I doubt that your db will compress that far.. Google introduced the "APK expansion files" for packaging large files with your apk. You can read all about those here: http://developer.android.com/google/play/expansion-files.html
I think (but I have no experience with expansion files myself) that this will not fix the issue you mentioned about having to deal with duplicate data because of the requirement to copy out your database file, but at least this will help you in the sense that you won't have to host your database file on your own webserver. I quickly scanned the documentation on the link I gave above and it clearly states that you should NOT throw away the expansion files when you are done with them, so no help on that part.
Of course, for the "setting up your own webserver part": I wouldn't even bother setting up my own webserver. There are plenty of parties out there that provide you some file hosting service. Probably you will have to pay for the bandwidth, but hey.. if people download your db file a lot, that also means that your app is doing well ;-)
I test of apk expansion files. I created a test app with a 100 MB sqlite database, uploaded the apk and database to the play store. I then downloaded the app onto the phone, the database was downloaded at the same time. The database was automatically renamed and placed in the publicly accessible directory:
/storage/sdcard0/Android/obb/com.example.app/main.1.com.example.app.obb
The app was able to open the database without any problems, but required external read and write permissions. The new and original database are bit for bit, the same. This appears to be the best solution for my use case.
A few issues, the new android developer console does not support uploading expansion files, so the old one needs to be used. Google only allows 2 expansion files, so that may be an issue for some use cases. Google say that you must not rename or delete the file, and that your app must be able to download the file from Google itself if it wasn't automatically installed.
I am working on a library app and have books stored as .sqlite files. Each book's sqlite database file is about 10MB in size. I first tried to put it in assets folder and then copying it to database/ folder but since the file is >1Mb this gives me an IOException. Then I tried to access it from raw folder but its still giving me IOException. So, what is the correct way to access such a file. Also, in future the app might need to download such files from server, so in that case where should I store such database files?
Thanks!!
Before Android 3.0 you are not allowed to open files larger than 1mb.
From the link: [P]rior to Android 2.3, any compressed asset file with an uncompressed size of over 1 MB cannot be read from the APK.
Here are some solutions:
Perhaps you want to look into shrinking the size of the databases, and that might be by making your own, or by removing some of the entries that you do not need/want.
Another solution would be to offload the databases to a server that you have access to and require the application to access the web to get the data that is necessary from this server. I can imagine a nice RESTful API to do this.
There is a workaround to this issue. The limitation of 1MB is only for SQLite files and not for other file types. Rename your SQLite file as something like "db.mp3" and then when your app starts, you can copy this file to your SD Card and use it as a normal SQLite file from SD card.
I have implemented this solution and it works perfectly fine on all Android versions.
Have the app download the database from a web server to the sdcard - this saves the waste of storing both a compressed version in the .apk and an uncompressed version outside. And it's not really any more insecure as an .apk is just a zip file anyone who really wants to can read.
I have a database of questions from an external SQlite manager. I would like to create a database on the first run of my app.
Is this possible instead of writing hundereds of lines of code?
Yes, this is possible. You can put the SQLite file in the assets folder of your Android project (or download it on demand) and copy it to the database folder of the installed app.
A tutorial explaining all the neccesary steps can be found here.
Note however that if you include the database file into your APK, it will consume twice the disk space than absolutely necessary (since it will exist twice, as asset and as usable database). Depending on the size of your database, it might be preferable to download it instead of embedding it into the APK.
Note furthermore, that prior to Android 2.3, any compressed asset file with an uncompressed size of over 1 MB cannot be read from the APK (and any asset will be compressed by default).