Basically, I'm trying to store some data (~300 rows, ~10 columns) for an android app. This data will not be changed by the app, just used in calculations and stuff.
However, everything I've found online (example) talks about using a database that is created at runtime. I do not want this, the data will be the same every time the app is run.
So is there a way of using a databse like this? (Or any other way of storing data in a table-like fashion?)
Generate the SQLite database as part of your build and keep it in your app's raw resources. Since all you need is the file's path and name to open it, you can still read it fine. Your open helper will still go through onCreate() the first time unless you include the table Android uses for its own bookkeeping, but that should be okay.
Make sure you only open it for reading, and you should be good to go.
Put your custom file in the assets folder under the project root.
To get a inputstream from the file, just do:
context.getAssets().open(file);
In this way you can store your static data in conma separated or any model you want.
If you want the data constantly changing, you can create a temporary file in the SDCard, by accessing and creating a new file under some path at:
Environment.getExternalStorageDirectory()
How about storing this data in raw file under Assets or Res/raw folder. You can either dump this data on the fly in Database or read it and process it [which may be costly]. Dynamic handling may be costly, test it and compare performance.
Related
I saw a similar question but it was a decade old:
My objective is to create an app (Android/Kotlin) where the users can search/reference/lookup an individual item from a reference 'book'. Something along the lines of a dictionary or book of recipes.
The intent is for the app to be able to work offline, when the user may be anywhere.
As there could be thousands of items/records to reference it would seem to be too much for an array.
Is a Room database a good option and, if so, can a pre-loaded database be included in the package that is uploaded to Google Play?
What is best in 2022?
Yes, Room will do exactly what you want, but pre-loading it with data will be tricky (but doable).
You can't technically "pre-load" the database, but what you can do is save all the database objects to a file, using either XML, JSON, or any other format. You will then need to save that file in your project as either an Asset or Raw, and on first app launch, you'll need to read that Asset or Raw resource, parse each record, and save it to the database.
Needless to say, that initial process may take some time, so I'm not sure how good that will be from the user's perspective. But it's definitely pretty straight-forward to implement
Creating the Assets folder
Reading files from Assets
I'm trying to implement a simple pre populated database into my assets folder of my Android application which I will then query at a later point. But the problem I'm facing is an encoding issue which I can convert the database inside of Android studio but then it implements a password using cipher encryption.
My current process of adding the database to my project
Firstly I went through the process of building a simple database and table inside of the sqlitebrowser application and made sure that the encoding within the preferences section was set to UTF-8.
http://sqlitebrowser.org - Link to the application which I'm currently using to build the database.
After I was happy with the database I then simply copy and past it into my assets folder.
I'm now presented with the image below (example)
4.My next step is that I convert it to UTF-8 as requested but I thought the sqlite browser would of already put it in this format.
5.The Code will now build the database correctly and I can see it within my data folder.
I then take a copy of the database off the virtual device, but when attempting to open it using sqlitebrowser I'm presented by a window asking for a password (Cipher Encryption). Now when I first built the database at no point did I add a password, so this makes think when Android asked to covert it did it also implement encryption?
Going back to the main question, how can I build a simple database in UTF-8 format which I can then implement into Android Studio assets folder?
At step 2 STOP ignore steps 3 on.
Instead step 3 write an application (perhaps using Android SQLiteAssetHelper, there are numerous tutorials on using SQLiteAssetHelper) to open and use the Database.
Android Studio is not designed to browse/interogate SQLite files, so basically it doesn't have a clue what the underlying data means. However from your screen shot you can see that some of the data does in fact indicate an SQLite file.
For example the following is a screen shot of opening an SQLite Database, (actually created and used in an Android App) in Notepad++ (see any similarities?) :-
e.g. It starts with SQLite format 3, the below mentions tablestudent and then goes on with CREATE TABLE students (_id......
So in all likelihood the data is fine it just needs to be opened/used by something that knows how to interpret the data in the file i.e. SQLite.
What can be seen from your screen shot, is that in all likelihood you have a column named _id.
Therefore your an answer to your question (ignoring opinions such as the best) :-
Hello Zoe what is the best application for building a simple sqlite
database which will be in UTF-8 format that I can then import into my
assets folder?
is, it looks as though you have a suitable tool already (personally I use SQlite Manager for maniuplating and interrogating SQLite Databases, can't say whether SQLite Brwowser is better or worse or the same). All you need to do now is ascertain how to do the new step 3.
regarding your statement
I then past the database into my assets folder which means 3 must be
an image from the assets folder.
That's not the case, all it is is that Android Studio has made an attempt to open the file and perhaps believes that it should be an image. I suspect you could drop in a PDF or Word document (might have to change the extension to fool Android Studio) and it may well come up with a similar response. i.e (not that I've tried this) it may be that Android Studio has not been designed to read in MS Word or PDF documents, so knows nothing about them.
Ok, tried this now, here's part of a PDF file (extension renamed to xxx, as AS opened it as a PDF with pdf extension) :-
I have a static table around 100 000 records. How should I use realm so that it can save my time? As I know I can put my static table (in case of sqlite) directly in assets and can access it's data.
Is there any way to put static data (rows), so that I can save my time to manually enter the data?
I already used realm's
realm.createAllFromJson("myfile")
But for this I have to put myfile in assets folder, and I guess it will need double the disk space.
There are two restrictions why this is impossible now.
Any sqlite database is stored inside a particular path:
'//data/data//databases/'
To get access to any database you have to use a path
You can only get data from assets (or raw) in order to copy to somewhere. This is not possible to manipulate with SQLite without namely 'file' data.
That's why you have to copy data from app resourses into filesystem. Imho, this is ugly but I had not found another solution.
Android default API (Realm uses it) use only the files stored in 'databases' folder.
In my android projects I need database to store data for offline usage.
For that I am looking at two options 1)Creating the empty db and copying it to asset 2)Creating the db via code
which option will be good as my app is handling secure data.
weather it will cause any security vulnerability if we store the db structure in asset folder as it will be easily available if we extract .apk file.
Thanks for your support
If your database has something you don't want your legitimate user to see, look for a new project because you cannot attain that. Using a static database in the APK is as you know, easily viewable, but creating it from your app also leaves something that is viewable. You could encrypt the contents, but then you have the problem of how you store the key in a way the user cannot access -- so you really haven't solved anything.
Back to your original question though -- is the data static, or is your application ever going to make changes to it? If the data is static, providing a pre-populated database is fine. If the application is intended to be able to make changes though, you are much better served by letting the application create and manage updates on the database, its schema, and its contents. Following Google's examples will get you there.
I am trying to build dictionary app(actually it is a modification of google SearchableDictionary sample), whose source of words and their definitions is very big, around 5MB. I tried many ways and using many formats and it still cant run properly on android. Sqlite database should be the best solution, I have built it and its size is 10MB(tried building it both before runtime and during runtime).
The main problem is the size of the definitions, but I have seen some other applications have managed to do this. It might be that there are some file size limits built into android system, but anyways if they werent it all takes so much to search and run queries in this sqlite database.
What am I doing wrong?
BTW: It HAS to be offline dictionary (download definitions max 1 time).
Problem in a nutshell:
word -definition
word2 -definition2...
Stored in a 10MB sqlite database (tried loading it from assets), not working.
With some hacks (loading it manually with eclipse DDMS tool) it is working but terribly slow.
Are you loading the database from the Assets folder? If yes, then that's your problem. There is a file size limit on what is in the assets folder (1mb I believe).
You have two options:
Split up your database into multiple 1mb files
Create a webservice. Have your application call the webservice which in turn downloads the database to your Android device. OR create a webservice API that your application uses to get data on as it needs it basis.
I have achieved by zipping the database and unzipping it into external caching direcotry (SD Card). you can look at the sample code here - http://www.android.manutech.mobi/2011/03/how-to-manage-sqlite-databases-with.html
Rename it to something like "databasename.mp3" or any media format. It won't be compressed by the package manager and therefore you can use it just like you need.
Have you tried compressing the file? If the data is just raw text I bet it'll compress to smaller than 1 MB.
1) Store a compressed version of only the words using the GZIP. (this will be very small only about 350k, and must be pre-sorted)
2) Load the list into a:
new ArrayList<String>();
3) Use a binarySearch to find the word
.binarySearch
4) When you need a definition, call an API like this with the word
http://services.aonaware.com//DictService/DictService.asmx/Define?word=aardvark
5) parse the resulting XML
10M file is just way too small to worry about. It didn't work, it might be because of the way of opening the database file. By default, Android read database file from ://data/data/PACKAGENAME/databases/DATABASENAME.
A simple solution could be: 1) compress the db file to res/raw/DATABASE.zip, 2) then unzip it to //data/data/PACKAGENAME/databases/.
You can get sqlite android demo code from here: http://www.cs.dartmouth.edu/~campbell/cs65/lecture15/lecture15.html