I am building a database for an Android app using SQLite. The database consists of Zip code long/lat information. At runtime, a row will be added to a table in the database. I am trying to find the best way to design the database for both scalability and speed. This app will be released for all US states.
My question is this:
Would it be better to create a table for each US zip code, which stores this new information (and query the DB at runtime)
Or, would it be better to create 1 table in which all new data is added (and query the DB at runtime)?
-I might have answered my own question here, but the data added will only be kept in the database for approx. 1 week (most likely only a few days).
Any advice would be great!
One table that contains all zip codes is better. Otherwise you would have a maintenance nightmare on your hands and the code itself would be more complex. SQL queries are designed to be very fast as long as you have proper indexing.
Related
I am doing application for learning words in foreign language, so I have this words stored in my database. These words are separated for example into 3 levels of difficulty. Every level is made of some groups of words, these groups introduces TABLES of SQLite db. I am using SQLiteOpenHelper as communication between application and databases.
Now my question. What is better?
Make 3 smaller databases, each for every level and use own
SQLiteOpenHelper, so together 3 dbs with 3 open helpers.
Make 1 large database, where will be that 3 levels, which means
many TABLES, but just only 1 SQLiteOpenHelper.
Thanks for any advice or opininon.
I suggest 1 large database (DB).
You should not be worried about making large DBs, DBs are invented to store a large amount of data (and even many-many tables). It is much easier to create and maintain one DB than multiple ones and your code will be much clearer using one DB.
And I don't know your program, but I would go even further: I would rather store all words in the same table if you store the same information of them, and add a column to show the level and another one to show the group which they belong to.
The main idea of SQL is that you don't really care how much space your DB will require and how much time it gonna take to find the result of a query because DataBase Managent Systems (in your case the SQLiteOpenHelper and SQLite) are insanely efficient considering space and time. Instead you should rather concentrate on designing a system that can be expanded easily (for example if you want to add another column to tables containing words (e.g. you want to store a new information about words) or want to add new levels or groups in a later stage of development) and has clear structure. You might lose a few milliseconds separating groups and levels via the SELECT command of SQL, but your DB will be much more flexible - you can add levels and groups and add more information about words with ease. The key of desinging a good DB: You should store different kind of data in different tables and same kind of data in same table...
The error that you mention in your comment is almost certainly a bug in your application code. There is no reason that an application with multiple databases should encounter that sort of error.
That said, my answer to your original question is that it is objectively "better" to use a single database.
It is better because you will have less code to maintain, no possibility of attempting to access the wrong database in a given situation, and the code will be more idiomatic - i.e. there's no benefit to using multiple databases, so if you were to use multiple databases, anyone reading your code would spend a lot of time trying to figure out why you did it.
How does a CSV database compare to an SQLite database in Android?
Looking at other questions on StackOverflow, and reading Android Developer Documentation, I have seen SQLite databases being used far more often than reading data from a CSV file. There are also some questions where users have wanted to import a CSV file into an SQLite database (for example, this question or this one). Is there an advantage to using SQLite over CSV?
I have tried using both CSV and SQLite a little, and in terms of performance I don't see a huge difference, but correct me if I'm wrong here.
As I know there are different methods of reading a CSV file, I opened and read it using the BufferedReader like so:
BufferedReader reader =
new BufferedReader(new InputStreamReader(context.getAssets().open(fileName)));
And the SQLite database was opened in the usual way:
SQLiteDatabase db = helper.getReadableDatabase();
I'm not too sure about differences in functionality, although I am assuming SQLite is easier to manage and filter through, and that's why I'm asking this question.
So to summarise:
Are any of the two faster in terms of performance?
Does SQLite (or CSV) have any additional functionality that the other does not, especially in Android (as I am aware Android has its own SQLiteDatabase class?
Why does it seem that SQLite is used far more than CSV databases (i.e. reading and filtering a CSV file)?
EDIT:
Just to clarify, I know a CSV file is just a file with commas separating values, i.e. not a 'database' as a SQL database is. However I can still use a CSV file as a sort of database, where the comma separated values indicate different columns, which can also be filtered, by checking to see if a particular column matches a particular value. So I am asking which is better to read data from.
SQLite and CSV are not really the same thing, which makes the comparison rather difficult. SQLite is a (limited) SQL database, which means that you can use it to store structured and related data in a way that ensures consistency (related records not messing up when either is deleted) and simple way.
Even for flat data, SQLite makes it easier to query a subset of your data (say, all records where one of the columns is less than three), reorder the data depending on your needs and insert a record somewhere in the middle.
CSV on the other hand, is just a text file, that you can use for very simple data. It has little overhead, but it won't help you very much in the sense of data integrity and easy querying. If you don't need this, and always read all of it, sure, you can use CSV.
If none of this rings any bells to you (related records, and data integrity and such) I suggest reading up on relational database systems. As always, Wikipedia is an excellent source.
I wonder when to use several tables in the same database file, and when to use multiple files? For example a separate database file for each table. What are the advantages and disadvantages?
I have "places", "persons" and "bookings". A person is booked to be in one place (at a certain time). Since there are some many-to- relations between them, they need to be in separate tables. Btw, this is a common task, so I wonder if anyone has links to examples or suggestions on how to solve it in the best way.
Your data model is simple, and can be perfectly implemented using 3 tables in the same database. If you use a SQLite database, this database would be one file.
There is no benefit here in creating several files/databases. It would only bring more complexity (and even slightly impact performance/startup time because you would need to open 3 db files)
You can find a tutorial on how to use SQLite (which is the best option if you want to store the data locally on the user device) with Android here.
I'm creating my first android app that will make use of SQlite. I have zero experience with databases, except for creating a mysql database to use with wordpress...
Edit: After doing some research about rest, I'm still confused about how rest, sqlite, and android dev fit together. My goal is to access a rest-based web service through a url and access certain datasets, then store them in my SQlite database. Then I want to access the contents of the database through my java program, and use them accordingly.
The datasets can be downloaded individually in CSV format, but because I will be using so many of them, I don't want to go through every line individually and store them in the database. I'm hoping there's a more efficient way to store these datasets in the database.
My main questions are:
How can I copy the XML contents of a webpage from a url into my sqlite database? Can I do this with my java program, through the sqlite database, or a java library?
Do I only need to copy the contents of the webpages from the url into the sqlite database one time? If so, what can I do if any information is changed in the datasets?
You first need a schema for your sqllite DB. That schema should map to the objects behind the web service. For e.g, you need a Person table in your DB if there is a Person entity on the web. It depends on what all you want to capture.
When you are done designing the schema, you should start writing the code that help you create & manage DB on android. This is done with the help of SQLiteOpenHelper class:
http://developer.android.com/reference/android/database/sqlite/SQLiteOpenHelper.html
If you need to keep the DB synce'd with the data on the cloud (web services), you should implement sync. Android provides a very efficient sync framework.
Also, do watch this video from Android engineers explaining the best practices: http://www.youtube.com/watch?v=xHXn3Kg2IQE
Note, to actually fetch the data from the web service you would use UrlConnection API:
http://developer.android.com/reference/java/net/URLConnection.html
This sample probably captures most of it.
http://developer.android.com/resources/samples/SampleSyncAdapter/index.html
In terms of reading CSV files, there are some good resources here:
Can you recommend a Java library for reading (and possibly writing) CSV files?
Once you have read each CSV line into an object, then you can turn around and persist it to the database. I'm the author of ORMLite so I'll talk about using it. I don't believe there is a hibernate port for Android.
There are a number of Android examples to help you to get up to speed with ORMLite. Also some good tutorials. If you want to write a number of rows at once then I'd recommend using the batch tasks ORMLite feature. For information, see the discussion about creating lists of objects on the mailing list.
I can answer your first question about " I'm not sure how to add them efficiently"?
yes, SQlite is very powerful and intelligent, you can add thousand of records in one transaction, just like traditional database, It significantly improve performance.
about second question, as my understanding, because CVS file is very simple, so you can download and analyze it by yourself.
I have zero experience with databases, and am looking for some direction in getting started. I am making an Android app that needs to read in a large amount of data quickly... Eventually this will be a stock market app that accesses real-time stock information. But before I spend money trying to get access to such real-time data, I'm working with a Microsoft excel file that has times and information on stocks from the past.
I've read up on databases and apparently Androids come with SQLite but I can't figure out how to start using it, or how to import my excel file.
Please help get me started or point me in the right direction. I've found some tutorials but they were difficult for me to follow with my complete lack of experience, and not necessarily related to my project; I'd need really basic step-by-step instructions...
Here is one way of doing this, think only in terms of data you need, rather than trying to get all spreadsheet data into your SQLite DB.
So, think in terms of learning basic SQL and SQL queries and create the tables (on a piece of paper) you need, how you will manipulate the data in these tables (updates, inserts, deletes etc).
Now that you have the table schema and your queries, the task remains of importing this data into the tables. For this I haven't done this but I am sure you will find a lot of resources on the Internet.
Now, as you can see, the seemingly complicated problem is broken into tangible actionable steps.
Here is a reasonable tutorial on SQLite. Here is another for SQLite and Android. You can google for more help.
Also take a look here. The blog talks about importing from excel into SQLite DB.