Hello this is probably a typical question but i cant seem to find a clear answer?
I have a backend application that will serve data in json form.
The data will be in form [code] [name].
The data sets might vary from 100-2000 rows.
What would be best...
Store directly these json responses as files and then parse them if they exist?
Or store them in the android database?
In each case the data does not change that often maybe 1 per week.
Which way would be the faster and which more efficient?
Thanks
I think database is much more preferable way here. Text rows even in quantities like 2000 on smart phones better not to be handled in text I think.
I'd go for the Sqlite db too, I'm quite sure that it's much faster than using basic file i/o.
Related
I have around 1M strings, new strings may be added or deleted once every month. And my goal is to perform fastest possible live search in those string, in react native. I will discuss about the approaches I have thought and the challenges I am facing in the approach,
To request the backend on key hits and do a database lookup and return the result.
This approach is too slow for my application's performance.
Store the string locally and update the string once every month. I
think this would be the best approach here but there are some
problems I am facing in implementation. They are mentioned below,
a) I am trying to convert the string into a trie data structure and
search through it, but here the initial time for formation of trie
and loading into memory is large and I feel it is not efficient. So
is there a way to store the trie into the device and retrieve it
somehow?
b) Other approach is to store the data into frontend databases like
Sqlite3 or something and query it for the result. But I am not sure
how does the performance get affected?
Please help me with finding the most efficient way to find out the solution to the above problem, it may be outside of the approaches mentioned above. I have tried to be as elaborate as possible but do let me know if any more details is required.
I have too much of data to be stored and am looking for a proper way to store them.
I can use strings-arrays or text files. Either way, what I want to do can be accomplished but I am looking for a better way considering app speed and size.
I am thinking of using string-arrays ( a lot of).Will this be a good decision?
I want to make my app offline and that's the reason of not using Firebase.
Consider an example of all the trains and their route stations. So I need a perfect advice by which the thing I am looking for can easily be done.
Thank you very much for any help.
You can store your data into SQLITE.
SQLite provides an easy and efficient way to deal with data rather than do the data processing internally inside in-memory
variables.
I want to make my app offline - Data storing is very easy and efficient in SQLite. Sqlite offers OFFLINE Mode.
Considering app speed and size - No problem. Loading data from an SQLite database is faster than loading data out of individual files .
I have a few years of exp with webdev using .Net and C#... and one feature I really enjoy there is the linq-expressions for quering for data.. is there anything that simulair for Android using to query against JSONs or against the SqlLite database? (I guess im looking for a typed expression/query-framework).
And for my next question.. Which one of the JSON approach and the SQLLite approach am I supposed to use.. and when? I know this has been asked a million times.. but there doesnt seem to be any strict answers..
Is SQLLite for more complex querying and when Im having a bigger ammount of data, while JSON is used while having a smaller ammount of data and not very much querying is needed?
Could I store all my JSON data in the memmory for faster read/write access?..since I have heard that storing it in the storage/on disk might be a slow process.
But on the other hand I also have heard that SQLLite is slow in generall...
And finally.. when we are speaking of "slow" is it slow like, a single read/write looks up the entire application for a few milisecs or is it slow like compared to a in-memory-database that executes queries and stuff fast as lightning
When to use SQLite
SQLite is one form of how to persist your data, how to secure your data. Generally if you want to store larger quantum of data and have quick access to them(and also if these data are "sensitive"), SQLite is great choice. Hence i disagree with your opition that SQLite is slow, definitely not i guess.
I have SQLite database with approximately with 650 000 records and still smart performance(an usage of indexes is sometimes necessary).
When to use JSON
JSON is lightweight data structure designed for human-readable data interchange and what is main is language-independent. It's very good choice(maybe the best) if you want send data via network(send data to remote server etc.) but i think not very good for data persisting also it's not safe as SQLite is.
Generally you should compare JSON with XML but you can't compare SQLite and JSON which are two different things.
General Idea
SqlLite : For Local Storage
JSON : For Server SideStore
It doesn't make sense to compare SQLite and JSON. One is a storage solution, the other one is a data format.
You should probably read the official guide before deciding which storage solution to choose (SQLite, preferences, file system,...): http://developer.android.com/guide/topics/data/data-storage.html
If you want to go for SQLite persistence you can use some third-party ORM to make queries easier (e.g ORMLite is the most popular one)
I am new to Android Application Development and a new member at stackoverflow. I am currently trying to design a recipe application. I have decided upon the features of the app and the scope it will cover. The scope is very vast for me in terms of covering all the recipes from all over the world. I am to deal with a lot of data in this process.
I am currently trying to figure a good and efficient way of handling the data in my app. So far, as per what I have read in different forums, I believe that I have two options in terms of a database choice : 1) SQLite 2) Database on remote server (MySql/Postgre)
Following are some of the thoughts that have been going on in my mind when it comes to taking a decision between the two :
1) SQLite : This could be a good option but would be slow as it would need to access the file system. I could eliminate the slowness by performing DB data fetch tasks in the AsyncTask. But then there could be a limitation of the storage on different phones. Also I believe using SQLite would be easier as compared to using a remote DB.
2) Remote Database : The issue that I can see here is the slowness with multiple DB requests coming at the same time. Can I use threads here in some way to queue multiple requests and handle them one by one ? Is there an efficient way to do this.
Also I have one more question in terms of the formatting of my data once I pull it out from the above DB's. Is there a way I could preserve the formatting of my data ?
I would be more than thankful if someone could share their knowledgeable and expert comments on the above scenario. Also this is not a homework for me and I am not looking for any ready made code solutions. I am just looking for hints/suggestions that would help me clear my thoughts and help me take a decision. I have been looking for this for sometime now but was not able to find concrete information. I hope I will get some good advice here from the experienced people who might have encountered similar situation.
Thanks for reading this long post.
What about combining both approaches?
A local SQLite database that has the least recently used receipes so you don't need network all the time. Network is way slower than accessing the filesystem.
Some remote database accessed via some HTTP interface where you can read / write the whole database. And if you want users to be able to add receipes for other users to see you'll need an external database anyways.
SQLite : This could be a good option but would be slow as it would need to access the file system.
Accessing a local database is pretty fast, 5ms or so if it's just a simple read only query on a small database.
But then there could be a limitation of the storage on different phones
Depends on your definition of huge database. It is okay if it is only 2MB which would be enough to store lots of text-only receipes.
Also I believe using SQLite would be easier as compared to using a remote DB.
Yes, Android has a nice built-in SQLite API but no remote database API. And you don't need to setup a database server & interface.
The issue that I can see here is the slowness with multiple DB requests coming at the same time.
A decent database server can handle thousands of requests. Depends on your server hardware & software. https://dba.stackexchange.com/ should have more info on that. Required performance depends on how much users you have / expect.
I'd suggest a simple REST interface to your database since it's pretty lightweight but does not expose your database directly to the web. There are tons of tutorials and books about creating such interfaces to databases. There are even hosted database services like nextDb that do most of the work for you.
Is there a way I could preserve the formatting of my data ?
You could store HTML formatted data in your database and display it in a WebView or a TextView (via Html#fromHtml()) - both can display formatted text.
Databases don't care what type of text you store, for transfer over the internets you may need to encode the text so it does not interfere with the transport formatting (XML, JSON, ...).
A simple way is to integrate Parse into your app. They have a nice framework that easily integrates into iOS and Android. Their plan is freemium, so you'll be able to use up to 1 million API request for no charge, and then its 7 cents for every request after that.
You'll have 1gb to store all your data sets / images, etc.
I don't use parse for everything, but I HIGHLY recommended it for large data schemes because they do all the scaling for you. Check out the API, I think it would be worth your time.
I just started to work on a few of my own projects, and I'm using Parse again. I have to say it's improved a lot over the last 6-8 months. Especially with the Twitter and Facebook integration.
The key issue here is the size of the data - any significant database of recipes would be too large to store on the phone imho,thus you seem stuck with the remote database solution.
As opposed to trying access the remote database from android I suggest you use a a go between web application that will process requests from the application and return JSON objects that you need.
It totally depends on your software requirements. If you need to deal with a small amount of data then you may choose SQLite, but for a huge amount to data better use a remote DB.
SQLite: It works fine with little amount of data & I experienced it response time is good.
Remote DB: I think you may use small server side app to submit the data to your client app. It will solve/reduce your thread related issues/complexities.
I want to store structure type data (i.e. information of call logs like name, number, type of number, date, time, duration). Which is the best way and which is faster? SQLiteDatabase (make table and insert, delete logs in it) or use file storage (means make one class for all parameters and write their objects in file using input/output Stream and serializable) or the another way i heard about it is XML parser but i don't know this.
Please help me. Thanks in advance.
It depends on what you are trying to do.
If your goal is speed, the SQLite will give you a serious run for your money (especially if you wrap multiple inserts into transactions). SQLite has been optimized for everything you mentioned and it would be very easy to leverage the speed it can give you.
If portability is your goal, then files may be a slight bit easier. Files can be moved back and forth very easily easily, whereas SQLite might take some more effort.
If being able to search is your goal, then you'd be a fool not to use SQLite, as it is extremely good at searching and filtering results.
I can't give a very informed answer here because I'm just as new to the subject as you are, but here is the link from the developers page that goes over the different types of data storage. I hope you find it useful. http://developer.android.com/guide/topics/data/data-storage.html
Personally, given you know the basics of Databases I would use a sqlite database. It's pretty straight forward in Android. In terms of speed I don't know which is faster, but if you don't have millions of datasets it won't matter.
In my experience in most cases JSON in a file is enough (mostly you need to store an array or an object or just a single number or string). I rarely need SQLite (which needs more time for setting it up and using it).