I have around 1M strings, new strings may be added or deleted once every month. And my goal is to perform fastest possible live search in those string, in react native. I will discuss about the approaches I have thought and the challenges I am facing in the approach,
To request the backend on key hits and do a database lookup and return the result.
This approach is too slow for my application's performance.
Store the string locally and update the string once every month. I
think this would be the best approach here but there are some
problems I am facing in implementation. They are mentioned below,
a) I am trying to convert the string into a trie data structure and
search through it, but here the initial time for formation of trie
and loading into memory is large and I feel it is not efficient. So
is there a way to store the trie into the device and retrieve it
somehow?
b) Other approach is to store the data into frontend databases like
Sqlite3 or something and query it for the result. But I am not sure
how does the performance get affected?
Please help me with finding the most efficient way to find out the solution to the above problem, it may be outside of the approaches mentioned above. I have tried to be as elaborate as possible but do let me know if any more details is required.
Related
I am building an app as part of a project and I am stuck at the moment.
Background: My team is making a "Time-Wasting" App, and the user can press buttons such as "study" to track their time spent on activities - so there will be lots of integers values in the growing ArrayList.
Task:
Store a growing Arraylist(integers) in the app store and retrieve the values for calculation and graph display purposes.
Problem:
I have looked so far how do to this, but I am a bit stuck. Do I use SharedPreferences, the internal app storage or something else. Also, I have a hard time finding code that I could look at and follow what it is doing.
I need:
An efficient way to store the ArrayList (integers) in a storage place and retrieve it for my app. Any suggestions what would be best and where would I find the code for that?
I looked for about 2 weeks through videos and a through stack overflow but I am still stuck and not closer to how I should store this data type in my app.
Thanks, any help much appreciated :)
My opinion avoid the array list and use a database.
It will allow your app to grow including storing extra information along with the time waster including when they chose to do it, how long , etc.
I recommend Room Database it is an ORM for Sqlite it is super easy to use and there is plenty of documentation on implementation as it's apart of the Android Jet Pack.
If you are really stuck on using a list you can use SharedPreferences it doesn't handle ordered lists but you can store your list as a json string.
I am writing a social networking android application. I have create a .Net webservice with a database on Microsoft Azure, and I plan to call that web service to get data from the cloud and display it to the user. Similar to Facebook.
Now, I have two approaches in mind, and I'm not sure which one to implement. The approaches follow:
"Every time an activity loads, call the web service and reload all the data." This, of course, is the easiest approach, but is it right? I mean, I have around 30 activities, and half of them load data, while the other half post. As far as I see, this approach can be a problem, because it might slow down the application. It can also increase my cloud bill with so many requests. And I don't know if it's right to reload everytime.
"Call the webservice every 10 minutes, and store all the data in a SQLite database, and only update the data if it's been over 10 minutes, or possibly even have a refresh button." This approach is probably the better one, but I'm not sure if it is even worth writing so much code.
I need you advice in deciding on the right technique. Number 2 looks good, but what if there is something I don't know, and I'm writing all that extra code for no reason.
Please help me out here. If there is even a better approach, please do tell me.
It really depends on the sort of data, what sort of latency is required for the data and the quantity of data. Also the size of the project and benefit you will get from implementation as complexity will be increased. For a more precise answer provide further information.
Local caching can be a very good idea in this sort of situation. It is fairly common practice and there are multiple mechanisms which could be used. Depending on the format of data retrieved from your web service, you can store in a
Database, when data needs to processed (searched, queried etc) or there is a lot of data.
File(s), sometimes useful if you are working with formatted data such as xml or json as you can maintain the structure. You can use the native android caching to assist in managing the storage.
Preferences, when data is simple types (and strings) and there isn't much of it.
Local caching will reduce bandwidth consumption (will end up saving the user money which is always popular) and if implemented correctly memory and processing consumption. Possibly most important (depending on the data) it could allow for the effective use of the application when the user doesn't have connectivity.
By the way 30 activities sounds like a lot, you should really look at reducing that by sharing functionality across activities, this should improve navigation, code bulk and memory foot print.
UPDATE from comments
From the limited information available about your project I would suggest using the database as your data store (if any) as you don't want to be caching complete SOAP messages in files, and the quantity 40+ records may make preference storage difficult to manage.
However, as I have mentioned previously you need to consider complexity. When using the database you will have to create a method of construction (perhaps some sort of ORM) separate to your de-serialisation of SOAP objects because technically you will 2 separate persisted data formats.
I am not able to get a definitive answer because there is still very limited information but you need to evaluate the cost of adding such a feature to your project and the benefits you will receive.
I few other things worth mentioning when considering this sort of caching.
How you will manage the cache, it's size and data integrity.
When will you cache, as soon as you have de-serialised your SOAP objects? when you have finished with the data? etc..
How will decide when to use the cache and when to hit the network?
I have an Android app that needs to work offline and requires a lot of static data.
Currently I'm using a JSON file in the /res/raw and loading it with the Jackson parser into my POJO scheme. It works really well since I have an external program that will be generating this data and once in a while when there is a change I'll just publish new version to the Market so I don't have to deal with running an update server and so on.
However, right now my JSON file is about 2.5MB with limited dataset for testing, in the end it'll be about 5-10MB.
The issue is that it already takes about 3-5 seconds to parse the file and this needs to be done every time the application is restarted.
So, what are my options here? I could put the data to a sqlite database, but that would require rewriting the external application and changing the data structure quite a bit. But then I could only query the things I need at the moment and not loading the entire thing at once.
Is there some easier/better way? Also, is there a good way to publish the app with the sqlite database? All the articles I've found talk about creating the database for user data at first startup, but this is not user data and I need it to be deployed from the Market.
JSON feels like the wrong approach for this - it's a good way to encode data to transfer, but that's pretty much it.
It'd be nice to have a bit more info on what exactly your app does, but I'm struggling to imagine a use-case where having several MB of POJOs in memory is an efficient solution. I think it'd be much better to use SQLite, and this is why:
I could put the data to a sqlite database, but that would require rewriting the external application and changing the data structure quite a bit.
You can still use your other program's JSON output, but instead of loading everything into POJOs with Jackson, you could populate the database on first app launch. This way, the app boot time is negligible if the dataset is unchanged.
If you still want to work with POJOs in the rest of your app, it'd be trivial to write a query that retrieved data from the database, and created objects in the same manner as Jackson.
But then I could only query the things I need at the moment and not loading the entire thing at once.
What're you doing that requires access to all the data at once? Searching or ordering a set of objects is always going to be slower than a SQL query to achieve the same thing.
Also, is there a good way to publish the app with the sqlite database?
You can definitely ship your app with a database, though I've not done so personally. This is a relevant question:
By Default load some data into our database sqlite
Hope that's of some help.
There's an excellent API called JExcel (just google it) that works with .xls spreadsheets. If you're not going to be doing any selecting and just loading data from a source, I like to use JExcel because it's more manageable from a desktop and produces easier-to-read code.
Not sure of any performance differences, however. Just throwing in my 2 cents :p
I want to store structure type data (i.e. information of call logs like name, number, type of number, date, time, duration). Which is the best way and which is faster? SQLiteDatabase (make table and insert, delete logs in it) or use file storage (means make one class for all parameters and write their objects in file using input/output Stream and serializable) or the another way i heard about it is XML parser but i don't know this.
Please help me. Thanks in advance.
It depends on what you are trying to do.
If your goal is speed, the SQLite will give you a serious run for your money (especially if you wrap multiple inserts into transactions). SQLite has been optimized for everything you mentioned and it would be very easy to leverage the speed it can give you.
If portability is your goal, then files may be a slight bit easier. Files can be moved back and forth very easily easily, whereas SQLite might take some more effort.
If being able to search is your goal, then you'd be a fool not to use SQLite, as it is extremely good at searching and filtering results.
I can't give a very informed answer here because I'm just as new to the subject as you are, but here is the link from the developers page that goes over the different types of data storage. I hope you find it useful. http://developer.android.com/guide/topics/data/data-storage.html
Personally, given you know the basics of Databases I would use a sqlite database. It's pretty straight forward in Android. In terms of speed I don't know which is faster, but if you don't have millions of datasets it won't matter.
In my experience in most cases JSON in a file is enough (mostly you need to store an array or an object or just a single number or string). I rarely need SQLite (which needs more time for setting it up and using it).
Hello this is probably a typical question but i cant seem to find a clear answer?
I have a backend application that will serve data in json form.
The data will be in form [code] [name].
The data sets might vary from 100-2000 rows.
What would be best...
Store directly these json responses as files and then parse them if they exist?
Or store them in the android database?
In each case the data does not change that often maybe 1 per week.
Which way would be the faster and which more efficient?
Thanks
I think database is much more preferable way here. Text rows even in quantities like 2000 on smart phones better not to be handled in text I think.
I'd go for the Sqlite db too, I'm quite sure that it's much faster than using basic file i/o.