I'd like a bit of advice on how to retrieve, process and store data.
I'm building an app which gets finds the nearest laser tag site to where you are. The adderss data is stored (due to some bad design) in one field in an external database, with it's country coming from another table (told you it was a bad design). I may change where it's getting it's data from if I can, as it currently requires a lot more JSON parsing.
Anyway, as I've got the address, I need to get the locations and put them on the map. As there's several hundred sites to add, it takes a while to request the co-ordinates for all of them, and process them etc.
So my question is this: would it be better to use the google georeader on android, which I'm having problems with, or to use a web server to send the requests via google's http geocoding requests.
Also, would it be better to then store the data on the phone and check for updates every time it loads, or just not store anything on the phone and get all the data every time it loads?
Ta muchly!
So my question is this: would it be
better to use the google georeader on
android, which I'm having problems
with, or to use a web server to send
the requests via google's http
geocoding requests.
It's not like laser tag locations move terribly frequently. Do a one-time geocoding lookup on your server, cache the data in a local database, and serve out of the cache. However, there may be legal implications here, see below...
Also, would it be better to then store
the data on the phone and check for
updates every time it loads, or just
not store anything on the phone and
get all the data every time it loads?
That's actually more of a question of law than of coding, IMHO. From a coding standpoint, if the database is small, keeping it on-device and updating it periodically would be OK. However, if it is not your data, it will depend on whether or not it is legal for you to hold onto it and copy it around.
Related
For better UX mobile apps store data on the client side (on the device) to provide immediate information when loading an app without having to wait for data from the internet and providing data even when the device is offline. Of course data is updated/fetched whenever possible later on.
I am building an app (in flutter) which is a social network/information feed like app: there are users, profiles, feed, posts etc. When the user opens the app I would like to show data that was available the last time the app ran.
My question is what is the right way to implement cache? There are two main ways I can think of, an easier/uncertain way and a more difficult/stable way, and I would like your opinion about them. I have time/resource constraints ofc. Most information is through HTTP requests, so:
The easier way: HTTP Cache interception
I would use an out-of-the-box cache plugin for my HTTP client. I think I can just cache the response for each request I make (for some time) and rely on the cached info. Images are also cached based on url. When I make a request on application load I return the cached result if there's any, and if it was a cache hit I fire the request again, so when you open the app you will see immediate information, but after a sec or so you will get the fresh data too. Usability of this solution ofc depends on how well I design my API.
Harder: Store data in a structured database.
This is the option I try to avoid, because it's more time implementing this. It could be either a SQL or document store, and I would have to implement the cache look up/save/update mechanism. Since I am just building the app, I think this would slow me down because data types/ architecture might still change. But is this the ultimate way to go with mobile side caching?
Thank you
I think the easier way is your best bet the only time i can think of that cache could be a problem is if you need critical data that has to be correct and not a old cache value, but you can avoid this problem by not caching the critical values.
Also if you use firebase it does some automatic caching which might be useful.
My Android app is fetching data from the web (node.js server).
The user create a list of items (usually 20-30 but it can be up to 60+). For each item I query the server to get information for this item. Once this info is fetched (per item), it won't change anymore but new records will be added as time go by (another server call not related to the previous one).
My question is about either storing this info locally (sqlite?) or fetching this info from the server every time the user asks for it (I remind you the amount of calls).
What should be my guidelines whether to store it locally or not other than "speed"?
You should read about the "offline first" principles.
To summarize, mobile users won't always have a stable internet connection (even no connection at all) and the use of your application should not be dependant on a fulltime internet access.
You should decide which data is elligible for offline storage.
It will mainly depend on what the user is supposed to access most often.
If your Items don't vary, you should persist them locally to act as a cache. Despite the fact that the data mayn't be really big, users will welcome it, as your app will need less Internet usage, which may lead to long waits, timeouts, etc.
You could make use of Retrofit to make the calls to the web service.
When it comes to persisting data locally within an Android application, you can store it in several ways.
First one, the easiest, is to use Shared Preferences. I wouldn't suggest you this time, as you're using some objects.
The second one is to use a raw SQLite database.
However, I'd avoid making SQL queries and give a try to ORM frameworks. In Android, you can find several, such as GreenDAO, ORMLite, and so on. This is the choice you should take. And believe me, initially you might find ORMs quite difficult to understand but, when you learn how do they work and the benefits they provide us, you'll love them.
I am running into an issue with rate-limiting and think I have found a good solution.
I would like to create a server which periodically requests data from 2 different APIs (Twitter, RSS feed) and caches it on the server. The client would then request the data from the server and display it. I only have limited experience in server-side development. Can anyone provide any tips or point me in the right direction in order to implement this. I was thinking about using Amazon Web Services to host the server? Any pointers are much appreciated! Thanks!
Lots of ways to do this, but if you want to use AWS, you could choose S3 as the place to store a copy of the RSS feed (perhaps in JSON format), and your android ap could just request it from there.
S3 is super cheap, super redundant and super fast - plus you can use http://aws.amazon.com/cloudfront/ and push out copies to geographically closer regions around the world (for pennies), if you think you would benefit from that reduction in latency (and assuming you have geographically diverse users).
The only part you would still need to do would be a small/simple back-end process that polls the two source API's, on a schedule you determine, and writes the data to S3.
For something like this a micro-EC2 instance would probably be plenty of horsepower and only cost a few dollars per month.
I think you might have a bigger privacy concern if you started saving other peoples data yourself (I'm not sure how many people would be ok with you to saving their Twitter data on your server).
I'm sure you can cache Twitter and RSS data locally, and just ask for whatever has changed? which might reduce the number of requests your sending?
Alternatively, you could save the API key your using on a web service and get your apps request that key each time. When you've run into limitations with one API key, just switch to using another? Google actually recommend this method for using their own API's (this way you can revoke an API key easily without having to upload a new app).
Hope that makes sense,
Danielle
I would ask a semi-theorical question about web services and client-server architecture.
I have a server with a database with about 50 tables. This server holds even one table which contains information about users associated with the clients.
Each client has associations with a subset of all data in db.
The defined architecture implies that each client (running on Android app) calling ,with a predefined frequency, a certain number of Web Services, fill the local database, creating a copy of all the tables residing on server sb, containing the only information related to the requesting client.
Furthermore, the data in db server side can change, so frequent synchronization is required.
Considering that the client application can be imaginated as a shop online application, so should be possible browsing through providers, articles, make a order,remove orders etc.
So, when I talk about fill the local database I mean store providers, articles, ...that is, all information realated to the requesting client.
Can make sense fill the local database with this information?
I think that is more reasonable call a web service only exactly when the information is required, and not store information in local db.
So, synchronization is no more required.
Tall me what you think about ? thank you.
I know NOTHING about Android development, but have tons of experience with using web services and SOA.
In my experience, especially when the client device has limited storage and processing power, all the business logic and data logic should be in the web services, and the client app used only for display and calling those services. How you implement that is something that we can't answer for you. It's different on every project.
The only exception is when you absolutely have to have the app running while not connected. In that case, be sure you know your specific requirements, and be very dilligent to only persist at the client the data that you absoutely need, and only provide the business logic that you absolutely need while disconnected.
(This helps to keep security simpler as well - lost or stolen devices are one of the highest sources of data breaches, so the less potentially sensitive data on the device the better.)
I don't know if that's helpful, but I throught I'd throw it out there.
If your application is going to work only online, then YES. You don't need to worry to store the data into the database and retrieve it later. If your requirement that needs to support both online and offline, then you have to go withe the database.
You can run a service or Async task or Handlers to invoke the Webservice from the Android Application, that way your UI thread won't be blocked.
If your response is too big, then probably you have to think about middle tier, that way you can convert the SOAP response to JSON that would be easier for the device to process the response and network transmission is faster.
I believe based on your requirements, its good enough to support only online mode... Because the data might vary if the user is not using the app for two days and logs in back and show offline data that might be irrelevant
I have a database and I need to present a data via HTTP using JSON web services. Currently, I'm designing JSON datasets that will be provided as web services. The data from the tables will be aggregated to suite an app needs.
If the data size is large and we will try to download that at once then it might take too much time and an app will not be responsive at start. That's bad. It's well known we should minimize the number of http requests the app will make to download the data. However if we would split the data into small chunks then during every app action step an app will performe http requests and that might be unresponsive solution too.
We are talking here about mobile app development, so internet will be exposed over cellular ISP or wifi, so the speed might be quite slow. I understand the split process depends on app workflow process and etc., just curious are there any general guidelines? For example: if JSON data is larger than 1MB then definitely split it into smaller chunks...
Look at how your mail read work. You probably have tens of thousands of emails in your account. The app will show the first ones then provide a button at the bottom of the list to display more items. It's a pretty good solution usually to provide a lot of data.
Also #Selvin ideas are just great. Don't use UI thread to download stuff, use a different thread. Services are pretty good for getting data asynchronously.
One of the way is to create a service which gets started when there is network. All downloaded data can be cached in sqlite and use a content provider to get the data. But it depends on your app.
Sometimes it depends on your UI Screen. For eg. You can create a Listview with load more. On click of it load extra data.
Other way is to create API such as which gives only updates based on timestamp. But its all depends on the app. Just sharing my ideas. Might not be perfect. Others can surely give a better one.