I'm building a listview and getting it's data from Parse.com. At the moment, every time the app loads up it queries for new data from Parse.com, causing the whole listview to load.
I'd like a situation where the listview references a local datasource and only go to Parse.com if new data is available. Somewhat similar to what the instagram app does whereby when you load it up, the list view is already populated and would get updated if needed.
I have tried ParseQuery Cache policies but the behavior still stays the same. What would be the most efficient way of implementing this feature?
Thanks in advance.
Sync Adapters can help you with your your problem. It is generally used for account and cloud synchronization. But there is no limitation using it.
http://developer.android.com/training/sync-adapters/index.html
Synchronizing data between an Android device and web servers can make
your application significantly more useful and compelling for your
users. For example, transferring data to a web server makes a useful
backup, and transferring data from a server makes it available to the
user even when the device is offline. In some cases, users may find it
easier to enter and edit their data in a web interface and then have
that data available on their device, or they may want to collect data
over time and then upload it to a central storage area.
Although you can design your own system for doing data transfers in
your app, you should consider using Android's sync adapter framework.
This framework helps manage and automate data transfers, and
coordinates synchronization operations across different apps. When you
use this framework, you can take advantage of several features that
aren't available to data transfer schemes you design yourself
You can access sample project here: http://developer.android.com/shareables/training/BasicSyncAdapter.zip
Note: Sync adapters run asynchronously, so you should use them with the
expectation that they transfer data regularly and efficiently, but not
instantaneously. If you need to do real-time data transfer, you should
do it in an AsyncTask or an IntentService.
Related
This is just a information question. I'm new to Android app development and currently I'm working on my first app and and it is ready for the release. Now I'm concerned about how to handle heap of users and where to save all their details my app is a service booking app so it needs to save all the order details products details and lots other stuff.
Currently I'm using cloud firestore to load and save all the data of app. But I'm having some issues like without authentication it won't allow users to access some of my data and other. I wonder how large apps save their data and load them perfectly.
I wish someone will help me how can I save all my app data and load them perfectly in app. And suggest me for a best way to manage large user base. And other stuff.
First of all, firestore is good option if you don't have complex backend logic on the database. For simple CRUD operations on data firestore is a good choice but as you said you have a bulk of data then you must go for the Backend database and then connect your database with Rest API. So that all your complex queries will be done on the backend and you can simply consume your API in the app.
If you have lots of data from different users, maybe you should use a central server(DB), something like Postgres or MySQL should work fine.
At the same time, you can also do some sort of caching to accelerate the fetching process, like create a small database locally(you can use Room) to store some user specific data.
The Setup
I have native iOS and Android apps which sync data to and from my webserver. A requirement of the apps is that they work offline so data is stored on the apps in sqlite databases.
The apps communicate with the server with a series of REST calls which send JSON from the server for the apps to store in their databases.
My Problem
The scale of this data is very large, some tables can have a million records, and the final size of the phone databases can approach 100mb.
The REST endpoints must limit their data and have to be called many times with different offsets for a whole sync to be achieved.
So I'm looking for ways to improve the efficiency of this process.
My Idea
An idea I had was to create a script which would run on the server which would create an sqlite file from the servers database, compress it and put it somewhere for the apps to download. Effectively creating a snapshot of the server's current data.
The apps would download this snapshot but still have to call their REST methods in case something had changed since the snapshot happened.
The Question
This would add another level of complexity to my webapp and I'm wondering if this is the right approach. Are there other techniques that people use when syncing large amounts of data?
This is a complex question, as the answer should depend on your constraints:
How often will data change? If it is too often, then the snapshot will get out of date really fast, thus apps will be effectively updating data a lot. Also, with the big volume of data, an application will waste CPU time on synchronization (even if user is not actively using all of that data!), or may become quickly out of sync with the server - this is especially true for iOS where Applications have very limited background capabilities (only small window, which is throttled) compared to Android apps.
Is that DB read-only? Are you sending updates to the server? If so, then you need to prepare conflict resolution techniques and cover cases, in which data is modified, but not immediately posted to the server.
You need to support cases when DB scheme changes. Effectively in your approach, you need to have multiple (initial) databases ready for different versions of your application.
Your idea is good in case there are not too many updates done to the database and regular means of download are not efficient (which is what you generally described: sending millions of records through multiple REST calls is quite a pain).
But, beware of hitting a wall: in case data changes a lot, and you are forced to update tens/hundreds of thousands of records every day, on every device, then you probably need to consider a completely different approach: one that may require your application to support only partial offline mode (for most recent/important items) or hybrid approach to data model (so live requests performed for most recent data in case user wants to edit something).
100mb is not so big. My apps have been synching many GBs at this point. If your data can be statically generated and upated , then one thing you can do is write everything to the server, (json, images, etc...) and then sync all on your local filesystem. In my case I use S3. At a select time or when the user wants to, they sync and it only pulls/updates what's changed. AWS actually has an API call called sync on a local/remote folder or bucket. A single call. I do mine custom, but essentially it's the same, check the last update date and file size locally and if it's different, you add that to the download queue.
My Android app is fetching data from the web (node.js server).
The user create a list of items (usually 20-30 but it can be up to 60+). For each item I query the server to get information for this item. Once this info is fetched (per item), it won't change anymore but new records will be added as time go by (another server call not related to the previous one).
My question is about either storing this info locally (sqlite?) or fetching this info from the server every time the user asks for it (I remind you the amount of calls).
What should be my guidelines whether to store it locally or not other than "speed"?
You should read about the "offline first" principles.
To summarize, mobile users won't always have a stable internet connection (even no connection at all) and the use of your application should not be dependant on a fulltime internet access.
You should decide which data is elligible for offline storage.
It will mainly depend on what the user is supposed to access most often.
If your Items don't vary, you should persist them locally to act as a cache. Despite the fact that the data mayn't be really big, users will welcome it, as your app will need less Internet usage, which may lead to long waits, timeouts, etc.
You could make use of Retrofit to make the calls to the web service.
When it comes to persisting data locally within an Android application, you can store it in several ways.
First one, the easiest, is to use Shared Preferences. I wouldn't suggest you this time, as you're using some objects.
The second one is to use a raw SQLite database.
However, I'd avoid making SQL queries and give a try to ORM frameworks. In Android, you can find several, such as GreenDAO, ORMLite, and so on. This is the choice you should take. And believe me, initially you might find ORMs quite difficult to understand but, when you learn how do they work and the benefits they provide us, you'll love them.
I am working on an android fitness app and I want to get the hang of using ContentProviders. I was thinking about using the myfitnesspal api to get a list of exercises, but the api is private and my request has not been addressed yet. Then I considered scraping exercises from a website--but I am a little concerned about the reliability of this approach (if the site goes down, app won't keep working).
What is the best way to go about this? Is it "safe" to get information from a website (rather than an api) with a ContentProvider?
Correct me If I am wrong. You want user to acces your data even if there is not web API active. So my thought for it would be
Download all the data from web API and store it in database
create an node to check if data has been updated in the API if updated download new data in background and update the database else show same data from database
Benefits
User don't need to download data all the time they use app and their volume of internet would be saved
User don't need to see blank page if there's not any data
If im wrong, i would really like to know.. but the answer is simple: a ContentProvider allows only the connection to a database in your cellphone. You can't use a ContentProvider to get data from the internet. What you're trying to do is achievable with a WebService, which is an application running on a determined domain on the Internet and allows you to call some pre-defined methods linked to URLs of this same domain (but i imagine you know about that, right?).
I would ask a semi-theorical question about web services and client-server architecture.
I have a server with a database with about 50 tables. This server holds even one table which contains information about users associated with the clients.
Each client has associations with a subset of all data in db.
The defined architecture implies that each client (running on Android app) calling ,with a predefined frequency, a certain number of Web Services, fill the local database, creating a copy of all the tables residing on server sb, containing the only information related to the requesting client.
Furthermore, the data in db server side can change, so frequent synchronization is required.
Considering that the client application can be imaginated as a shop online application, so should be possible browsing through providers, articles, make a order,remove orders etc.
So, when I talk about fill the local database I mean store providers, articles, ...that is, all information realated to the requesting client.
Can make sense fill the local database with this information?
I think that is more reasonable call a web service only exactly when the information is required, and not store information in local db.
So, synchronization is no more required.
Tall me what you think about ? thank you.
I know NOTHING about Android development, but have tons of experience with using web services and SOA.
In my experience, especially when the client device has limited storage and processing power, all the business logic and data logic should be in the web services, and the client app used only for display and calling those services. How you implement that is something that we can't answer for you. It's different on every project.
The only exception is when you absolutely have to have the app running while not connected. In that case, be sure you know your specific requirements, and be very dilligent to only persist at the client the data that you absoutely need, and only provide the business logic that you absolutely need while disconnected.
(This helps to keep security simpler as well - lost or stolen devices are one of the highest sources of data breaches, so the less potentially sensitive data on the device the better.)
I don't know if that's helpful, but I throught I'd throw it out there.
If your application is going to work only online, then YES. You don't need to worry to store the data into the database and retrieve it later. If your requirement that needs to support both online and offline, then you have to go withe the database.
You can run a service or Async task or Handlers to invoke the Webservice from the Android Application, that way your UI thread won't be blocked.
If your response is too big, then probably you have to think about middle tier, that way you can convert the SOAP response to JSON that would be easier for the device to process the response and network transmission is faster.
I believe based on your requirements, its good enough to support only online mode... Because the data might vary if the user is not using the app for two days and logs in back and show offline data that might be irrelevant