We'd like to have our web app available offline, mainly on mobile devices. We've written code for that, using a service worker. Application data is stored in an IndexedDb and the application code (html, js, css, etc) is stored in the SW cache. So far so good. We are aware that the user can delete the browser cache and our data, that's not a problem. But what about the browser itself wiping the app data ? We haven't found a comprehensive specification for that, the main info we found are:
1) the StorageManager feature that is currently marked as "experimental" (since 2016);
2) a short article from Google here about it (also from 2016).
The code sample is the following:
if (navigator.storage && navigator.storage.persist)
navigator.storage.persist().then(granted => {
if (granted)
alert("Storage will not be cleared except by explicit user action");
else
alert("Storage may be cleared by the UA under storage pressure.");
});
The Google article says:
When storage on the local machine is running tight (“under storage
pressure”), user agents automatically clear storage to make more
available space. Of course, for offline apps, this may be unfortunate,
as they may not have synced their data to the server yet, or they may
be apps that the user expects to just work offline (like a music
player); so the Storage spec defines two different modes for storage
for a given domain - “best effort” and “persistent”. The default mode,
of course, is “best effort”. Storage for a domain that is “best
effort” (aka “not persistent”) can be cleared automatically, without
interrupting or asking the user. However, “persistent” data will not
be automatically cleared. (If the system is still under storage
pressure after clearing all non-persistent data, the user will need to
manually clear any remaining persistent storage.)
...
Beginning with Chrome 55, Chrome will automatically grant the persistence
permission if any of the following are true:
The site is bookmarked (and the user has 5 or less bookmarks)
The site has high site engagement
The site has been added to home screen
The site has push notifications enabled
The permission is automatically denied in all other cases.
The goal is to ensure that users can rely on their favorite web apps
and not find they have suddenly been cleared.
That's for Chrome 55, let's suppose the information is up to date. A first glance, their goal sounds reasonable, but if you take a closer look the implementation is geared for "big" sites (à la Google) and not for niche applications that are more task-oriented.
Indeed, when testing on various Android phones on Chrome 80+, the persistence is always refused, with no user interaction. So, "best effort" it is.
We could have stopped the investigation here and called it a day. After all, current phones and PC are sporting ungodly amount of storage, and we only use a few hundred of KB, so we should be fine. Problem is, we're not: testing on a brand new flagship Android phone with Chrome, our code is erased only with a few seconds of fiddling (closing and opening the page a few times is enough). On other platforms it's different, but Android+Chrome will get the most use.
Oddly, only the code in the SW cache (<100KB) is erased, and the bigger IndexedDb is not. So we tried to also put the code in the IndexedDb, and it seems more "persistent" that way, but the code to manage that is also more involved, so it feels somewhat hackish.
Are we alone with that problem ? If not, how are you people dealing with it ?
Bonus question: is there more up to date documentation on the subject somewhere ?
If I understand you correctly, the main issue is that a Chrome browser on Android keeps emptying the browser cache for a website which does not fulfill the Chrome conditions for automatically granted persistence permission.
Up to now, I have not been in this situation yet, but I observed the behavior, you are quoting from https://developers.google.com/web/updates/2016/06/persistent-storage - I just did not know up to now that it explicitly documented.
I see three ways how some websites enforce persistent user data storage:
Repeatedly ask the user to add the website to the home-screen and / or to enable push-notifications. I observe that this request comes often somehow under false-flag, i.e. something as "Subscribe to notifications to show your appreciation" rather than "Are we allowed to store data persistently". It may even go so far that a "Install our app" essentially means that a full-screen Chrome instance is added to the home screen of a mobile device, rather than a real app from an app store.
Some sites offer a Chrome extension which allows them to store stuff there and even gain more access rights. I personally do not suggest that approach, it somehow creates a strange feeling for security-aware users.
Yet another alternative would be an hybrid approach that you offer a native app which in fact is just a customized browser. Bear me with me a second, if this sounds strange at first glance. This option is in fact quite readily available e.g. in React Native as react-native-webbrowser component. It obviously requires programming effort, but quite a few news sites seem to use as approach.
Option 1 and 2 both stick with Chrome, but clearly do not apply for you, since you just want to avoid bothering the user.
Option 3 is unconventional, but may be an option worth considering. Only users who are programmers might realize that they are being somewhat fooled to install something which is essentially just a browser. Nevertheless it is indeed a clean solution: The app store takes care of access right control and you give the user the full choice what happens with the data.
Related
I'm new to app dev...
I read somewhere in the doc: "...In most cases you want to use SharedPreferences as it is automatically backed up and migrated to new devices..."
On first install my app saves a few settings with SharedPreferences. It works great but if I uninstall the app or install it on another device the preference settings are lost.
How can I have these settings saved online within google somehow to be able to retrieve them if the user changes his phone or similar...
Could someone point me in the right direction ?
I read somewhere in the doc: "...In most cases you want to use SharedPreferences as it is automatically backed up and migrated to new devices..."
Your words "backed up and migrated to new devices" is nowhere written nor its true.
We use SharedPreferences in order to minimise the database operations, its like keeping variables handy.
On first install my app saves a few settings with SharedPreferences. It works great but if I uninstall the app or install it on another device the preference settings are lost.
If you wants to store or remember the device dependent settings, use device id / imei_id and store it on your web server mysql database
Even if user uninstalls app from the device and installs again anytime in future, make a call with async task to server by sending deviceid / imei_id and fetch its settings from mysql database and show it.
How can I have these settings saved online within google somehow to be
able to retrieve them if the user changes his phone or similar...
If user changes device, you can do nothing.
One way is, keep public device_id levels keys on server.
If user changes device and uses that key, then show him a response, this key is assigned to another device, but if you are the same, wait for our support
Call him, confirm he is the same old user with new device and delete his old entry from mysql and assign old key to the new device entry
Or use OTP SMS system to identify already existing customers with unique phone numbers
If OTP authentication code is correct then fetch settings for that user from the server, delete old mysql entry, modify new entry with old key and mobile number
This should be the your direction
Edit : 2 ##
I was hoping an easier solution exist but....
There is no short cuts for developers till the date, and it will be never.
Why, no short cuts / easy ways ?
Any device ( mobile, desktop / laptop / any AI device ) which is operated by a system software, is able to perform the tasks as per it is structured.
Ex : android is java based, obviously you can Make javascript based apps, but it is the extensions to the existing system, Android still has the base of Java virtual machine. ( Dalvic / Malvic like )
So, it is always better to use native java
Yes, Kotlin is best option now a days and better than hybrid approach
Every way has its own advantages, disadvantages
If you are developer, should go with native approach
Now your java code never knows, which version it is running on, so you have to, check android versions programming wise, and decide the flow for above Marshmallow & below marshmallow too, and it is explicitly done by developer by coding.
Ex, once user registers, he never shown please register again screen, it is not the magic, nor google, nor, java, nor android does anything, developer has decided, planned, architectured, designed, coded, tested that.
Even developers needs to take care of exceptions, you need to handle it in order to save your app from crashing.
In short developer is god, who creates his own universe, and everything is pre-planned and verified thats it.
You should use allowBackup = "true" in your manifest file. More details can be found here: AutoBackup
Hi I'm using the Google Drive Api to store a database using the AppDataFolder facility. I have a test app running successfully on one device. I am able to upload/update/delete/download the database file and reintegrate it into the program with no problems.
The issue I'm having is when I want to share this database with another device running the same app, then things don't work as expected. Example device A I upload the database, device B - I want to download the database but no file is found (this delay can vary greatly from seconds to hours). Reason for this - when using the Api it decides when it wants to 'sync' data, as it is queued rather than being instantaneously uploaded. So when used on one device this is not a problem because it takes either the 'synced' file from the cloud storage, or file waiting to be synced.
I have tried various things like trying to list all AppDataFolder files or retrieving metadata through a query with searchable filters before making a request to update/delete etc.. However I can't get it to work as desired fundamentally it chooses when to sync.
So my actual question is: How can I force Google Drive to sync my file when I want it to i.e. every time a request is made, so that synchronisation is achieved across multiple devices using the same app. There must be an answer as I would think this is quite a fundamental reason why you would use the AppDataFolder is the first place.
Thanks in advance
EDIT/UPDATE:
I have been able to find an option in the Api to 'sync' the drive content using this code:
// try to sync
Drive.DriveApi.requestSync(mGoogleApiClient).setResultCallback(new ResultCallback<com.google.android.gms.common.api.Status>() {
#Override
public void onResult(com.google.android.gms.common.api.Status status) {
if (!status.getStatus().isSuccess()) {
Log.e("SYNCING", "ERROR" + status.getStatusMessage());
} else {
Log.e("SYNCING", "SUCCESS");
// execute async task to list AppFolderContents
new AppFolderContentsAsyncTask(getActivity()).execute();
}
}
});
This works well for 3/4 attempts in quick succession, however I reach a syncing limit and status message:
ERRORSync request rate limit exceeded.
Is there any way to increase the request rate as this is not really desirable if I have to have a app that prompts the user 'please try again later to sync - not sure how long you'll have to wait until you can though!'
SOLUTION - OF SORTS, AND MY THOUGHTS (for what its worth)
The solution that I am going for (after emailing a app dev whose published a drive app that synchronizes without problems) is to use the Drive REST Api, rather than the newer (and Google preferred) Drive API. I tried limiting the 'requestSync' to only when the user navigated to a fragment with the Drive options (rather than every file transaction). However this would probably solve the requestSync rate limit for the most part, but it still could hit that limit. Also if multiple devices are running the app, and linked to the same Drive account were both syncing/uploading/deleting files at the same time in the app then there is a possibility of losing synchronization - maybe a rare scenario, but still possible. I don't feel that making a user wait to sync files is a viable option in terms of user experience or app design.
Curious though - The actual Drive app lets you refresh (requestSync?) as many times as you like. I created 20 folders on the web interface in quick succession, after each folder was created I refreshed the Drive app on my phone and it synced all 20 times. It seems Google understands the importance of synchronization, but chooses to make this quite difficult ensure this in their new Drive API. As already stated uploading files to the cloud storage happens usually instantly (it is queued, however if you have connectivity it happens almost straight away). I would have thought that this is the costly transaction in terms of moving data/updating drive contents, rather than just querying for synchronization of files. However you can keep adding files to your your drive app one at a time, and it uploads them one at a time almost instantly - where as you request sync more than 4 times in 30 seconds and it then fails and you have to 'cool off' for a while.
I'm very new to programming/Android in general - as soon as I learn something new, I realize how little I actually know, so if their is a better solution out there using the Drive API (rather than REST API) I'd very much welcome it.
DriveApi#requestSync will let you request a "sync down" from the server to the device. This way, on the second device you should be able to see the content uploaded from the first device. However, the calls to request sync are rate limited (per-device) to avoid abuse and guarantee a reasonable amount of data usage by the Drive API. You should ideally call request sync only when you need to. There's no way to increase the rate limit.
Regarding upload completion, after committing an upload, you shouldn't use request sync since that won't help you (it only syncs down but not up). The actual upload will happen as soon as possible (based on the device's connectivity and the preferences specified by your app through DrivePreferencesApi, which by default are unrestricted).
If you want to know when the actual upload happened, you can use CompletionEvents. This way you can test your app with more insights on what's actually going on.
Is there any possible solution to "protect" the webview cache from the "Clear Cache" option in the default web browser. Any time I clear the cache completely I lose my app data, I'd like to preserve it.
Is there any possible solution to "protect" the webview cache from the "Clear Cache" option
No. That runs foul on the principal of "users over authors". That means the user controls their data, and not the authors of the web app. If the user wants to delete the data, then its gone. See the W3C's HTML Design Principals, Priority of Constituencies.
I think this is going to be a problem when (1) WebCrypto goes live because it means provisioning information, like keys and certificates provisioned by a bank or organization, and can deleted at any time; and (2) the document you created and edited with your word processor offline web app was accidentally deleted.
For (1), I think WebCrypto intended to try and solve it with some storage specification. But I think they found it was too difficult and offloaded it to another working group (web storage).
(2) is kind of solved with "cloud storage", but that moves data out of your control and into someone else's control. Its not just Apple back dooring their products...
I'm doing an android application much like the gmail app for android. I can see the app is very fast and very responsive.
I'm sure gmail uses local caching for better performance. But I wonder how does gmail does the following use cases.
Lets say i login for first time and the app loads all email and put in sql lite or any other caching.
use case1
every time i delete or add a star it makes an asyncronous server req to server. this is fine.
use case 2.
how does the app get notified in case the user makes changes from another client. (from a browser).
To download the entire mails will be costly. Lets assume a case in which the user adds a star to a already cached email.
How can we implement the api in such a way that to get only updates which made from other clients. Updates like new email, deleted email, star added, etc. I'm considering the scenario when the user doesn't use the background sync.
This is been bothering me for a long time. Request you guys to let me know some tutorials or links to understand the secret behind the gmail app.
Why don't you look at the gmail web app. That will be using the same/similar design principals (gmail web app is also very fast for me). Then you can apply the same principals. With the web app you can look at the web requests using something like firebug with firefox. Under the Net tab it will show the json of what is being sent across the network. It will also show you the way the request was structured. You wont be able to see how they go about optimally returning the data, but that should be fairly trivial.
I would recommend for returning data that you do the following:
1) create a cache for the most recent page of emails.
2) Updates of new emails will be immediately prefetched into the cache.
3) setup a cache for the next page and previous page of emails.
Essentially what I am saying is:
1) You can look at googles web version which is very visible to plain snooping.
2) You can optimize based on statistical usage of how the user would want to see their emails. This can suggest cache improvements.
What you are asking for in terms of links is harder because your question relates to many general concepts and proriatary software.
During my activity I'm sending an intent to the browser in order to display a webpage :
Intent i = new Intent(Intent.ACTION_VIEW, Uri.parse("http://ww.mywebpage.com");
startActivity(i);
I need to make sure that before sending the intent the browser cache and history are cleared so that the page get loaded from server directly and not from phone.
So far I've found the 2 following but I'm not sure they are used correctly :
Browser.clearHistory(getContentResolver());
Browser.clearSearches(getContentResolver());
Also with that cache is not cleared.
Do you know how to do that ?
First, you are assuming there is only one Web browser for Android. You are mistaken, and will be increasingly mistaken over time. Steel, Dolphin, Opera, etc. are already in production for Android, and Mozilla's Fennec is coming along nicely. This solution will not help you with other browsers.
Second, if a browser is incorrectly caching your data, your problem is probably on the server (i.e., not sending proper cache control headers). I'd try to fix it there, so that it will behave properly across all browsers.
Third, wiping the user's entire history and searches, to satisfy your requirements, is rather unprofessional. How would you like some desktop app wiping out your desktop browser's history and searches?
Fourth, you cannot clear the browser's cache programmatically.
Yes... and if you must assert more control on the client side rather than fixing it at the server, you'll need to display the content with a webview inside your application where you have full control, rather than delegating to the browser (which is a separate application running under a separate user id and separate security context that you can't mess with).