I am committing a transaction to database. If the client loses wifi connection during transaction but after after commit it will not know the transaction completed. The problem is that the server should only commit the transaction permanently if the mobile app agrees and the server knows that the 1) mobile app has agreed and also 2) mobile app is aware that the transaction has been completed. The solution would seem to be for the mobile app to send a confirmation message that it confirmed the transaction, but what if a disconnect issue occurs after its confirmation is received on the server but before the server sends the 200 back? Now server thinks mobile app is aware but mobile app does not know for sure that the server got this message. This almost seems like handshake of some type because both sides must know what is on each side. The problem is how to handle transactions in unreliable networks where the server needs to know that the client both accepts the transaction and is aware that the transaction has completed before making the transaction permanent.
This is a Two General's Problem, and as such is unsolvable. A confirmation's confirmation is just yet another message to confirm and thus given finite time going back and forth you'll never perfectly confirm the original message is sent and both parties know it for sure.
In practical life, you will have to pick what kind of risk you accept. On low-risk messages like friends agreeing to meet each other, one confirmation reply is usually enough for them to proceed, despite the chance that the reply never actually made through. Now think about paying through GPRS-based EDC. There's always a possibility the connection gone off right after completely delivering the payment instruction and doesn't come back before the customer decides to leave. Without the confirmation receipt printed, the store will usually treat the payment as never completed, only reverting the transaction later. Both the store and the customer accept this risk.
The "Your Coffee Shop Doesn’t Use Two-Phase Commit" post is a really good read when designing your system. Consider the requirements. Is rollback unacceptable? Is there a certain level of inconsistency that's tolerable, or even expected? Maybe you can even reduce the client's role to merely attempt to send the command and then read-back from the server, discarding it's own.
Related
so I'm working with realtime database and I'm trying to make a chat app for practice.
I want to add the message to my list then let the message item inside the list upload the message to the server, I'm also showing an indicator that tells the user if the message is being uploaded, there are other approaches to apply this but I want to go with this one, anyways.
there isn't any problem when there is an internet connection. the problem is when there isn't any internet connection, the message is added to the list and the indicator appears, when the internet connection returns everything works fine still.
but if I send a message (while offline) and then leave the chat room and return to the chat room, the messages will get loaded and the indicator won't appear altho it isn't uploaded to the server (the data is cached now).
I want to find a way to tell if the data has been uploaded or not? I don't want to check the server to see if the node exists, I can't do that to every message it will cost too much, thank you.
If you enable disk persistence, Firebase keeps all of its pending writes in its disk cache. When the app restarts, it reads those pending writes and starts trying them. This is usually the right behavior for your users.
Unfortunately there is no built-in way to persist completion handlers for the Realtime Database. So upon a restart it becomes impossible for you to detect when the pending writes have been committed on the server.
So this typically means that you need to do something custom to detect the situation, and will have to determine for yourself whether the use-case is worth the effort.
If your messages are in some way ordered/timestamped (for example, if you add them by calling push()) you can keep track of what the last message is for which you received a confirmation from the server. That way you will know when the client restarts, which messages may not have been sent to the server yet.
Your onDataChange or onChildAdded will be called for those unconfirmed message straight away though when your app restarts, so you'll need an additional mechanism to detect when those unconfirmed messages are written on the server.
The best approach I know if is to write a "dummy" message when the app starts. Since the pending writes are treated as a first-in-first-out queue, your new dummy message write will be sent to the server after all the pending writes from the previous run. So when your completion handler gets called for this dummy message, you can be sure that all messages before it have also been committed (or rejected in case they violate your security rules).
Firebase cloud functions fires an onFinalize event when a file has been uploaded to the storage. So you could probably write a cloud function like this.
exports.uploadedServer = functions.storage.object().onFinalize((object) => {
const filename = object.name
//mark this filename or filekey as upload complete
return
})
You should be able to find more explanation here.
I have a system where one user can be logged into multiple devices. The use case is as follows:
Suppose the user A subscribes to a weather topic from device A. This user will now get regular push messages from this topic.
Now assume that the same user is logged in to a device B. This device also needs to get subscribed to the weather topic as there is no gurantee that the user will just use device A.
Same use case can be applied in reverse case of unsubscription as well as more than two devices.
What is the best possible way to solve such a problem.
The current approaches which i have thought of are:
1) Make a node under each user id inside firebase db called subscriptions. This node will have subscribed data in the form of
Topidname :boolean issubscribed
This approach has a listener attached to this location in a background service. Each device with the same user id will always listen to the same location thus solving the issue in most cases. It can't gurantee integrity due to the normal service lifecycle reasons.
A service is used because if the listener is kept only for the lifetime of the app, if the user unsubscribes from a topic on one device and the second device app is closed, the second app will keep receiving notifications as it's registration token is still related to that topic and will only unsubscribe on app restart.
Here the service has been made START_STICKY, has a listener inside onCreate and has a backup alarm set to restart the service in onDestroy method. Thus the only case where this won't work is an app force stop or the listener being in a backoff mode due to long non connectivity.
When the app launches it also syncs once with the location and subs the left out topics. Service will stop on signout and unsub from all the previous user's topics. I have kept a keepSynced on the location to ensure it stays in sync
Potential issues are -> service killed by force stop. May cause some ram usage. Sync might not be instantaneous depending on how long the connectivity was lost and how long it takes firebase to resync with the db. Resync and sub unsub all topics in each service restart.
2) This is a relatively more complicated approach:
Save the firebase token from the instance id service for each user in the app server.
On each signout, instead of calling delete iid, keep the same token but overwrite the user attached with the new logged in uid.
On each subscription, do a bulk subscribe to the topic using the Instance Id server api using all the registration tokens for one user.
Each time a user signs out call unsubscribe on all the topics one by one using the instance id api for that device's particular token. Same process for token refresh.
Do to same on unsubscription of each topic.
This case will need one synchronous call for all the topics subscriptions and unsubscriptions. It will also require the token to be constantly updated in the app server.
Please suggest any other better scenario to solve this issue.
If my scenarios have any flaws or needs modifications, please do suggest. I'm stuck and can't think of anything else at the moment.
I'm assuming that you're keeping track of each of the tokens for a user's devices. So the code on your server should be able to know, for a given user, how to send a message to all of their devices, regardless of subscriptions.
Try thinking of user topic subscriptions in the same way that you think of the data broadcasted within those topic. When topic subscriptions change for a user, that event could be broadcast to all of their devices with FCM, telling your app to read the subscriptions from your database and set subscriptions again as necessary. Your app will only wake a receive a notice to change subscriptions when the user requests it.
Then, when it comes time to publish a message to the topic, you can be sure that all the devices that checked in like this will receive the message to the topic.
To boil it down:
1. When a user's subscriptions change in your app, notify the server of that change.
2. In your server, update the user's subscriptions to your database.
3. Then, send an FCM data message to all of the user's devices to reload subscriptions.
4. When that message is received in your app, read the subscription updates out of your database and update the FCM subscriptions accordingly. Your app will have to arrange to stay alive for as long as it takes to do this.
I have an android app that sends and receives information from a vb.net web service. Things are great if there is a solid and fast internet connection but otherwise, everything becomes shaky.
The process goes like this:
1. The app sends information (such as an ID)
2. The web service returns information based on the ID (like a name)
3. The app updates the database within it based on the info sent by the web service.
The problem comes when the internet connection is weak. What happens is the app still sends the information but sometimes the information being received is incomplete or a connection timeout occurs; and the app still goes into step 3.
The app shouldn't be updating the database because the transaction was either incomplete or with errors. Any ideas on how I should go about this?
I was thinking of making sure that the transaction is complete before updating the DB. So the web service also sends a sort of 'complete' signal to tell the app that it's time to update the DB. Is this a good idea or is there a better way to go into this?
As for the connection timeout issue any way to catch that? I've seen a catch for the timeout but are there other alternatives which are better? Also if the connection times out what happens to the data that was sent and received?
Much obliged
I've got a large mailbox being managed by a custom android Exchange EWS client.
Due to client constraints the mailbox is only syncing 100 messages at a time. In one test account there are over 10,000 messages.
The problem I'm experiencing is that when new messages arrive and they are marked as read in a browser based client, the read state change doesn't seem to be being sent to the Android client unless the mailbox has already been fully synced.
When the mailbox isn't fully synced, it doesn't seem to receive the "read state" changes until all the items from the mailbox are synced.
What's the best way to solve this problem? I'm thinking I have to sync all message headers to the android client. This may clear the sync queue, so the read state changes are ready to be received.
Any suggestions?
The answer is as I expected - I needed to sync all messages (just their headers) to the client. I discarded the messages that fell outside of my desired date range. Then the read state changes came through automatically.
I have developed android apps, and have a web server application which serves REST style JSON, to the apps.
My apps are strongly dependent on that web services but as traffic gets higher, users' complaint started, as force close problems. I am not sure but maybe my server (AWS small instance) may not answer all requests correctly or in time.
I am planning to retry the web request when a problem on getting json response arise instead giving the error/net-connection alert.
I guess there are many developers who integrates apps with web services, so what is the good practice on handling network problems?
Or is the frequency of such network problems acceptable?
I take about 10-20 problem per day.
I have about 200.000+ web requests per day, for a AWS small instance (1.7 RAM), dedicated to server Tomcat. I analyze the logs there is no clue, no error log. Also the errors are spreaded.
You need to start with analyzing the problem, and determine the root cause or root causes of your issues. You always need to take into account that
a network connection might drop
a users switches from 3G / WiFi
the android devices "thinks" it's connected while in fact it's not
Also, be very sceptical when using the Android ConnectivityManager / NetworkInfo. Only trust it when it states that it is not connected. If it is connected, check it yourself (as sometimes, user is on a hotspot and the only connectivity he has is with a login page).
The application needs to handle all these scenarios properly. The way it's presented to the user depends on the use-case (do you want the user to be informed of the error, do you silently ignore it and just retry, ....)
In terms of retrying webservice connections, there are several ways to implement this :
exponential backoff
periodic rescheduling
event-driven triggering
retry-after moratorium intervals
You need to start by putting sufficient logging both on the client (Android) and on the server (AWS) so that you can analyze the issues and draw the proper conclusions.
I think the answer to your problem lies in the design of your android app.
You need to take into consideration the worst case scenario and redesign your application to take that into account and recover. Dealing with the chaos monkey - jeff atwood.
Personally I never allow an android app to be in a state where it needs to force close. For any or all network connection I assume that the connection is down, lossy, not all data can be retreived and (finally) up and working correctly.
That way my app will degenerate gracefully. If it needs web access it'll make an attempt in a background thread allowing the user to continue using the app, it will cache previous requests and will retry until it gets a connection or gives a nice toast to the end user.