I'm sending requests from my android phone one at a time as fast as possible to my server, and then graphing the time (x axis) vs hit number (starting at 0). After running this on Andrid for about 10 minutes, I get a graph like this:
The flat parts are unexpected. This signifies that every 3 minutes or so, there's a 1 minute lag that happens. Why might this be happening?
A few things I've noticed:
It doesn't matter if the screen is on or off
My server is not the problem. During the long pauses, I'm able to hit the endpoint on my server separately from android.
Both POST requests and requests over websockets have the same problem
It is possible the problem is my android data connection; I'm not sure how to remove this as a possible variable. It would be odd that the data connection would do this.
Related
This is not a question about concurrency or event loop of node
I have a question in how to magane a scenario in wich thosands of users are connected by socket and autentified by jswtoken to a Node.js server.
Let´s say there are between 10,000 and 50,000 users connected throught an Android app with socket.io client, all of them are pushed into an array users.push(newUser) when the connection is stablished by socket.io and removed on disconnect, users.splice(id,1) no problem there.
But all of them wants to update a var on his respctive object in the array each 5 seconds
I can idetify their respective object by his user.id using Array.prototype.find() but it takes too long.
In my tests (mocha,chai) it takes, to find and update X users:
For 1,000 users , 2XXX.XX ms (more than 2 seconds)
For 10,000 users , 4XXX.XX ms (more than 4 seconds)
For 25,000 users , 3XXXX.XX ms (more than 30 seconds)
Over that 4 seconds there is no way to keep a real time experience.
Is there any design pattern to walkaround this?
I have a few ideas but no one seems to be practical and scalable
Store in the client the position it is using currently (is not good because each time a user is disconnected i have to notify all other if index has changed or not)
Separate the users in multiple arrays of maximum 1000 positions and relate them to another mapping to do the .find() function directly on a relatively small array to search. (I don´t know if it is a good practice but the task of identify users relay on the server and it doesn´t take too long)
Totaly abandon Node and look for another solution (I would like to keep it in Node)
I don't think theres a simple answer to this question. Having 10 000 - 50 000 active users at the same time in a chat / game / what ever? Thats a massive payload and I really don't think your app has to worry about that kind of traffic.
My app has the feature to send a particular app data to a server once a day in every 24 hr.
Total number of my Android app installs on play store - 5,00,000 (approx)
Total no of seconds in 24 hr - 86400 sec.
I need to distribute the load on server over the entire day, in order to assure that all the installed apps are not sending data at the same time or in a short time window.
In short I need some logic on client side to distribute these calls to server evenly across 86400 sec.
A random number should give you enough spread with so many installs. So, first time app runs, call Rand(0, 86399), and save it in preferences. Then send sms at that second in day.
We had similar situation, and we switched this type of call from Apache server to node.js server, and never looked back. Node.js can handle many many more connections AND can return and close connection before all the work is done (so for example if data is not critical, you can close connection and return, before saving to database).
My code was working fine initially, then All of a sudden I'm not able to display posters on my app. I'm getting this particular error.
I've removed almost all while(true) loops. Please help.
You are probably making too many API requests.
From the TMDB.org FAQ:
We currently rate limit requests to 30 requests every 10 seconds.
You should make sure your app never uses more than that. How you implement this is up to you, but you could keep track of a request counter that:
decrements every time you make a request
resets to 30 or so every ten seconds
only allows requests to continue if it is positive
In general, you should try to query the API as little as possible: cache results in your app as much as you can.
I have an application that sends data to a server with a post request. This request can fail, and if it does I want it to retry until it's finally sent, something similar to WhatsApp: if u send a message when u are offline it stays as pendant and when you go online again the message is sent.
Since I don't know how WhatsApp internally works I have some doubts in how to implement that. I thought two ways:
1- Setting an AbstractThreadedSyncAdapter to be executed every X time (like 30 seconds) that checks if there is data to send and, if there is, it sends it to the server.
2- When the user clicks to send some data, I create a thread that tries to send it and, if it fails, it sleeps some seconds and try again.
I really don't like any of these options. The first one is going to increase the battery usage since the application is going to perform operations every X seconds even if it isn't needed. The second one is going to use a lot of battery if the request fails a lot of times.
Is there any better way to do it? It'd be awesome if there was an easy way to detect if the phone has connection to internet.
Thanks!
In your scenario 2, you can set an alarm when the initial post fails to trigger a re-sent some time later. If the send succeeds, you cancel the alarm (or don't schedule another one).
For getting noticed when the device goes online you may look at this answer: https://stackoverflow.com/a/11084311/100957
Recently google introduced push-to-device service, but it's only available 2.2 and up.
I need a similar system in my app, and I'm trying to get around limitations.
The issue is battery life. Since the user must be notified immediately about the changes on the server, I thought to implement a service that would live in the background (standard Android service) and query the server for updates.
Of course, querying the server, even each second, will cost a lot of bandwidth, as well as battery, so my question is this: does it make a difference if the server is holding the response for some period of time? (the idea behind Comet type ajax request)
Works like this:
Device sends request for data update
Server gets the request and goes in the loop for one minute, checking if there are updates on each iteration
If there are updates, server sends response back with updates
If not, service goes on to the next iteration.
After a minute, it finally sends the response that no data is yet available
After response (no matter whether empty or with data) Android fires another such request.
It will definitely cost less bandwidth, but will it consume less (or even more) battery?
Holding a TCP socket (and consequently waiting for an HTTP response) as you suggest is probably going to be your best option. What you've described is actually already implemented via HTTP continuation requests. Have a look at the Bayeux protocol for HTTP push notifications. Also, check out the Android implementation here. For what it's worth, that's definitely what I would use. I haven't done any sort of analysis of it, but this allows you to minimize the amount of data transmitted over the line (which is directly proportional to the power consumption) by allowing the connection to hang for as long as possible.
In short, the way Bayeux works is very similar to what you've suggested. The client opens a request and the server waits on it. If it has something to send, it sends it otherwise it simply waits. Eventually, the request will timeout. At that point, the client makes another request. What you attain is near instantaneous push to the client from the server without constant polling and duplication of information like HTTP headers, etc.
When the phone is actively using the networks, it's battery is used more. That is to say when it sends the request and when it receives the response. It will also be using battery just by listening for a response. However, will the phone download data, checking to see if there's a response? Or will the phone just be open to receiving it and the server will push the response out to the phone? That is mainly what it depends on. If the phone is just open to receiving the response but does not actually use the network while trying to download some response the whole time it's waiting, it should use less battery.
Additionally, the phone sending a query every minute instead of every second definitely uses less battery, as far as using the networks goes. However it depends on how you make the phone hold, if you tie it up with very complex logic to make it wait it may not help battery life. That's probably not the case, however, and I would say that in all likelihood this would work out for you.
In closing, it should help the battery but there are ways you could make it in which it would not. It wouldn't hurt to write the program and then just change some type of variable (for example WAIT_TIME to 1 second instead of 1 minute) and test battery usage though, would it?