Develop Reference

Implementing long polling with retrofit and kotlin channels

I am trying to implement long polling which is lifecycle aware(in Activity/Fragment). The polling will be scoped to the fragment which sends API request to the server every fixed interval of time. However, I am unable to implement it.
This is how I want the implementation to be like
Have a hard timeout on client-side without considering any extra delay incurred in receiving the response.
Wait for the response of previous API call before sending the next request. i.e., the request in polling queue should wait for response irrespective of its priority due to polling interval
Consider:
HARD_TIMEOUT = 10s
POLLING_INTERVAL = 2s
Request 1: Started at: 0sec Response delay:1.2sec Duration left: 0.8sec
Request 2: Started at: 2sec Response delay:0.4sec Duration left: 1.6sec
Request 3: Started at: 4sec Response delay:2.5sec Duration left: 0sec
Request 4: Started at: 6.5sec Response delay:0.5sec Duration left: 1.0sec
Request 5: Started at: 8sec Response delay:0.8sec Duration left: 1.2sec
For this use case, I want to use polling instead of socket. Any Idea/solutions would be appreciated. Thank you.

Ok, Figured out a solution for polling using channels. this should help someone searching for an example.
private val pollingChannel = Channel<Deferred<Result<OrderStatus>>>()
val POLLING_TIMEOUT_DURATION = 10000L
val POLLING_FREQUENCY = 2000L
A channel is required to hold your asynchronous request just in case more request comes in while your async task is being executed.
val pollingChannel = Channel<Deferred<Pair<Int,Int>>>()
QUEUE EXECUTOR: It will pick an async task and start executing them in FIFO order.
CoroutineScope(Dispatchers.IO).launch {
for (i in pollingChannel) {
val x = i.await()
println("${SimpleDateFormat("mm:ss.SSS").format(Calendar.getInstance().time)} Request ${x.first}: value ${x.second}")
}
}
POLLING FUNCTION: Adds your async task to the polling channel every fixed interval of time until timeout.
CoroutineScope(Dispatchers.IO).launch {
var reqIndex = 1
val timedOut = withTimeoutOrNull(POLLING_TIMEOUT_DURATION) {
while (receiverJob.isActive) {
pollingChannel.send(async {
getRandomNumber(reqIndex++)
})
delay(POLLING_FREQUENCY)
}
}
}
ASYNCHRONOUS OPERATION
to avoid answer being verbose I created a function with a random delay, please replace with the required API call
private suspend fun getRandomNumber(index: Int): Pair<Int,Int> {
val randomDuration = (1..6L).random() * 500
delay(randomDuration)
return Pair(index,(0..100).random())
}
SAMPLE OUTPUT

Why do I have multiple dangling OkHttp ConnectionPool threads

I have an Android app which is performing HTTP get requests in the background. I'll give some details of how this is performed shortly. I'll focus on describing the issue first.
I've observed in the debugger that OkHttp ConnectionPools are stacking up in the Threads window in Android Studio. This is concerning me as it seems to indicate a resource leak and over time these will increase indefinitely (it seems).
My main problem is that I have no idea why this would be happening. I'm almost certain that it's my fault, but I don't have a lead on what to look for. So my general question is:
"What might cause a new pool to be created each time a HTTP request is made?"
Firstly, here's the code I use to do the get request. Note that it's simplified, but I've run with this exact code and see the problem.
private TestResult test() {
final long startTime = SystemClock.elapsedRealtime();
final String server = "connectivitycheck.gstatic.com";
URL url;
try {
url = new URL("http", SOME_PATH);
} catch (MalformedURLException e) {
// We shouldn't get here since the url is well known
return new TestResult(
"failed to generate url " + e,
SystemClock.elapsedRealtime() - startTime
);
}
HttpURLConnection urlConnection = null;
int httpResponseCode;
try {
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setInstanceFollowRedirects(false);
urlConnection.setConnectTimeout(SOCKET_TIMEOUT_MS);
urlConnection.setReadTimeout(SOCKET_TIMEOUT_MS);
urlConnection.setUseCaches(false);
InputStream is = urlConnection.getInputStream();
httpResponseCode = urlConnection.getResponseCode();
is.close();
} catch (IOException e) {
return new TestResult(
e,
SystemClock.elapsedRealtime() - startTime
);
} finally {
if (urlConnection != null)
urlConnection.disconnect();
}
if (httpResponseCode != HttpURLConnection.HTTP_NO_CONTENT) {
return new TestResult(
"invalid error code, expected 204 but got " + httpResponseCode,
SystemClock.elapsedRealtime() - startTime
);
}
return new TestResult(
SystemClock.elapsedRealtime() - startTime
);
}
So, when is this code run...?
Firstly let me say that I've tried running this code from my home activity on my app (in a separate thread - as HTTP isn't allowed on the main android thread). When I do that, everything is fine. I can make multiple requests and never see more than one pool.
The situation in which I run the code and see the issues is as follows:
I have a background Service
In that service I start a new Thread
In that thread I create an ExecutorService:
ExecutorService executorService = Executors.newCachedThreadPool();
CompletionService<FutureResult> completionService = new ExecutorCompletionService<>(executorService);
Using that ExecutorService I submit a callable task which runs my test method (above).
Note that this may all sound convoluted but there's purpose to all of this. For one, the ExecutorService will be performing many other tasks other than just running the test method.
I've had a good look for hanging references that somehow might be keeping the underlying HTTP resources alive but I can't see any.
Does anyone with good knowledge of OkHttp know why this might happen? I'm hoping for some insight to help me uncover the problem.

The easiest way to find out is to set a breakpoint in the ConnectionPool constructor and run your app. That'll show who is creating it. My guess is it's an unrelated class like a crash reporter or analytics library.

RxJava. Sequential execution

In my Android App I have a presenter which handles user interactions, contains kind of request manager and if needed sends user input over request manager to request manager.
Request manager itself contains server API and handles server request using this RxJava.
I have a code, which sends a request to server everytime a user enters a message and show the response from server:
private Observable<List<Answer>> sendRequest(String request) {
MyRequest request = new MyRequest();
request.setInput(request);
return Observable.fromCallable(() -> serverApi.process(request))
.doOnNext(myResponse -> {
// store some data
})
.map(MyResponse::getAnswers)
.subscribeOn(Schedulers.newThread())
.observeOn(AndroidSchedulers.mainThread());
}
However now I need to have kind of queue. The user may send a new message before the server has responded. Each message from the queue should be processed sequentially. I.e. the second message will be sent after we've got a response to the first message and so on.
In case an error occurs no further requests should be handled.
I also need to display the answers within a RecyclerView.
I have no idea how to change the code above to achieve the handling described above
I see kind of problem. On one hand, this queue can be anytime updated by the user, on the other hand anytime server sent a response the message should be removed from the queue.
Maybe there is a rxjava operator or special way I just missed.
I saw a similar answer here, however, the "queue" there is constant.
Making N sequential api calls using RxJava and Retrofit
I'll be very thankful for any solution or link

I don't fnd any elegant native-RxJava solution. So I will custom a Subscriber to do your work.
For your 3 points:
For sequential execution, we create a single thread scheduler
Scheduler sequential = Schedulers.from(Executors.newFixedThreadPool(1));
For stop all requests when error occur, we should subscribe all request together instead of create a Flowable every time. So we define following functions (here I request is Integer and response String):
void sendRequest(Integer request)
Flowable<String> reciveResponse()
and define a field to make association of request and response flow:
FlowableProcessor<Integer> requestQueue = UnicastProcessor.create();
For re-run the not-sent request, we define the rerun function:
void rerun()
Then we can use it:
reciveResponse().subscribe(/**your subscriber**/)
Now let us implement them.
When send request, we simply push it into requestQueue
public void sendRequest(Integer request) {
requestQueue.onNext(request);
}
First, to do the request sequentialy, we should schedule work to sequential:
requestQueue
.observeOn(sequential)
.map(i -> mockLongTimeRequest(i)) // mock for your serverApi.process
.observeOn(AndroidSchedulers.mainThread());
Second, to stop request when error occur. It's a default behavior. If we do nothing, an error will broken the subscription and any futher items will not be emitted.
Third, to re-run the not-sent requests. First because that the native operator will cancel the stream, like MapSubscriber do (RxJava-2.1.0-FlowableMap#63):
try {
v = ObjectHelper.requireNonNull(mapper.apply(t), "The mapper function returned a null value.");
} catch (Throwable ex) {
fail(ex);// fail will call cancel
return;
}
We should wrap the error. Here I use my Try class to wrap the possible exception, you can use any other implementation that can wrap the exception instead of throw it:
.map(i -> Try.to(() -> mockLongTimeRequest(i)))
And then it's the custom OnErrorStopSubscriber implements Subscriber<Try<T>>, Subscription.
It request and emits items normally. When error occur(in fact is a failed Try emitted) it stopped there and won't request or emit even downstream request it. After call rerun method, it will back to the running statu and emit normally. The class is about 80 lines. You can see the code on my github.
Now we can test our code:
public static void main(String[] args) throws InterruptedException {
Q47264933 q = new Q47264933();
IntStream.range(1, 10).forEach(i -> q.sendRequest(i));// emit 1 to 10
q.reciveResponse().subscribe(e -> System.out.println("\tdo for: " + e));
Thread.sleep(10000);
q.rerun(); // re-run after 10s
Thread.sleep(10000);// wait for it complete because the worker thread is deamon
}
private String mockLongTimeRequest(int i) {
Thread.sleep((long) (1000 * Math.random()));
if (i == 5) {
throw new RuntimeException(); // error occur when request 5
}
return Integer.toString(i);
}
and output:
1 start at:129
1 done at:948
2 start at:950
do for: 1
2 done at:1383
3 start at:1383
do for: 2
3 done at:1778
4 start at:1778
do for: 3
4 done at:2397
5 start at:2397
do for: 4
error happen: java.lang.RuntimeException
6 start at:10129
6 done at:10253
7 start at:10253
do for: 6
7 done at:10415
8 start at:10415
do for: 7
8 done at:10874
9 start at:10874
do for: 8
9 done at:11544
do for: 9
You can see it runs sequentialy. And stopped when error occur. After call rerun method, it continue handle the left not-sent request.
For complete code, see my github.

For this kind of behaviour I'm using Flowable backpressure implementation.
Create outer stream that is parent for your api request stream, flatMap the api request with maxConcurrency = 1 and implement some sort of buffer strategy, so your Flowable doesn't throw exception.
Flowable.create(emitter -> {/* user input stream*/}, BackpressureStrategy.BUFFER)
.onBackpressureBuffer(127, // buffer size
() -> {/* overflow action*/},
BackpressureOverflowStrategy.DROP_LATEST) // action when buffer exceeds 127
.flatMap(request -> sendRequest(request), 1) // very important parameter
.subscribe(results -> {
// work with results
}, error -> {
// work with errors
});
It will buffer user input up to given threshold, and then drop it(if you don't do this it will throw exception, but it is highly unlikely that user will exceed such buffer), it will execute sequentially 1 by 1 like a queue. Don't try to implement this behaviour yourself if there are operators for thing kind of behaviour in libary itself.
Oh I forgot to mention, your sendRequest() method must return Flowable or you can convert it to Flowable.
Hope this helps!

My solutions would be as follows (I did something similar in Swift before):
You will need a wrapper interface (let's call it "Event") for both requests and responses.
You will need a state object (let's make it class "State") that will contain request queue and the latest server response, and a method that will accept "Event" as parameter and return 'this'.
Your main processing chain will look like Observable state = Observable.merge(serverResponsesMappedToEventObservable, requestsMappedToEventObservable).scan(new State(), (state, event) -> { state.apply(event) })
Both parameters of the .merge() method will probably be Subjects.
Queue processing will happen in the only method of "State" object (pick and send request from the queue on any event, add to queue on request event, update latest response on response event).

i suggest to create asynchronous observable methods , here a sample :
public Observable<Integer> sendRequest(int x){
return Observable.defer(() -> {
System.out.println("Sending Request : you get Here X ");
return storeYourData(x);
});
}
public Observable<Integer> storeYourData(int x){
return Observable.defer(() -> {
System.out.println("X Stored : "+x);
return readAnswers(x);
}).doOnError(this::handlingStoreErrors);
}
public Observable<Integer> readAnswers(int h){
return Observable.just(h);
}
public void handlingStoreErrors(Throwable throwable){
//Handle Your Exception.
}
the first observable will send request when he get response will proceed the second one and you can chain , you can customize each method to handle errors or success, this sample like queue.
here the result for execution :
for (int i = 0; i < 1000; i++) {
rx.sendRequest(i).subscribe(integer -> System.out.println(integer));
}
Sending Request : you get Here X
X Stored : 0
0
Sending Request : you get Here X
X Stored : 1
1
Sending Request : you get Here X
X Stored : 2
2
Sending Request : you get Here X
X Stored : 3
3
.
.
.
Sending Request : you get Here X
X Stored : 996
996
Sending Request : you get Here X
X Stored : 997
997
Sending Request : you get Here X
X Stored : 998
998
Sending Request : you get Here X
X Stored : 999
999

How to read available text (not ending with new line) from TCP socket using BufferedReader

I'm communication with a server through a tcp socket connection, i'm able to read lines that ends with \n fine, however when the line is not terminated (ends in \n) i'm not able to read it. I tried the following but it didn't work and caused my app to freeze at startup:
private Socket socket;
private BufferedReader input;
public boolean isConnected;
#Override
public void onCreate()
{
try
{
socket = new Socket ("server.ip.add.ress", 23456);
input = new BufferedReader (new InputStreamReader (socket.getInputStream());
handshake();
isConnected = true;
}
catch // Handle IOException and UnknownHostException
}
// custom runnable to read availabe input from the server
private class MyRunnable implements Runnable
{
private volativle String value;
public String getValue()
{
return value;
}
#Override
public void run()
{
int count;
char[] buffer = new char[10]; // expected message 'username: '
try
{
count = input.read (buffer, 0, 10);
if (count > 0) value = new String (buffer);
}
catch // IOException
}
}
// when connection is established with server expect 'username: ' from
// the server and send the user name back to it
public void handshake()
{
MyRunnable runnable = new MyRunnable();
try
{
Thread thread = new Thread (runnable);
thread.start();
thread.join();
String greeting = runnable.getValue();
if (greeting.equals ("username: ")) // Send username back
}
catch // InterruptedException
}
why is it hanging? and how can i read a non terminated line?
Edit:
To clarify: The server sends the greeting message username: immediately after the connection is established with a client, the client wait for the greeting and send back it's username when received (that's what handshake() does), if no handshake the client disconnects otherwise it start listening for incoming messages. Because i need to know if handshake is complete before starting the listener i had to use Thread.join().
The problem: Thanks for the comments and answers below, it turned out that BufferedReader.read() blocks the thread and waits until something is sent from the server and if nothing is being sent it causes the app to hang, Therefor there's no way to find out if the line has ended.
The solution: In my specific situation i just wanted to know if a specific message is sent "username: " so i used read (buffer, 0, 10) to read exactly 10 characters (the length of "username: "), and because it blocks if nothing is sent i used Thread.join (1000) which waits only one second and then if nothing received i disconnect the client.

Why is it hanging?
This is what it is suppose to be. It will block the thread if no data is available to read. This is also why you want to put it in a background thread.
Can it not just return if nothing is available?
What you are looking for is ready(), which will tell you whether there is available data or not.
Indicates whether this reader is ready to be read without blocking.
Returns
true if this reader will not block when read is called, false if unknown or blocking will occur.
But you should be very careful when using this function. Because networking is a lot about timing. The fact that you don't have any data to read at this second doesn't necessary mean that it won't be any data in the next second.
So a better design of the server should be more or less as the following:
If the username is found, return the username
If the username is not found, return an error message to let the client side know that the username is not found

There's no need for the thread. Your goal is to wait until you've read what you've been waiting for. Why not just let read() perform the wait for you?
What you're struggling with is the classic problem of TCP communication: "when do I know that I've got everything the server sent?"
In your case, you're expecting to read bytes until the collection of bytes ends with "username: ". So, change your algorithm to perform 1 byte reads (filling a buffer as you go) until that buffer ends with "username: ".
You can make a more complicated algorithm -- which would be more efficient -- that would attempt to read multiple bytes at a time and append them to a buffer -- performing your check each time. But either strategy is logically equivalent.
I also recommend just using the InputStreamReader. It has various read() methods. I am a bit suspicious about the BufferedInputReader, especially when dealing with data that isn't newline terminated. I'm probably just paranoid. I've just never used it when writing TCP client/server programs, so I'm not sure.

Is it a good way to call AsyncTask in infinity loop to pull data from server?

Well in app I'm trying to pull the data from sever for every 4 sec,and update the app.
I'm using handler,in that I'm calling AsynTask to fetch the data from server for every 4 sec.
Just I'm worried about the instance created for AsynTask every 4'sec causes any problem ?
This is what I'm doing.
private static final int DELAY = 1000 * 4;
final Handler printHandler = new Handler();
private boolean keepLooping = true;
printHandler.postDelayed(printStuff, DELAY);
Runnable printStuff = new Runnable(){
#Override
public void run(){
// call AsynTask to perform network operation on separate thread
new DownloadMainScore().execute("http://server/root/score.php");
if(keepLooping)
printHandler.postDelayed(this, DELAY);
}
};

On your choice of concurrency tool:
You are right that this is not so good. AsyncTasks are designed to be useful helpers when designing occasional asynchronous calls that then need to update a UI. As such, in old (< 1.6) versions of Android the maximum thread pool size was 10!
It would be better to go straight to the very robust Java out of which AsyncTask is built. Given you want to do this repeatedly, try a ScheduledExecutorService. I see they've even made a nice example for you.
Or, given that you seem to be getting a score down, best might be to maintain a persistent connection over a protocol like XMPP, for which there are many Java server and clients.
Finally, you might like to look at gcm.
On design issues in general
I see you want to print a score frequently. Once every four seconds in fact. But what's the point is the score hasn't changed? Furthermore, what if you've got a slow internet connection, and eight seconds later the one for four seconds ago hasn't finished? Right now you will set off yet another download request, even though the other one when it comes back will be up to date!
The solution is to decouple the download mechanism and the UI update mechanism. One way to do it is to have your scheduled download on a single threaded executor- not something you can control in an AsyncTask, which when finishes causes the UI to update and show the score.
Wishing you the best of luck!
Code sketch
Don't have environment set up right now, but in a very rough code sketch (check syntax), using a scheduled executor would look like:
In class:
private final ScheduledExecutorService downloadScheduler = Executors.newSingleThreadScheduledExecutor(1);
Then elsewhere, wherever you start doing this
final Runnable scoreHttpRunnable = new Runnable() {
#Override public void run() {
...
//do Http Syncronously here- I guess whatever is in the doInBackground(...) part of that Async task you wrote!
...
final int newScoreResult = ... (do whatever you do here)
...
runOnUiThread(new Runnable() { #Override public void run() { yourView.updateHoweverYouLike(newScoreResult); } })
...
};
downloadScheduler.scheduleAtFixedRate(scoreHttpRunnable, 0, 4, TimeUnit.SECONDS);
Going one of the other two routes is really too much to post in a single answer to a question. That'd be a another SO question if there isn't already one.

Be sure that next call send to asyc class only after once its done for that make a variable(IsLoadRunning) and make it true in on preExecute() and false in onPOstExecute and add a condition if(!IsLoadRunning){new DownloadMainScore().execute();}

As official documentation states
AsyncTasks should ideally be used for short operations (a few seconds at the most.)
Services can serve better in you case. Have a look at the accepted answer here

#Override
protected String doInBackground(String... params) {
Log.d(TAG, "type - " + params[0] + ", url = " + params[1] + ", name = " + params[2]);
downloadFile(params[1], params[2]);
return null;
}
here is download method
URL url = new URI(Url.replace(" ", "%20")).toURL();
URLConnection connection = url.openConnection();
connection.setConnectTimeout(1000);
int fileLength = connection.getContentLength();
mSavePath = CommonUtilities.getFileSavePath(mContext, fileName, fileLength);
Log.d(TAG, "*** saveFilePath - " + mSavePath);
InputStream inputStream = connection.getInputStream();
if (inputStream != null) {
File file = new File(mSavePath);
BufferedOutputStream bufferOutputStream = new BufferedOutputStream(new FileOutputStream(file));
byte byteArray[] = new byte[1024];
int len = 0;
long total = 0;
while ((len = inputStream.read(byteArray)) != -1) {
bufferOutputStream.write(byteArray, 0, len);
total += len;
}
bufferOutputStream.flush();
bufferOutputStream.close();
inputStream.close();
} else {
Log.d(TAG, "*** inputStream is null");
}