Develop Reference

How to perform "RXJava 2 queue jobs" until value changes?

I'm using RXJava2 in my Android app, and I have a somewhat peculiar scenario.
I want to perform unknown amount of jobs (determined by the user), but I want them to start only after a certain value has changed.
My specific requirement is to use a Socket for server communication,
and the flow is the following:
User requests some data - data is retrieved by sending data to the socket and wait for the response.
The module that communicates with the server should open a Socket connection, and only after the connection established, it may fetch the requested data.
While Socket attempt to connect, the user may request some more data.
After the connection established successfully the module should perform all the requests sent by the user while connection process was in progress.
The module also should publish the results that came for each data sent to the socket.
How can this be accomplished using RXJava2?

You could use an UnicastSubject for the queue part and do some flatMap-ping once the connection is established:
UnicastSubject<String> userRequests = new UnicastSubject.create();
Single.fromCallable(() -> new Socket("server", port))
.subscribeOn(Schedulers.io())
.flatMapObservable(socket -> {
InputStream in = socket.getInputStream();
OutputStream out = socket.getOutputStream();
byte[] responseBuffer = new byte[4096];
return userRequests
.observeOn(Schedulers.io())
.map(request -> {
out.write(request.getBytes());
int n = in.read(responseBuffer);
if (n >= 0) {
return new String(responseBuffer, 0, n);
}
throw new IOException("Socket closed while waiting for response");
})
.doFinally(() -> socket.close());
});
Since you are working on the Socket level, it is your responsibility to work out the proper encoding of the requests to be written and the proper decoding of the response to be read (i.e., how long (in bytes) the response is to a particular request).

I believe you would need FlowableTransformers.valve() for this, from RxJava2Extensions.
It should work something like this
PublishSubject<String> jobs = PublishSubject.create().toSerialized();
BehaviorSubject<Boolean> isConnected = BehaviorSubject.createDefault(false);
CompositeDisposable disposables = new CompositeDisposable();
public void connect() {
disposables.add(socketService.subscribe((success) -> {
isConnected = true;
}));
}
public void addJob(String job) {
jobs.onNext(job);
}
public void executeQueuedTasks() {
disposables.add(jobs
.toFlowable(BackpressureStrategy.BUFFER)
.compose(FlowableTransformers.valve(isConnected))
.subscribeWith(new DisposableObserver<>() {
...
})
);
}
public void destroy() {
disposables.clear();
}
}
But the UnicastSubject sample above is more likely to work, I wrote this off the top of my head.

How to get HTTP request/response timings in android

I need to log these:
DNS time
Connection time
SSL time
Device network Bandwidth
First byte time
Transfer time
No of objects/No of bytes
I am using OKHttp library for network requests.

Take a look at OkHttp's new EventListener: https://github.com/square/okhttp/wiki/Events
It provides a way to hook-up listener to every step of the hcian, so you can get info like so:
class PrintingEventListener extends EventListener {
private long callStartNanos;
private void printEvent(String name) {
long nowNanos = System.nanoTime();
if (name.equals("callStart")) {
callStartNanos = nowNanos;
}
long elapsedNanos = nowNanos - callStartNanos;
System.out.printf("%.3f %s%n", elapsedNanos / 1000000000d, name);
}
#Override public void callStart(Call call) {
printEvent("callStart");
}
#Override public void callEnd(Call call) {
printEvent("callEnd");
}
#Override public void dnsStart(Call call, String domainName) {
printEvent("dnsStart");
}
#Override public void dnsEnd(Call call, String domainName, List<InetAddress> inetAddressList) {
printEvent("dnsEnd");
}
...
}
And you hook it up like this:
Request request = new Request.Builder()
.url("https://publicobject.com/helloworld.txt")
.build();
System.out.println("REQUEST 1 (new connection)");
try (Response response = client.newCall(request).execute()) {
// Consume and discard the response body.
response.body().source().readByteString();
}
Which will output the following:
REQUEST 1 (new connection)
0.000 callStart
0.010 dnsStart
0.017 dnsEnd
0.025 connectStart
0.117 secureConnectStart
0.586 secureConnectEnd
0.586 connectEnd
0.587 connectionAcquired
0.588 requestHeadersStart
0.590 requestHeadersEnd
0.591 responseHeadersStart
0.675 responseHeadersEnd
0.676 responseBodyStart
0.679 responseBodyEnd
0.679 connectionReleased
0.680 callEnd

AS you have mention you are using OKHttp library for HTTP calling, OKHttp library provide facility to print logs of every single API call that you need using logging-interceptor dependency.
You can have more details in below link and follow the steps of below link.
https://www.learn2crack.com/2016/06/retrofit-okhttp-logging-interceptor.html

Add HttpLoggingInterceptor.It logs the total request time. It also will log the Ok-Http-Sent and Ok-Http-Received headers.
public static Retrofit getInstance() {
if (instance == null) {
HttpLoggingInterceptor logging = new HttpLoggingInterceptor();
logging.setLevel(HttpLoggingInterceptor.Level.BODY);
OkHttpClient.Builder httpClient = new OkHttpClient.Builder();
httpClient.addInterceptor(logging); // <-- this is the important line!
instance = new Retrofit.Builder().baseUrl(Constant.BASE_URL)
.addConverterFactory(GsonConverterFactory.create())
.client(httpClient.build())
.build();
}
return instance;
}
And add the following dependency:
compile "com.squareup.okhttp3:logging-interceptor:3.3.1"

Caching network calls using RxJava for some duration

I am making a network using Retorfit + RxJava2 and I want to cache the response for 30 seconds. Any calls made after 30 seconds interval should get the latest results from server. I tried doing this using Replay operator but it still makes a network call every time I call subscribe. I am not an expert in RxJava so maybe my understanding of using Replay for caching like that is wrong.
public Observable<Name> getName() {
return retrofitBuilder.getName()
.subscribeOn(Schedulers.io())
.replay(30, TimeUnit.SECONDS,Schedulers.io())
.autoConnect();
}
and I am calling the above code like this:
service.getName()
.subscribe(new Consumer<Name>()
{
#Override
public void accept(Name name) throws Exception
{
Log.d("getName", "Name: " + name.toString());
}
}
, new Consumer<Throwable>()
{
#Override
public void accept(Throwable throwable) throws Exception
{
Log.d("getName", throwable.getMessage());
}
});
UPDATE: My apology if I didn't explain my question clearly. What I want is caching on a particular request instead of caching it on HttpClient level which applies the caching strategy to all the request being made through it. In the end I would like to define different caching expiration for different request when needed. Not all my request needs caching for small duration. I was wondering if I could do just that.
Appreciate your help in this.

The are 2 problem with your approach:
as #drhr mentioned, you are creating a new Observable each time you call service.getName() you're creating a new instance of Observable, you should keep the same replayed instance and give to the caller outside the same instance each time it calls service.getName().
even if you will return the same instance, replay with 30 seconds, will replay the sequence emitted by the source Observable over the last 30 sec, meaning after cache expiration time, you will get nothing as your request happened more than 30 sec ago. it doesn't mean that the Observable will restart automatically after this period.
In order to cache for specific period, you basically need to invalidate the cached response after cache period, and perform new request after this period, that's mean you should control your subscribe, and do it there.
You can achieve it with something like that:
public class CachedRequest<T> {
private final AtomicBoolean expired = new AtomicBoolean(true);
private final Observable<T> source;
private final long cacheExpirationInterval;
private final TimeUnit cacheExpirationUnit;
private Observable<T> current;
public CachedRequest(Observable<T> o, long cacheExpirationInterval,
TimeUnit cacheExpirationUnit) {
source = o;
current = o;
this.cacheExpirationInterval = cacheExpirationInterval;
this.cacheExpirationUnit = cacheExpirationUnit;
}
private Observable<T> getCachedObservable() {
return Observable.defer(() -> {
if (expired.compareAndSet(true, false)) {
current = source.cache();
Observable.timer(cacheExpirationInterval, cacheExpirationUnit)
.subscribe(aLong -> expired.set(true));
}
return current;
});
}
}
with defer you can return the right Observable according to cache expiration status, so every subscribe happened within the cache expiration will get cached Observable (using cache()) - meaning request will be performed only once.
after cache expiration, additional subscribe will trigger new request and will set a new timer to reset the cache expiration.

Try to look at okhttp interceptors.
Add CacheInterceptor:
public class CacheInterceptor implements Interceptor {
#Override
public Response intercept(Chain chain) throws IOException {
Response response = chain.proceed(chain.request());
CacheControl cacheControl = new CacheControl.Builder()
.maxAge(30, TimeUnit.SECONDS)
.build();
return response.newBuilder()
.removeHeader("Pragma")
.removeHeader("Cache-Control")
.header("Cache-Control", cacheControl.toString())
.build();
}
}
And add it and cache to your OkHttp Client like this:
File httpCacheDirectory = new File(context.getCacheDir(), "http-cache");
int cacheSize = 10 * 1024 * 1024; // 10 MiB
Cache cache = new Cache(httpCacheDirectory, cacheSize);
OkHttpClient httpClient = new OkHttpClient.Builder()
.addNetworkInterceptor(new CacheInterceptor())
.cache(cache)
.build();

PendingResult.setResultCallback() always returns the statusCode success

if i connect my google watch with a mobile device successfully, and then disable the bluetooth connection (for test reasons) and make a google api client call to my mobile device, the pending result always returns the status code success, even if its not successfull because there is no more connection
async task for the request
class DataTask extends AsyncTask<Node, Void, Void> {
#Override
protected Void doInBackground(Node... nodes) {
Gson gson = new Gson();
Request requestObject = new Request();
requestObject.setType(Constants.REQUEST_TYPE);
String jsonString = gson.toJson(requestObject);
PutDataMapRequest dataMap = PutDataMapRequest.create(Constants.PATH_REQUEST);
dataMap.setUrgent();
dataMap.getDataMap().putString(Constants.KEY_REQUEST, jsonString);
PutDataRequest request = dataMap.asPutDataRequest();
DataApi.DataItemResult dataItemResult = Wearable.DataApi
.putDataItem(googleApiClient, request).await();
boolean connected = googleApiClient.isConnected();
PendingResult<DataApi.DataItemResult> pendingResult = Wearable.DataApi.putDataItem(googleApiClient, request);
pendingResult.setResultCallback(new ResultCallback<DataApi.DataItemResult>() {
#Override
public void onResult(#NonNull DataApi.DataItemResult dataItemResult) {
com.google.android.gms.common.api.Status status = dataItemResult.getStatus();
DataItem dataItem = dataItemResult.getDataItem();
boolean dataValid = dataItemResult.getDataItem().isDataValid();
boolean canceled = status.isCanceled();
boolean interrupted = status.isInterrupted();
float statusCode = status.getStatusCode();
if(status.isSuccess()){ // expected to be false because there is no bluetooth connection anymore
Log.d(TAG, "Success");
}else{
Log.d(TAG, "Failure");
}
}
});
return null;
}
}
why do i not get a false for status.isSuccess?
the only solution i found is to write following code inside the AsyncTask:
Wearable.NodeApi.getConnectedNodes(googleApiClient).await().getNodes();
if(connectedNodes.size() == 0){
// no connection
}
is it not possible to check if the request was successfully inside the ResultCallback?

I believe that the getStatus() call for DataItemResult is only indicating whether the call was successfully passed off to the Data API, not whether it was successfully relayed to another node. The Data API is asynchronous - it's a "store and forward" architecture - so it's not reasonable to expect it to notify you immediately of successful delivery.
In fact, I don't think that there is a way to determine from the Data API when your DataItem has been delivered at all. Your getConnectedNodes technique is only telling you that the watch is connected, not that the data has been delivered. If you need proof of delivery, you'll probably have to implement that yourself, perhaps using the Message API.
One other note: given you've wrapped your code in an AsyncTask, there's no need to use PendingResult.setResultCallback. You can simply await the result inline: http://developer.android.com/training/wearables/data-layer/events.html#sync-waiting

Android OkHttp, refresh expired token

Scenario: I am using OkHttp / Retrofit to access a web service: multiple HTTP requests are sent out at the same time. At some point the auth token expires, and multiple requests will get a 401 response.
Issue: In my first implementation I use an interceptor (here simplified) and each thread tries to refresh the token. This leads to a mess.
public class SignedRequestInterceptor implements Interceptor {
#Override
public Response intercept(Chain chain) throws IOException {
Request request = chain.request();
// 1. sign this request
request = request.newBuilder()
.header(AUTH_HEADER_KEY, BEARER_HEADER_VALUE + token)
.build();
// 2. proceed with the request
Response response = chain.proceed(request);
// 3. check the response: have we got a 401?
if (response.code() == HttpURLConnection.HTTP_UNAUTHORIZED) {
// ... try to refresh the token
newToken = mAuthService.refreshAccessToken(..);
// sign the request with the new token and proceed
Request newRequest = request.newBuilder()
.removeHeader(AUTH_HEADER_KEY)
.addHeader(AUTH_HEADER_KEY, BEARER_HEADER_VALUE + newToken.getAccessToken())
.build();
// return the outcome of the newly signed request
response = chain.proceed(newRequest);
}
return response;
}
}
Desired solution: All threads should wait for one single token refresh: the first failing request triggers the refresh, and together with the other requests waits for the new token.
What is a good way to proceed about this? Can some built-in features of OkHttp (like the Authenticator) be of help? Thank you for any hint.

I had the same problem and I managed to solve it using a ReentrantLock.
import java.io.IOException;
import java.net.HttpURLConnection;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
import okhttp3.Interceptor;
import okhttp3.Request;
import okhttp3.Response;
import timber.log.Timber;
public class RefreshTokenInterceptor implements Interceptor {
private Lock lock = new ReentrantLock();
#Override
public Response intercept(Interceptor.Chain chain) throws IOException {
Request request = chain.request();
Response response = chain.proceed(request);
if (response.code() == HttpURLConnection.HTTP_UNAUTHORIZED) {
// first thread will acquire the lock and start the refresh token
if (lock.tryLock()) {
Timber.i("refresh token thread holds the lock");
try {
// this sync call will refresh the token and save it for
// later use (e.g. sharedPreferences)
authenticationService.refreshTokenSync();
Request newRequest = recreateRequestWithNewAccessToken(chain);
return chain.proceed(newRequest);
} catch (ServiceException exception) {
// depending on what you need to do you can logout the user at this
// point or throw an exception and handle it in your onFailure callback
return response;
} finally {
Timber.i("refresh token finished. release lock");
lock.unlock();
}
} else {
Timber.i("wait for token to be refreshed");
lock.lock(); // this will block the thread until the thread that is refreshing
// the token will call .unlock() method
lock.unlock();
Timber.i("token refreshed. retry request");
Request newRequest = recreateRequestWithNewAccessToken(chain);
return chain.proceed(newRequest);
}
} else {
return response;
}
}
private Request recreateRequestWithNewAccessToken(Chain chain) {
String freshAccessToken = sharedPreferences.getAccessToken();
Timber.d("[freshAccessToken] %s", freshAccessToken);
return chain.request().newBuilder()
.header("access_token", freshAccessToken)
.build();
}
}
The main advantage of using this solution is that you can write an unit test using mockito and test it. You will have to enable Mockito Incubating feature for mocking final classes (response from okhttp). Read more about here.
The test looks something like this:
#RunWith(MockitoJUnitRunner.class)
public class RefreshTokenInterceptorTest {
private static final String FRESH_ACCESS_TOKEN = "fresh_access_token";
#Mock
AuthenticationService authenticationService;
#Mock
RefreshTokenStorage refreshTokenStorage;
#Mock
Interceptor.Chain chain;
#BeforeClass
public static void setup() {
Timber.plant(new Timber.DebugTree() {
#Override
protected void log(int priority, String tag, String message, Throwable t) {
System.out.println(Thread.currentThread() + " " + message);
}
});
}
#Test
public void refreshTokenInterceptor_works_as_expected() throws IOException, InterruptedException {
Response unauthorizedResponse = createUnauthorizedResponse();
when(chain.proceed((Request) any())).thenReturn(unauthorizedResponse);
when(authenticationService.refreshTokenSync()).thenAnswer(new Answer<Boolean>() {
#Override
public Boolean answer(InvocationOnMock invocation) throws Throwable {
//refresh token takes some time
Thread.sleep(10);
return true;
}
});
when(refreshTokenStorage.getAccessToken()).thenReturn(FRESH_ACCESS_TOKEN);
Request fakeRequest = createFakeRequest();
when(chain.request()).thenReturn(fakeRequest);
final Interceptor interceptor = new RefreshTokenInterceptor(authenticationService, refreshTokenStorage);
Timber.d("5 requests try to refresh token at the same time");
final CountDownLatch countDownLatch5 = new CountDownLatch(5);
for (int i = 0; i < 5; i++) {
new Thread(new Runnable() {
#Override
public void run() {
try {
interceptor.intercept(chain);
countDownLatch5.countDown();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}).start();
}
countDownLatch5.await();
verify(authenticationService, times(1)).refreshTokenSync();
Timber.d("next time another 3 threads try to refresh the token at the same time");
final CountDownLatch countDownLatch3 = new CountDownLatch(3);
for (int i = 0; i < 3; i++) {
new Thread(new Runnable() {
#Override
public void run() {
try {
interceptor.intercept(chain);
countDownLatch3.countDown();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}).start();
}
countDownLatch3.await();
verify(authenticationService, times(2)).refreshTokenSync();
Timber.d("1 thread tries to refresh the token");
interceptor.intercept(chain);
verify(authenticationService, times(3)).refreshTokenSync();
}
private Response createUnauthorizedResponse() throws IOException {
Response response = mock(Response.class);
when(response.code()).thenReturn(401);
return response;
}
private Request createFakeRequest() {
Request request = mock(Request.class);
Request.Builder fakeBuilder = createFakeBuilder();
when(request.newBuilder()).thenReturn(fakeBuilder);
return request;
}
private Request.Builder createFakeBuilder() {
Request.Builder mockBuilder = mock(Request.Builder.class);
when(mockBuilder.header("access_token", FRESH_ACCESS_TOKEN)).thenReturn(mockBuilder);
return mockBuilder;
}
}

You should not use interceptors or implement the retry logic yourself as this leads to a maze of recursive issues.
Instead implement the okhttp's Authenticator which is provided specifically to solve this problem:
okHttpClient.setAuthenticator(...);

Thanks for your answers - they led me to the solution. I ended up using a ConditionVariable lock and an AtomicBoolean. Here's how you can achieve this: read through the comments.
/**
* This class has two tasks:
* 1) sign requests with the auth token, when available
* 2) try to refresh a new token
*/
public class SignedRequestInterceptor implements Interceptor {
// these two static variables serve for the pattern to refresh a token
private final static ConditionVariable LOCK = new ConditionVariable(true);
private static final AtomicBoolean mIsRefreshing = new AtomicBoolean(false);
...
#Override
public Response intercept(#NonNull Chain chain) throws IOException {
Request request = chain.request();
// 1. sign this request
....
// 2. proceed with the request
Response response = chain.proceed(request);
// 3. check the response: have we got a 401?
if (response.code() == HttpURLConnection.HTTP_UNAUTHORIZED) {
if (!TextUtils.isEmpty(token)) {
/*
* Because we send out multiple HTTP requests in parallel, they might all list a 401 at the same time.
* Only one of them should refresh the token, because otherwise we'd refresh the same token multiple times
* and that is bad. Therefore we have these two static objects, a ConditionVariable and a boolean. The
* first thread that gets here closes the ConditionVariable and changes the boolean flag.
*/
if (mIsRefreshing.compareAndSet(false, true)) {
LOCK.close();
// we're the first here. let's refresh this token.
// it looks like our token isn't valid anymore.
mAccountManager.invalidateAuthToken(AuthConsts.ACCOUNT_TYPE, token);
// do we have an access token to refresh?
String refreshToken = mAccountManager.getUserData(account, HorshaAuthenticator.KEY_REFRESH_TOKEN);
if (!TextUtils.isEmpty(refreshToken)) {
.... // refresh token
}
LOCK.open();
mIsRefreshing.set(false);
} else {
// Another thread is refreshing the token for us, let's wait for it.
boolean conditionOpened = LOCK.block(REFRESH_WAIT_TIMEOUT);
// If the next check is false, it means that the timeout expired, that is - the refresh
// stuff has failed. The thread in charge of refreshing the token has taken care of
// redirecting the user to the login activity.
if (conditionOpened) {
// another thread has refreshed this for us! thanks!
....
// sign the request with the new token and proceed
// return the outcome of the newly signed request
response = chain.proceed(newRequest);
}
}
}
}
// check if still unauthorized (i.e. refresh failed)
if (response.code() == HttpURLConnection.HTTP_UNAUTHORIZED) {
... // clean your access token and prompt user for login again.
}
// returning the response to the original request
return response;
}
}

If you wan't your threads to bock while the first one refresh the token you can use a synchronized block.
private final static Object lock = new Object();
private static long lastRefresh;
...
synchronized(lock){ // lock all thread untill token is refreshed
// only the first thread does the w refresh
if(System.currentTimeMillis()-lastRefresh>600000){
token = refreshToken();
lastRefresh=System.currentTimeMillis();
}
}
Here 600000 (10 min) is arbitrary this number should be big enouth to prevent muliple refresh call and smaller than your token expiration time so that you call the refresh when the token expires.

Edited for thread safety
Havent looked at OkHttp or retrofit but how about having a static flag that is set as soon as a token fails and check for that flag before you request a new token?
private static AtomicBoolean requestingToken = new AtomicBoolean(false);
//.....
if (requestingToken.get() == false)
{
requestingToken.set(true);
//.... request a new token
}