Develop Reference

How To Write Retry Interceptor For Data Stream In OkHttp?

Or thinking the interceptor for this scenario applicable ?
Our app using OkHttp for downloading files (new version of app, daily databases etc.)
Sometimes server fails just while the app streaming bytes (btw the problem is, recvfrom failed: ECONNRESET)
So to fix this case i just wanted to write OkHttp retry interceptor. But seems this is appropriate for operations which aren't streaming.
Is there a solution(like interceptor) to handle this case ?
To make more clear exposition
0%==============================100% (Just started streaming)
0%==============================100% (10% completed)
0%==============================100% (20% completed)
0%==============================100% (ECONNRESET - Connection reset by peer)
At just this point, streaming gets stopped. The thing i'm wishing from OkHttp is recognizing this situation then starting the stream from scratch (not from 20%)
Related code here, pay attention to comments
Call call = client.newCall(new Request.Builder().url(url).get().build());
Response response = call.execute();
// PROBLEM DOES NOT OCCUR THERE
// PROBLEM DOES NOT OCCUR THERE
// PROBLEM DOES NOT OCCUR THERE
if (response.code() == 200 || response.code() == 201) {
InputStream inputStream = null;
try {
long downloaded = 0;
byte[] buff = new byte[1024 * 4];
inputStream = response.body().byteStream();
long target = response.body().contentLength();
while (true) {
// EXCEPTION OCCURS THERE
// EXCEPTION OCCURS THERE
// EXCEPTION OCCURS THERE
int read = inputStream.read(buff);
if (read == -1) {
break;
}
downloaded += read;
}
...
} catch (IOException e) {
// EXCEPTION SAYS
// ECONNRESET - Connection reset by peer
...
}
}

You can write a custom Interceptor like below:
OkHttp has Interceptors. You need a custom Interceptor like one below:
public class CustomResponseInterceptor implements Interceptor {
private final String TAG = getClass().getSimpleName();
#Override
public Response intercept(Object object) throws IOException {
Request request = chain.request();
Response response = chain.proceed(request);
if (response.code() != 200//Your error code here,) {
//Cancel your Request here
return something;
}
Log.d(TAG, "INTERCEPTED:$ " response.toString());
return response;
}
The code shown is extracted from this Medium Article on Interceptors.
You can also look at this library which implements Retry Interceptors, But you can modify it for your use.

When ECONNRESET - Connection reset by peer occurs why don't you cancel your ongoing call in your catch block and start a new network call for the same file
catch (IOException e) {
// EXCEPTION SAYS
// ECONNRESET - Connection reset by peer
...
call.cancel();//something like this
startDownloadingFromScratch();//new network request to start from scratch
}

Try this.
OkHttpClient client = new OkHttpClient();
client.setConnectTimeout(CONNECT_TIMEOUT_MILLIS, TimeUnit.MILLISECONDS);
client.setReadTimeout(READ_TIMEOUT_MILLIS, TimeUnit.MILLISECONDS);
client.interceptors().add(new Interceptor() {
#Override
public Response intercept(Chain chain) throws IOException {
Request request = chain.request();
// try the request
Response response = chain.proceed(request);
int tryCount = 0;
while (!response.isSuccessful() && tryCount < 3) {
Log.d("intercept", "Request is not successful - " + tryCount);
tryCount++;
// retry the request
response = chain.proceed(request);
}
// otherwise just pass the original response on
return response;
}
});
set this client as your retrofit client.
new Retrofit.Builder()
...other codes
.client(client)
...other codes
.build();
Good luck.

using http2 with okhttp in android, why 20 stream is created for 20 request

Using okhttp 3.8.1
I use okhttp http2 to request for 20 same pictures at the same time using multiple thread with same okhttpclient.
I use tcp dump to capture the traffic. Found that there are still 20 tcp stream, with one much larger, and the other 19 smaller.
My question is what are these 19 streams for. Or am I using ok http in the wrong way?
up down count
2510 56320 1
352 3492 19
my code is something like this
OkHttpClient client = builder.build();
for (int i = 0; i < urls.length; i++) {
runTest(urls[i], client);
}
...
private void runTest(...)
new Thread(new Runnable() {
#Override
public void run() {
Request request = new Request.Builder()
.url(url)
.build();
try {
Response response = client.newCall(request).execute();
result = response.body().string();
} catch (IOException e) {
}
}
}
}).start();
}

the answer lies in the discussions here https://github.com/square/okhttp/issues/3442
in brief, when the concurrent requests starts, there's not a usable tcp channel yet, so okhttp create new tcp channel for each request(20). after the first one is establish, with the coalescing mechanism, the other requests will abandon the tcp channel it create and use the first established one.

OkHttp3 cache seems to be unchecked with Retrofit 2

I'm trying to setup an HTTP cache using Retrofit (2.1.0) and OkHttp (3.3.1). I have seen many posts related to this topic, but none of them helped.
I wrote some unit tests to see how the cache works. It works just fine, but once integrated in my app, the magic ends. I will first show you my implementation and then explain some of my investigation.
First, here is my Retrofit instantiation :
OkHttpClient.Builder httpBuilder = new OkHttpClient.Builder();
HttpLoggingInterceptor loggingInterceptor = new HttpLoggingInterceptor();
interceptor.setLevel(HttpLoggingInterceptor.Level.HEADERS);
OkHttpClient client = httpBuilder
.addNetworkInterceptor(INTERCEPTOR_RESPONSE_SET_CACHE)
.addNetworkInterceptor(INTERCEPTOR_REQUEST_ADD_CHECKSUM)
.addInterceptor(loggingInterceptor)
.cache(cacheHttpClient).build();
Retrofit retrofit = new Retrofit.Builder()
.addConverterFactory(GsonConverterFactory.create())
.client(client)
.baseUrl(BASE_URL)
.build();
Here is the interceptor adding a header to set cache control:
private final Interceptor INTERCEPTOR_RESPONSE_SET_CACHE = new Interceptor() {
#Override
public Response intercept(Chain chain) throws IOException {
Response response = chain.proceed(chain.request());
response = response.newBuilder()
.header("Cache-Control", "max-age=600") //+ Integer.toString(3600 * 5)
.build();
return response;
}
};
The last interceptor adds 2 URL parameters:
private static final Interceptor INTERCEPTOR_REQUEST_ADD_CHECKSUM = new Interceptor() {
#Override
public Response intercept(Interceptor.Chain chain) throws IOException {
HttpUrl url = chain.request().url();
url = url.newBuilder().addQueryParameter("rd", "random1").addQueryParameter("chk","check1").build();
Request request = chain.request().newBuilder().url(url).build();
return chain.proceed(request);
}
};
Finally, the single method of my service :
#Headers("Cache-Control: public, max-stale=500")
#GET("/get_data")
Call<DataResponse> getData(#Query("year") int year, #Query("month") int month, #Query("day") int day);
About my investigation, I setup an interceptor logger (app side, not network) to see what is happening. I can see lines such as "Cache-Control: public, max-stale=500" in my logs. This means (at least to me) that the header should give an opportunity to the OkHttp client to check the cache.
The cache itself seems to be correctly initialised. When I create it, I force the initialisation and log all the urls present in the cache. Here is how it is implemented:
File httpCacheDirectory = new File(getCacheDir(), "responses");
httpCacheDirectory.getParentFile().mkdirs();
int cacheSize = 10 * 1024 * 1024; // 10 MiB
Cache cache = new Cache(httpCacheDirectory, cacheSize);
try {
cache.initialize();
Iterator<String> iterator = cache.urls();
Log.i(TAG, "URLs in cacheHttpClient : ");
while (iterator.hasNext()) {
Log.i(TAG, iterator.next());
}
} catch (IOException e) {
e.printStackTrace();
Log.i(TAG, "CACHE NOT INIT");
}
When I launch my app with Wifi available, I get the expected responses. Then I kill my app, disable Wifi and relaunch the app. I expect the cache to serve data at this moment. But it fails and I can only see OkHttp printed lines in logs :
HTTP FAILED: java.net.UnknownHostException: Unable to resolve host
"my-domain.com": No address associated with hostname
Last thing, in RFC 2616, one can read :
max-stale : Indicates that the client is willing to accept a response
that has exceeded its expiration time. If max-stale is assigned a
value, then the client is willing to accept a response that has
exceeded its expiration time by no more than the specified number of
seconds. If no value is assigned to max-stale, then the client is
willing to accept a stale response of any age.
When I don't specify an value, it actually works (I get a response even when the Wifi is down). For now this is the only way I found to make it "work". So maybe I just misunderstand the cache-control directive !?
At this point I'm really confused. I really would like to be able to use OkHttp cache system, but somehow I'm missing something.
Thank you for reading all that text !

Use this method to create cached okkhttpclient
private OkHttpClient createCachedClient(final Context context) {
File httpCacheDirectory = new File(context.getCacheDir(), "cache_file");
Cache cache = new Cache(httpCacheDirectory, 20 * 1024 * 1024);
OkHttpClient okHttpClient = new OkHttpClient();
okHttpClient.setCache(cache);
okHttpClient.interceptors().add(
new Interceptor() {
#Override
public com.squareup.okhttp.Response intercept(Chain chain) throws IOException {
Request originalRequest = chain.request();
String cacheHeaderValue = isOnline(context)
? "public, max-age=2419200"
: "public, only-if-cached, max-stale=2419200" ;
Request request = originalRequest.newBuilder().build();
com.squareup.okhttp.Response response = chain.proceed(request);
return response.newBuilder()
.removeHeader("Pragma")
.removeHeader("Cache-Control")
.header("Cache-Control", cacheHeaderValue)
.build();
}
}
);
okHttpClient.networkInterceptors().add(
new Interceptor() {
#Override
public com.squareup.okhttp.Response intercept(Chain chain) throws IOException {
Request originalRequest = chain.request();
String cacheHeaderValue = isOnline(context)
? "public, max-age=2419200"
: "public, only-if-cached, max-stale=2419200" ;
Request request = originalRequest.newBuilder().build();
com.squareup.okhttp.Response response = chain.proceed(request);
return response.newBuilder()
.removeHeader("Pragma")
.removeHeader("Cache-Control")
.header("Cache-Control", cacheHeaderValue)
.build();
}
}
);
return okHttpClient;
}
private boolean isOnline(Context context) {
ConnectivityManager connectivity = (ConnectivityManager) _context.getSystemService(Context.CONNECTIVITY_SERVICE);
if (connectivity != null) {
NetworkInfo[] info = connectivity.getAllNetworkInfo();
if (info != null)
for (int i = 0; i < info.length; i++)
if (info[i].getState() == NetworkInfo.State.CONNECTED) {
return true;
}
}
return false;
}
Call createCachedClient() method to create OkHttpClient add this client to retrofit
OkHttpClient okHttpClient = createCachedClient(MainActivity.this);
Retrofit retrofit=new Retrofit.Builder()
.client(okHttpClient)
.baseUrl(API)
.addConverterFactory(GsonConverterFactory
.create()).build();
Add this permission to manifest
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>
If internet is available first time it will call the service and cache the request,next time onwards upto 2419200 milliseconds it will use cache to give response.it won't hit server upto 2419200 milliseconds even if device if offline.

How do you encrypt / hide the body of an HTTPS call using Retrofit 2 + OkHttp 3?

I am currently working on a project where I am sending data via an https call to our server api. The base URL for the project supports ssl (Our url api endpoint starts with https://api.....). I am using Retrofit 2 and OkHttp3 and am setting up the client like this:
public static void buildClient(){
//Misc code here.... not showing for security reasons.
OkHttpClient client = RetrofitClient.configureClient(new OkHttpClient());
//I make calls here to update interceptors, timeouts, etc.
Retrofit retrofit = new Retrofit.Builder()
.baseUrl(BASE_URL)
.addConverterFactory(GsonConverterFactory.create(new Gson()))
.client(client)
.build();
}
//Setup the ssl stuff here
public static OkHttpClient configureClient(final OkHttpClient client) {
final TrustManager[] certs = new TrustManager[]{new X509TrustManager() {
#Override
public X509Certificate[] getAcceptedIssuers() {
return null;
}
#Override
public void checkServerTrusted(final X509Certificate[] chain,
final String authType)
throws CertificateException {
}
#Override
public void checkClientTrusted(final X509Certificate[] chain,
final String authType)
throws CertificateException {
}
}};
SSLContext ssl = null;
try {
ssl = SSLContext.getInstance("TLS");
ssl.init(null, certs, new SecureRandom());
} catch (final java.security.GeneralSecurityException ex) {
}
try {
final HostnameVerifier hostnameVerifier = new HostnameVerifier() {
#Override
public boolean verify(final String hostname,
final SSLSession session) {
return true;
}
};
client.setHostnameVerifier(hostnameVerifier);
client.setSslSocketFactory(ssl.getSocketFactory());
} catch (final Exception e) {
}
return client;
}
So after this, we are all set.
Now, here's what I know:
1) I am sending via HTTPS because if I were not, the server would throw an error, which it is not.
2) My code is working just fine in that it is communicating with the server and the app will work.
The problem here is that the actual Body data is not being encrypted. Here are 2 photo examples to show what I mean.
1)
2)
The first image shows proper obfuscation of the actual body data in that the data is being converted to encrypted 'stuff' while the second shows plain text. The second one is me sending a POST call to the server with an object.
My question is, how do I go about replicating this so that my body text is hidden / encrypted like the other?
Notes:
1) I am using obfuscation via Proguard
2) I to have minifyEnabled set to true
3) How I found this out is via a packet sniffer
Anyone have any ideas how to accomplish this? Or can anyone point me in the right direction as to what this is called specifically?
Thanks.
EDIT:
So, it looks like I was not understanding a key component here.
Short answer is, the call is already encrypted and is sending Https.
Long answer is, I have been comparing my data calls to ones like these:
1)
2)
Where I just assumed that These were encrypted, while mine was not. It turns out that the calls I am sending are encrypted just fine, as are these, but this data is zipped / compressed, which makes it unreadable to the eye, which is what made me think that it was what encrypted data looked like from a packet sniffer.

Your question is: Why I use HTTPS but the Packet Capture or Charles can view all of the SSL / HTTPS traffic between the client and the Internet?
Because the Packet Capture(the VPN proxy) or Charles cheated your client as an intermediary:
Your client <--> Packet Capture/Charles <--> Your target server.
So the proxy tool can view all your HTTPS content(In fact they are indeed encrypted).
Solution:
You can refer the OkHttp wiki: https://github.com/square/okhttp/wiki/HTTPS
and set a Certificate pinning for your HTTPS checking. For example:
public class MainActivity extends AppCompatActivity {
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
final TextView textView = (TextView) findViewById(R.id.text);
ConnectionSpec spec = new ConnectionSpec.Builder(ConnectionSpec.MODERN_TLS)
.tlsVersions(TlsVersion.TLS_1_2)
.cipherSuites(
CipherSuite.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
CipherSuite.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
CipherSuite.TLS_DHE_RSA_WITH_AES_128_GCM_SHA256)
.build();
OkHttpClient client = new OkHttpClient.Builder()
.connectionSpecs(Collections.singletonList(spec))
.certificatePinner(new CertificatePinner.Builder()
.add("drakeet.me", "sha256/gGOcYKAwzEaUfun6YdxZvFSQq/x2lF/R8UizDFofveY=")
.build())
.build();
Request request = new Request.Builder()
.url("https://drakeet.me?s=type")
.post(RequestBody.create(MediaType.parse("text"), "xxx...xxx"))
.addHeader("token", "xxx")
.build();
final Handler handler = new Handler();
client.newCall(request).enqueue(new Callback() {
#Override public void onFailure(Call call, IOException e) {
e.printStackTrace();
}
#Override public void onResponse(Call call, final Response response)
throws IOException {
final String t = response.body().string();
handler.post(new Runnable() {
#Override public void run() {
textView.setText(t);
}
});
}
});
}
}
As my above codes, I preset a certificatePinner relate to the true certificatePinner of my target server, so that if I use Packet Capture/Charles now, they will create a false certificatePinner by themselve, and OkHttp will compare the two pinning, if not equal, throw a javax.net.ssl.SSLPeerUnverifiedException: Certificate pinning failure!
And if you close the Packet Capture/Charles, the exception dismiss and send HTTPS content successfully.

You might see, that requests are still made using "http". Use different Retrofit.Builder method baseUrl(HttpUrl):
HttpUrl httpUrl = new HttpUrl.Builder()
.host(BASE_URL)
.scheme("https").build();
Retrofit retrofit = new Retrofit.Builder()
.baseUrl(httpUrl)
.build();

Why Http call using Retrofit2 (and rxjava observer) always waits for 5 secs on Wifi (Android M Nexus 6)

On Nexus 6 Android M (v23), calling HTTP Api using Retrofit waits for around 5 secs, before getting request hit on server.
Both the phone and the server are on the same WiFi. From browser on other workstation in same wifi gets results instantaneously (~40-60ms).
Opening same API from Android chrome on same mobile too takes 5 secs.
What can be the issue for delayed HTTP call?
Code:
Retrofit2/Okhttp service Factory
public class ApiGenerator {
public static String TAG = ApiGenerator.class.getSimpleName();
public static final String API_BASE_URL = BuildConfig.API_ENDPOINT;
public static final Gson gson = new GsonBuilder()
.setFieldNamingPolicy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES)
.create();
private static OkHttpClient.Builder httpClient = new OkHttpClient.Builder()
.connectTimeout(15, TimeUnit.SECONDS)
.writeTimeout(15, TimeUnit.SECONDS)
.readTimeout(15, TimeUnit.SECONDS)
.addInterceptor(new Interceptor() {
#Override
public Response intercept(Chain chain) throws IOException {
Request request = chain.request();
Response response = chain.proceed(request);
return response;
}
}).addInterceptor(new LoggingInterceptor());
private static Retrofit.Builder builder =
new Retrofit.Builder()
.baseUrl(API_BASE_URL)
.addConverterFactory(GsonConverterFactory.create(gson))
.addCallAdapterFactory(RxJavaCallAdapterFactory.create());
.....
Retrofit2 Service
public interface AuthService {
#GET("/api/v1/auth")
public void authenticate(#Query("token") String token);
#POST("/api/v1/signup")
Call<Object> signup();
}
Service Call -
Observable<Response<SignupV1Response>> observable = service.signupRx(signupReq);
observable.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.unsubscribeOn(Schedulers.io())
.subscribe(new Subscriber<Response<SignupV1Response>>() {
#Override
public void onCompleted() {
...
}
#Override
public void onError(Throwable e) {
...
}
#Override
public void onNext(Response<SignupV1Response> response) {
if (response != null) {
final int statusCode = response.code();
if (response.isSuccess()) {
// Do something
} else {
ResponseBody errorBody = response.errorBody();
// Show error
}
}
}
});

At last found that, it has nothing to do with Okhttp and Retrofit. Its android networking issue, which has IPv6 enabled, on Andoid L (v21) and higher.
If the WiFi network does not correctly handle IPv6 (most routers including recent releases), these http client libraries takes few secs before getting response.
Refer issue - https://code.google.com/p/android/issues/detail?id=79576
For a workaround, you can tunnel local development server to Amazon EC2 with ssh or use ngrok. Works fine with this approach.