Develop Reference

android - Volley response headers

I try to use volley for http request. I tried a request on postman and the response headers are below :
Content-Encoding →gzip
Content-Length →153
Content-Type →application/json; charset=utf-8
Date →Fri, 30 Jun 2017 13:36:10 GMT
ETag →W/"23-E742p6pP4kakmVh3lok1ww"
Server →Microsoft-IIS/8.0
Vary→X-HTTP-Method-Override, Accept-Encoding,Accept-Encoding
X-Powered-By →Express, ASP.NET
Volley adds some extra keys and removes Content-Length and Content-Encoding key. I write keys and values on parseNetworkResponse function. Response headers are below :
Content-Type : application/json; charset=utf-8
Date : Fri, 30 Jun 2017 09:05:23 GMT
ETag : W/"23-E742p6pP4kakmVh3lok1ww"
Server : Microsoft-IIS/8.0
Set-Cookie : ARRAffinity=16d81073e15abb17d2faba962adb6504734210ff2b9ff1ddfa770750ac7752e2;Path=/;xxx.net
Vary : X-HTTP-Method-Override, Accept-Encoding,Accept-Encoding
X-Android-Received-Millis : 1498813523832
X-Android-Response-Source : NETWORK 200
X-Android-Selected-Protocol : http/1.1
X-Android-Sent-Millis : 1498813523632
X-Powered-By : Express
Why does it change content? How can I get Content-Length and Content-Encoding?

if you are using the java.net.HttpURLConnection and the getHeaderFieldKey method, then do not forget to check the 0th element from the header according to the API reference:
Some implementations may treat the 0th header field as special, i.e. as the status line returned by the HTTP server. In this case, getHeaderField(0) returns the status line, but getHeaderFieldKey(0) returns null.

Webservice not working in Android Retrofit , but works in Postman and Swift / iOS, Getting 401 Unauthorized

BRIEFING BEFORE 'technical stuff'
Not new to working with Retrofit but came across this strange behaviour which I am having very hard time to understand and fix,
I have two web service, both work fine as expected in Postman and iOS but only one works in Retrofit and not the other,
In my defence I can say I am getting (Unauthorized) response,which means I was able to hit the server and get a result
In API developer's defence he says it works in Postman and other devices so not a service issue
If any Retrofit expert out there tell me what retrofit may be doing behind my back in order to get this error?
TECHNICAL STUFF
Talking about the type of service , It contains Authorization Bearer token as header which expires every 6 hours and contains no params at all (so it should be easy, right ?) and a simple url
http://hashchuna.nn-assets.com/api/locations
Unfortunately the header token cant be shared with valid key, cos it'l be expired before anyone can try it, but here it's anyway Authorization Bearer 3d44626a55dbb024725984e0d37868336fd7e48a
WHAT I'VE TRIED
I am using okhttp intercept to add Authorization Header to request using both addHeader/header method, no spaces in the url cos there r no params
Getting 401 unauthorized error in retrofit?
Java: Android: Retrofit - using Call but, Response{code = 401,message=unauthorized}
https://github.com/square/retrofit/issues/1290
But non of them helped
WARNING
Now the tricky part to keep in mind, the token when expired must give 401 error which is expected, but the problem is even for freshly created token I get 401 , which is my core problem
LOG
D/OkHttp: --> GET http://hashchuna.nn-assets.com/api/locations http/1.1
D/OkHttp: Authorization: Bearer 7c0d53de006b6de931f7d8747b22442354cecef9
D/OkHttp: --> END GET
D/OkHttp: <-- 401 Unauthorized http://hashchuna.nn-assets.com/api/locations (773ms)
D/OkHttp: Date: Mon, 20 Feb 2017 10:44:11 GMT
D/OkHttp: Server: Apache
D/OkHttp: X-Powered-By: PHP/7.0.15
D/OkHttp: Access-Control-Allow-Origin: *
D/OkHttp: Access-Control-Allow-Credentials: true
D/OkHttp: Access-Control-Max-Age: 1000
D/OkHttp: Access-Control-Allow-Headers: X-Requested-With, Content-Type, Origin, Authorization, Accept, Client-Security-Token, Accept-Encoding
D/OkHttp: Access-Control-Allow-Methods: POST, GET, OPTIONS, DELETE, PUT
D/OkHttp: Expires: Thu, 19 Nov 1981 08:52:00 GMT
D/OkHttp: Cache-Control: no-store, no-cache, must-revalidate
D/OkHttp: Pragma: no-cache
D/OkHttp: Set-Cookie: PHPSESSID=u477o8g0q387t92hms4nhc14n1; path=/
D/OkHttp: Vary: Authorization
D/OkHttp: X-Powered-By: PleskLin
D/OkHttp: Keep-Alive: timeout=5
D/OkHttp: Connection: Keep-Alive
D/OkHttp: Transfer-Encoding: chunked
D/OkHttp: Content-Type: application/json;charset=utf-8
D/OkHttp: <-- END HTTP
CODE
Intercept
Request request = chain
.request()
.newBuilder()
//.header("Authorization","Bearer "+ SharedPrefsUtils.getSPinstance().getAccessToken(context))
.addHeader("Authorization","Bearer 1ed6b7c1839e02bbf7a1b4a8dbca84d23127c68e")
//.addHeader("cache-control", "no-cache")
//.cacheControl(CacheControl.FORCE_NETWORK)
.build();
Retrofit Instance
private Api getApiInstance(Context context) {
HttpLoggingInterceptor logInter = new HttpLoggingInterceptor();
logInter.setLevel(HttpLoggingInterceptor.Level.BODY);
OkHttpClient mIntercepter = new OkHttpClient.Builder()
.addInterceptor(new RequestResponseInterseptor(context))
.addInterceptor(logInter)
.build();
Retrofit retrofitInstance = new Retrofit.Builder()
//.addConverterFactory(new NullOnEmptyConverterFactory())
.addConverterFactory(GsonConverterFactory.create())
.baseUrl(BASE_URL)
.client(mIntercepter)
.build();
return retrofitInstance.create(Api.class);
}

SOLUTION (its the COOKIE)
Thanks to some of the tips, the actual reason for Incompatibility of service is , Supposedly POSTMAN and iOS client store and reuse COOKIE all by itself when requests are made without any need for explicit handling, Cookie in Postman can be tested using Postman Intercepter ,but cant be edited because chrome does not allow editing cookie by plugins
However Retrofit/OkHttp unless specified will consider it disabled(for security reason maybe) ,
Cookie is added either inside Interseptor as one of the header addHeader("Cookie","KEY-VALUE")
or
Use cookieJar to add into
OkHttpClient mIntercepter = new OkHttpClient.Builder()
.cookieJar(mCookieJar)
.addInterceptor(new RequestResponseInterseptor(context))
.addInterceptor(logInter)
.build();
based on your need and cookie type

I think you are overriding other headers Retrofit is adding for you, causing your API to not care about your Authorization header. Code below will add a header to your existing headers instead of overriding them.
OkHttpClient mIntercepter = new OkHttpClient.Builder()
...
.addInterceptor(new Interceptor() {
#Override
public Response intercept(Chain chain) throws IOException {
Request request = chain.request().newBuilder().addHeader("Authorization", "Bearer " + "1ed6b7c1839e02bbf7a1b4a8dbca84d23127c68e").build();
return chain.proceed(request);
})
...
.build();
Format of those headers is correct here, key should be Authorization and value should be Bearer 1ed6b7c1839e02bbf7a1b4a8dbca84d23127c68e (in your case).

In my case, unfortunately, none of advices listed in #Ujju 's solution worked (i.e. neither "Cookie" header nor CookieJar applied). The only thing that helped me is just replacing addInterceptor with addNetworkInterceptor, and everything began to work.

401 Unauthorized http://www.stackoverflow.com/api/login?email=test#test.com&password=123456
Date: Fri, 07 Apr 2017 11:23:28 GMT
Server: Apache/2.4.25 (Amazon) PHP/5.6.29
X-Powered-By: PHP/5.6.29
Cache-Control: no-cache, private
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 59
Content-Length: 41
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: application/json
{"msg":"Invalid Credentials"}
I was facing issue like that not able fetch error message.
When server throwing error like 401 or other error we are getting null body from server.But you can get error message from server in errorBody
String response = response.errorBody().string()

I faced with this problem as well.
Requests work fine not only in POSTMAN, but also in CURl.
Having spent a lot of time I found a solution.
Service example for login:
#FormUrlEncoded
#POST("authentication/login")
fun login(
#Field("login") login: String,
#Field("password") password: String
): Single<Void>
Provide okhttp client:
private fun provideOkHttpClient(): OkHttpClient {
val cookieManager = CookieManager()
cookieManager.setCookiePolicy(CookiePolicy.ACCEPT_ALL)
val client = OkHttpClient.Builder()
.cookieJar(JavaNetCookieJar(cookieManager))
.addNetworkInterceptor(provideRequestInterceptor())
.addNetworkInterceptor(provideLoggingInterceptor())
.protocols(Arrays.asList(Protocol.HTTP_1_1))
.build()
return client
}
Make enable JavaNetCookieJar:
implementation "com.squareup.okhttp3:okhttp-urlconnection:$okHttpVersion"
Provide authorization:
private fun provideRequestInterceptor() = Interceptor {
val builder = it.request().newBuilder().url(it.request().url())
val tokenStr = BuildConfig.SONAR_TOKEN
builder.addHeader("Authorization", "Basic "+ getBase64String(tokenStr+":"))
it.proceed(builder.build())
}
private fun getBase64String(value: String): String {
return Base64.encodeToString(value.toByteArray(charset("UTF-8")), Base64.NO_WRAP)
}

Getting incorrect encoding when reading JSON Sinatra

Using Sinatra I've created a web server and now I'm trying to pass some data there. I'm passing it using Retrofit (android) and Gson as a converter.
Here is a logcat log:
--> POST http://192.168.88.147:4567/payload HTTP/1.1
Content-Type: application/json; charset=UTF-8
Content-Length: 245
[{"composition":"","name":"Чебурек с мясом","price":42,"type":"выпечка","weight":"100","mId":158},{"composition":"","name":"Ватрушка с картошкой","price":32,"type":"выпечка","weight":"100","mId":159}]
--> END POST (245-byte body)
<-- 200 OK http://192.168.88.147:4567/payload (164ms)
Content-Type: text/html;charset=utf-8
Content-Length: 448
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Connection: keep-alive
D/OkHttp: Server: thin
D/OkHttp: OkHttp-Sent-Millis: 1472197883936
OkHttp-Received-Millis: 1472197884075
[{"composition"=>"", "name"=>"\u0427\u0435\u0431\u0443\u0440\u0435\u043A \u0441 \u043C\u044F\u0441\u043E\u043C", "price"=>42, "type"=>"\u0432\u044B\u043F\u0435\u0447\u043A\u0430", "weight"=>"100", "mId"=>158}, {"composition"=>"", "name"=>"\u0412\u0430\u0442\u0440\u0443\u0448\u043A\u0430 \u0441 \u043A\u0430\u0440\u0442\u043E\u0448\u043A\u043E\u0439", "price"=>32, "type"=>"\u0432\u044B\u043F\u0435\u0447\u043A\u0430", "weight"=>"100", "mId"=>159}]
<-- END HTTP (448-byte body)
Pay attention that this is cyrillic.
Here is the Ruby serverside code how I'm doing this:
post '/payload' do
push = JSON.parse(request.body.read)
push.inspect.force_encoding("UTF-8")
end
I used some third party online decoder and it said that this unreadable text is UTF16.
Please help me to find out what is going on here.
I'm new in Ruby.
Also consider to rename this question, because maybe I'm misuse some terms.

Windows 7 was the problem. As soon as I switched to Ubuntu(virtual box) I got the encoding correctly withoud any encode\decode manipulations.

Retrofit ETAG and Caching

Is there a proper explanation on how to add caching and ETAG/If-None-Match support to Retrofit+OkHttp?
I'm struggling to add Etag support on 2 projects, and at first I suspected that there might be an issue with HTTP headers, another project has everything set correctly and caching still doesn't work as expected.
Following are my attempts to make it work. Results show that caching seems to be working within the same instance of the application, but as soon as I restart - everything loads long again.
Also, in my logs I didn't see If-None-Match being added to a request, so I assume that server isn't aware of ETag and still recalculates the response completely.
Here are some code samples:
public class RetrofitHttpClient extends UrlConnectionClient
{
private OkUrlFactory generateDefaultOkUrlFactory()
{
OkHttpClient client = new com.squareup.okhttp.OkHttpClient();
try
{
Cache responseCache = new Cache(baseContext.getCacheDir(), SIZE_OF_CACHE);
client.setCache(responseCache);
}
catch (Exception e)
{
Logger.log(this, e, "Unable to set http cache");
}
client.setConnectTimeout(READ_TIMEOUT, TimeUnit.MILLISECONDS);
client.setReadTimeout(CONNECT_TIMEOUT, TimeUnit.MILLISECONDS);
return new OkUrlFactory(client);
}
private final OkUrlFactory factory;
public RetrofitHttpClient()
{
factory = generateDefaultOkUrlFactory();
}
#Override
protected HttpURLConnection openConnection(retrofit.client.Request request) throws IOException
{
return factory.open(new URL(request.getUrl()));
}
}
Rest adapter is then created with FULL log level and a custom tag:
restAdapter = new RestAdapter.Builder()
.setClient(new RetrofitHttpClient())
.setEndpoint(Config.BASE_URL)
.setRequestInterceptor(new SignatureSetter())
.setConverter(new JacksonConverter(JsonHelper.getObjectMapper()))
.setLogLevel(RestAdapter.LogLevel.FULL)
.setLog(new AndroidLog("=NETWORK="))
.build();
I have a long request on the first screen of the app for testing.
When I open the app - it takes 7 seconds to complete the request. If I pause and resume the app - same request takes 250ms, clearly hitting the cache. If I close the app completely and restart - it again takes 7 seconds.
UPDATE:
As was suggested, I have used a custom Retrofit build and attached a LoggingInterceptor. Here's what I'm getting.
Received response for *** in 449,3ms
Date: Wed, 07 Jan 2015 09:02:23 GMT
Server: Apache
X-Powered-By: PHP/5.4.31
Access-Control-Allow-Credentials: true
Pragma:
Cache-Control: public, max-age=3600
X-Frame-Options: SAMEORIGIN
Etag: "hLxLRYztkinJAB453nRV7ncBSuU=-gzip"
Last-Modified: Wed, 24 Dec 2014 13:09:04 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Keep-Alive: timeout=2, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: application/json; charset=UTF-8
OkHttp-Selected-Protocol: http/1.1
OkHttp-Sent-Millis: 1420621288104
OkHttp-Received-Millis: 1420621288554
Sending request **** on Connection{****:80, proxy=DIRECT# hostAddress=**** cipherSuite=none protocol=http/1.1}
Accept: application/json;
Host: ****
Connection: Keep-Alive
Accept-Encoding: gzip
User-Agent: okhttp/2.2.0
Response is equal to described above
As you can see, no If-None-Match header is present in the next request.

I see this question keeps getting attention and as soon as there is no real answer I can pick from - I'm am providing my investigation on the topic and closing the thread for now.
The end result of investigation and some discussions in the retrofit and okhttp threads on GitHub was that there was supposedly an issue in OkHttp that could prevent If-None-Match tag being set for the outgoing requests.
The issue was supposed to be fixed in OkHttp 2.3, and I'm using 'supposed' here because I didn't yet test if it really works. The testing was difficult because I was using Retrofit, and Retrofit itself had to be updated to use the new version of OkHttp and add some new Interceptors support to be able to debug all headers that are set by OkHttp.
Related thread is here: https://github.com/square/okhttp/issues/831
I'm not sure if Retrofit was updated after that. Hopefully it was, so there is a good chance that issue is already fixed and Etag should properly work - just make sure you have latest versions of Retrofit and OkHttp.
I will try to test everything myself once I have time.

Using OkHttp interceptors will help you to diagnose the headers coming in & out of your application. The interceptors doc gives a code example of an interceptor that logs request & response headers on the network. You can use this as-is.
class LoggingInterceptor implements Interceptor {
#Override public Response intercept(Chain chain) throws IOException {
Request request = chain.request();
long t1 = System.nanoTime();
logger.info(String.format("Sending request %s on %s%n%s",
request.url(), chain.connection(), request.headers()));
Response response = chain.proceed(request);
long t2 = System.nanoTime();
logger.info(String.format("Received response for %s in %.1fms%n%s",
response.request().url(), (t2 - t1) / 1e6d, response.headers()));
return response;
}
}
To hook it up to Retrofit, you'll need to get a pre-release snapshot of Retrofit. As of January, 2015, the currently-shipping versions of Retrofit don't participate in OkHttp's interceptors. There will be a release shortly that does, but it's not ready yet.

I was having a similar problem: OkHttp not hitting cache ever, even when the server was sending same ETAG.
My issue was SIZE_OF_CACHE. I was defining a very small size.
Try to increase it (something like 10 * 1024 * 1024 works for me)
Also you can explore /data/data//files/cache to see if there is actually something stored there

Issues with Caching on OkHttpClient 2.0

I have an issue with caching using OkHttpClient 2.0. It seems that the Response is ignoring the Cache-Control header completely. This is how I am setting up the client and the cache.
OkHttpClient client = new OkHttpClient();
cache = new Cache(new File(Session.getInstance().getContext().getCacheDir(),"http"), 10 * 1024 * 1024);
client.setCache(cache);
client.setCookieHandler(CookieHandler.getDefault());
client.setConnectTimeout(CONNECTION_TIMEOUT, TimeUnit.MILLISECONDS);
client.setReadTimeout(SOCKET_TIMEOUT, TimeUnit.MILLISECONDS);
I believe that the cache directory is created correctly. This is what I see in journal in the /cache/http directory of my application.
libcore.io.DiskLruCache
1
201105
2
This is how I am creating the Request.
Request mRequest = new Request.Builder().url(mUrl).get().build();
Getting the response :
Response response = client.newCall(mRequest).execute();
When using curl, the headers as as follows.
< HTTP/1.1 200 OK
< Date: Fri, 27 Jun 2014 19:39:40 GMT
* Server Apache-Coyote/1.1 is not blacklisted
< Server: Apache-Coyote/1.1
< Cache-Control: no-transform, max-age=1800
< Content-Type: application/json
< Transfer-Encoding: chunked
The OKHttp response headers are as follows.
Connection:Keep-Alive
Content-Type:application/json
Date:Fri, 27 Jun 2014 18:58:30 GMT
Keep-Alive:timeout=5, max=100
OkHttp-Received-Millis:1403895511337
OkHttp-Selected-Protocol:http/1.1
OkHttp-Sent-Millis:1403895511140
Server:Apache-Coyote/1.1
Transfer-Encoding:chunked
The responses never get cached and the call client.getCache().getHitCount() always gives 0. Can someone please suggest what changes might be required here to make the cache work? Thanks.

Okay, the problem was all my get and post requests were using the Authorization Bearer xxxx header and http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html section 14.8 states that these requests can't be cached. The solution was to use s-maxage on the server instead of just max age according to this :
When a shared cache (see section 13.7) receives a request
containing an Authorization field, it MUST NOT return the
corresponding response as a reply to any other request, unless one
of the following specific exceptions holds:
If the response includes the "s-maxage" cache-control
directive, the cache MAY use that response in replying to a
subsequent request.

Are you reading the entire response body? OkHttp won't cache unless you consume the entire response.

I realize you solved your specific problem, but the symptom you describe has another cause.
When using the okhttp-urlconnection, caching doesn't kick in by default, unless we do this:
connection.setUseCaches(true)
(It should be on by default, but some library I was using was setting it to off)