I have an api that returns an object as the response, and an etag and max age as headers. The response looks like this:
HEADERS:
'x-frame-options': 'SAMEORIGIN',
'x-xss-protection': '1; mode=block',
'x-content-type-options': 'nosniff',
'x-download-options': 'noopen',
'strict-transport-security': 'max-age=15778476000; includeSubDomains',
'access-control-allow-origin': '*',
etag: 'c69148a0489a95058e729bde7fd4bf32bf2077b1cba8d4fcf0c2da6e696fa33e',
'cache-control': 'private,max-age=43200'
BODY:
{
id: 1985,
url: "https://example.com",
...
}
The desired scenario is that an android application makes the request to ask for this data. The Api returns the data, along with an max age of 43200 secs.
If a request is made before 43200 secs pass, the application has the data from the last response cached. The application makes the request nevertheless, the back-end service compiles the response data, uses the request's etag to decide whether the response data has changed. If the data has changed, it returns a 200 http status and the data. Otherwise it returns a 403 status and no data.
The application receives the response. It uses fast networking to handle caching (says my android teammate). If a 200 status code was returned, the data are updated. Otherwise the application keeps the old data.
If a request is made after 43200 secs have passed, the application no longer has the cached response or it's etag. The request is made, the data are considered as 'new' even if nothing has changed in the data, the status code 200 is returned along with max-age header as above.
What actually happens:
For some reason, after the first request is processed and the application receives the data, no request is made until 43200 secs have passed. The android developer says they see that the request is made and 0 bytes are returned, but when I monitor the requests in the server I don't see any made towards this API.
This doesn't make sense, since max-age does not imply that no requests are made. It simply instructs the application to keep the data in the cache for the duration.
Am I missing the idea of how cache, etags and max-age work?
Back-end is built in node js, and uses express for routing.
You've only set the max-age and private directives in the Cache-Control header. The actual behaviour you have described is the correct behaviour since max-age directive has no bearing on forcing the cache to validate responses each time a request is made. For that, you have to add the no-cache directive as well to the Cache-Control header.
The no-cache directives tells the cache to always validate the stored response with the origin server before serving it (i.e., the desired behaviour you have described). Upon revalidation, the stored response will be valid for another 43200 secs (max-age). Without the no-cache directive, the HTTP client is free to make use of cached responses. Which I guess is why your friend said the request was made, but 0 bytes were returned (browsers also show 0 bytes for responses served from the cache). And which is also why you didn't observe any incoming requests to the server.
Have a look at this article from Google for a good overview on HTTP caching:
https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching
If you need in-depth detail on how responses are constructed from caches, have a look at the RFC7234 spec: https://www.rfc-editor.org/rfc/rfc7234#section-4
Related
I have a simple use case where a server returns an Etag for a request and that etag is added as a header (i.e. If-None-Match) to all subsequent url requests. The server can respond with a 200 if there is a change in the response or a 304 otherwise. For the latter, it makes sense to reuse the response from cache. But okhttp always returns null as the cached response.
I did some troubleshooting and the response is written to disk internally by okhttp but the CachingStrategy doesn't return it to the CacheInterceptor. Looking closely into the CachingStrategy class, there is documentation specifically stating that the cache won't be used:
/**
* Returns true if the request contains conditions that save the server from sending a response
* that the client has locally. When a request is enqueued with its own conditions, the built-in
* response cache won't be used.
*/
private fun hasConditions(request: Request): Boolean =
request.header("If-Modified-Since") != null || request.header("If-None-Match") != null
}
Is there a way to workaround this behavior and use the cached response instead?
(I have caching enabled as described here and I am using okhttp version 3.10.0)
Also, strangely I am having to manually add the Etag. Okhttp fails to add the etag to subsequent requests on its own.
Edit - correction, Okhttp correctly adds the etag header. I needed to use a network interceptor vs a regular one to see the etag header being added to the subsequent requests
Somewhat related to these issues:
https://github.com/square/okhttp/issues/1400
https://github.com/square/okhttp/issues/831#issuecomment-57900979
Either you do your own caching, or you let OkHttp’s cache do the caching. When you add your own conditions, OkHttp assumes you want full control of the caching and it does not participate in that.
TLDR: Is there a way to force OkHttp to correctly handle unexpected/unrequested 100 Continue HTTP responses?
I'm using OkHttp 3.8.1 on Android to POST to a poorly-behaved server.
Even though the request does not include an "Expect: 100-continue" header, the web server returns a 100 Continue response. Rather than continuing to send the request body, then getting the actual (200) response, OkHttp stops there and sends back the 100 Continue response in my okhttp.Callback.
I tried explicitly including "Expect: 100-continue" in the request to trigger OkHttp's logic, but the server (possibly due to some bug) claims the header is malformed and rejects the request. I also tried sending "Expect:" (no value), but the server still sends the 100 Continue response and OkHttp stops there.
Other HTTP clients (I've tested 3 so far) can talk to that server just fine. They handle the 100 Continue response correctly even though they didn't see "Expect: 100-continue" in the request header. Is there an option I can set, or an interceptor I can write, to make OkHttp do the same?
Sounds like something we can fix in OkHttp. This is weird and the server is broken, but if other clients handle it OkHttp should too.
Please report a bug there?
I am using retrofit and okhttp in my Android app to download documents, images and music files. The files are hosted on Amazon through a CDN so the URLs change often. My backend server will try to use redirects to decrease the need to have to constantly update my content on my mobile app every time the CDN url changes. The devices should also cache responses in the case that the device is offline. For this reason, I am using 301 redirects, which I don't know is the best idea.
I was reading the description for 307 redirect at http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
10.3.8 307 Temporary Redirect
The requested resource resides temporarily under a different URI.
Since the redirection MAY be altered on occasion, the client SHOULD
continue to use the Request-URI for future requests.
This response is only cacheable if indicated by a Cache-Control or Expires header field.
At http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html,
13.4 - Response Cacheability
A response received with a status code of 200, 203, 206, 300, 301 or 410
MAY be stored by a cache and used in reply to a subsequent request,
subject to the expiration mechanism, unless a cache-control directive
prohibits caching. However, a cache that does not support the Range and
Content-Range headers MUST NOT cache 206 (Partial Content) responses.
A response received with any other status code
(e.g. status codes 302 and 307) MUST NOT be returned in a reply to
a subsequent request unless there are cache-control directives or another
header(s) that explicitly allow it. For example, these include the
following: an Expires header (section 14.21); a "max-age", "s-maxage",
"must- revalidate", "proxy-revalidate", "public" or "private"
cache-control directive (section 14.9).
It seems that the description states that caching is possible. I haven't tried this yet, but I did notice that in okhttp's CacheStrategy.java class, it has the following code:
public static boolean isCacheable(Response response, Request request) {
// Always go to network for uncacheable response codes (RFC 2616, 13.4),
// This implementation doesn't support caching partial content.
int responseCode = response.code();
if (responseCode != HttpURLConnection.HTTP_OK // 200
&& responseCode != HttpURLConnection.HTTP_NOT_AUTHORITATIVE // 203
&& responseCode != HttpURLConnection.HTTP_MULT_CHOICE // 300
&& responseCode != HttpURLConnection.HTTP_MOVED_PERM / 301
&& responseCode != HttpURLConnection.HTTP_GONE) { // 410
return false;
}
And to verify that caching isn't enabled for from 302-308, the unit tests from CacheTest.java states the following
for (int i = 302; i <= 308; ++i) {
assertCached(false, i);
}
So the okhttp code explicitly ignores caching for 307, and also 302. I just wanted clarification for this. Is the spec for 307 wrong? Or is something wrong with caching 307 that it was deliberately excluded? If I were to set my own cache headers for 307, it would seem to still be skipped.
Just another background information. I know Amazon will support caching with proper cache headers. But since I have to redirect through my own backend, if I don't have caching ability, my offline access to my redirect URL will not respond that I have a cached version from CDN locally. Of course, if I didn't redirect from my backend and go straight to Amazon, then caching wouldn't be a problem through Amazon.
Could you provide some advice in this scenario?
I have an established code base which successfully makes GET and POST requests to a server to download/upload data. We're looking to change the domain name and have set up a 301 HTTP redirect to go from the original domain to the new domain. When doing a GET request, it seems to automatically handle the redirect in the background and successfully retrieves the response. When I am sending a body via POST, however, it throws a java.net.ProtocolException: content-length promised xx bytes, but received 0.
Is there any way to ensure that the body is sent in the redirect request as well? Thanks for any help in advance!
i'm sending an http request to the google reader api and getting an unusual response code. following the documentation, i've requested an auth code and included it in the header of every request. after performing the login, and getting an auth code, i tried accessing this url, which is part of the documentation:
http://www.google.com/reader/api/0/stream/items/contents
when i send the request, i get a 411 status code, which is supposed to mean "Length Required". the length, as i've found, is supposed to be the length, in octets, of the message body. there is no message body in this request. there is only a single header, the POST parameter i="item id" and the URL itself. i tried setting the "Content-Length" header to "0" and also to "-1" to no avail.
what's really interesting is that this same code worked fine before google changed their authorization procedure. it's apparent they've changed something else...
so my question is what EXACTLY would cause a 411 response code and how can i prevent it?
This error happens only with POST and PUT request types, as these two (sort of) expect to have a request body that includes the request parameters (plain textual as well as attachments).
However as the documentation suggests, this is largely an obsolete value, and realistically the web services should handle requests without relying on Content-Length.
So it's not the problem of a request sender, but it is (I would say) a bug on the service side.
Nevertheless, setting a Content-Length (mind the proper capitalisation) request header to 0 should be the workaround.