I need to implement basic caching of API responses. I've made a little playground project that calls GitHub API and caching was successful (I've used Charles to verify that). However when I transferred this solution to my target project caching didn't work anymore. Could multiple interceptors in the chain be the reason?
Code from playground project (working):
Interceptor (same for target project):
public class CacheControlInterceptor implements Interceptor {
#Override
public Response intercept(Chain chain) throws IOException {
Response response = chain.proceed(chain.request());
return response.newBuilder()
.header("Cache-Control", "only-if-cached")
.build();
}
}
Cache and client declaration:
long SIZE_OF_CACHE = 10 * 1024 * 1024; // 10 MB
final Cache cache = new Cache(new File(getCacheDir(), "retrofit_cache"), SIZE_OF_CACHE);
OkHttpClient.Builder client = new OkHttpClient.Builder().cache(cache);
client.networkInterceptors().add(new CacheControlInterceptor());
Retrofit retrofit = new Retrofit.Builder()
.baseUrl("https://api.github.com/users/")
.addConverterFactory(GsonConverterFactory.create())
.client(client.build())
.build();
Screen from debugging of CacheControlInterceptor:
screen
Code from target project (NOT working):
Cache and client declaration:
private OkHttpClient provideOkHttpClient() {
HttpLoggingInterceptor loggingInterceptor = new HttpLoggingInterceptor();
loggingInterceptor.setLevel(HttpLoggingInterceptor.Level.BODY);
OkHttpClient.Builder okhttpClientBuilder = new OkHttpClient.Builder();
okhttpClientBuilder.connectTimeout(30, TimeUnit.SECONDS);
okhttpClientBuilder.readTimeout(30, TimeUnit.SECONDS);
okhttpClientBuilder.writeTimeout(30, TimeUnit.SECONDS);
okhttpClientBuilder.addInterceptor(loggingInterceptor);
okhttpClientBuilder.addInterceptor(new JwtRenewInterceptor(getUserSession()));
okhttpClientBuilder.addInterceptor(new AutoLoginInterceptor(getUserSession()));
okhttpClientBuilder.addNetworkInterceptor(new CacheControlInterceptor());
long SIZE_OF_CACHE = 10 * 1024 * 1024; // 10 MB
final Cache cache = new Cache(new File(getCacheDir(), "retrofit_cache"), SIZE_OF_CACHE);
okhttpClientBuilder.cache(cache);
return okhttpClientBuilder.build();
}
Screen from debugging of CacheControlInterceptor: screen
If you want apply some headers to all requests using OkHttp cache you should use Application interceptor, not network interceptor. Otherwise, you are not giving cache mechanism a chance to return cached responses.
It's nicely illustrated on OkHttp wiki
So most probably what is happening in your code is that you let Cache to store responses but you never use them since requests going to Cache are missing only-if-cached header.
Try
okhttpClientBuilder.addInterceptor(new CacheControlInterceptor());
Actually the mistake was caused by my poor reasoning about http headers. I thought that method addHeader or header will simply add key Cache-Control and then value only-if-cached. However it adds only value! And since in my target project's API there was no header key Cache-Control (unlike in GitHub API) there was no place for value only-if-cached to be stored.
Related
I have a JSON API which is returning 6000+ objects in a JSONArray.
I've implemented Rxjava calling a retrofit GET API call. There isn't any error, but just a sentence stating "Do partial code cache collection, code=30KB, data=19KB".
How do I get the API data? Can I increase the cache for the retrofit response?
I am not sure configuring your cache size will solve your problem but you can increase cache by configuring your Client. If you are using OkHttpClient, you can do it something like this
val clientBuilder = OkHttpClient.Builder()
//pass context here
val cacheDir = File(context.applicationContext.cacheDir, "someChildName")
//Required Cache size
val DISK_CACHE_SIZE = 50 * 1024 * 1024; // 50MB
val cache = Cache(cacheDir, DISK_CACHE_SIZE)
//You can use .cache() to configure your Client
clientBuilder.cache(cache).build()
In your Manifest file you can set it like below, android:largeHeap="true":
<application
android:icon="#drawable/app_logo4"
android:label="#string/app_name"
android:largeHeap="true"
But a phone with limited RAM capacity or old phones will be slow.
If the API is providing pagination then better use pagination, because 6000+ data is huge. Your app's performance will degrade, and you will probably also receive an out of memory (OOM) exception and in low connection scenarios your app always receives a timeout issue.
One solution is you set your timeout in minutes .connectTimeout(15, TimeUnit.MINUTES);, use #Sagar Vekariya code. And don't call your API in the main thread; better use a coroutine.
fun methodName(callback:(ArrayList<Bean>)->Unit) {
GlobalScope.launch(Dispatchers.IO) {
val list = get6KData()
callback.invoke(list)
}
}
You can do one more thing. Don't convert a response in Objects; just take it as a simple JSON text file. That will solve your OOM issue. Later parse the JSON content and insert in the database directly and fire a query in the database and use pagination to list in the app. For example,
fun methodName() {
GlobalScope.launch(Dispatchers.IO) {
val json = get6KData() // Get JSON text here
parsingNinsertDB(json)
// Here you can return the first batch or use another common method for pagination
}
}
fun pageMethod(pageNo:Int,callback:(ArrayList<Bean>)->Unit) {
GlobalScope.launch(Dispatchers.IO) {
val list = db.get6KData(pageNo)
callback.invoke(list)
}
}
HttpLoggingInterceptor logging = new HttpLoggingInterceptor();
logging.setLevel(HttpLoggingInterceptor.Level.BODY);
OkHttpClient.Builder builder = new OkHttpClient().newBuilder();
builder.addInterceptor(logging);
builder.readTimeout(60, TimeUnit.SECONDS);
builder.connectTimeout(5, TimeUnit.MINUTES);
Gson gson = new GsonBuilder()
.setLenient()
.create();
OkHttpClient httpClient = builder.build();
Retrofit retrofit = new Retrofit.Builder()
.baseUrl(BASE_URL)
.addConverterFactory(GsonConverterFactory.create(gson))
.client(httpClient)
.build();
I read dozens of tutorial and Stackoverflow answers to my problem but nothing is working for me! Also, most of them are old so probably OKHTTP changed somehow.
All I want is to enable offline caching for Retrofit.
I am using GET
I tried using only offlineCacheInterceptor as an Interceptor, but I kept getting:
Unable to resolve host "jsonplaceholder.typicode.com": No address associated with hostname
I tried using a combination of offlineCacheInterceptoras an Interceptor + provideCacheInterceptor() as a NetworkInterceptor, but I kept getting:
504 Unsatisfiable Request (only-if-cached) and a null response.body()
I even made sure to add .removeHeader("Pragma") everywhere!
I tried all these Links:
https://newfivefour.com/android-retrofit2-okhttp3-cache-network-request-offline.html (One interceptor, Not working!!)
https://medium.com/mindorks/caching-with-retrofit-store-responses-offline-71439ed32fda (One interceptor, Not working!)
https://caster.io/lessons/retrofit-2-offline-cache (Separate Online + Offline caching, Not working)
https://www.journaldev.com/23297/android-retrofit-okhttp-offline-caching (Not working, 504 Unsatisfiable Request (only-if-cached))
http://mikescamell.com/gotcha-when-offline-caching-with-okhttp3/ (One interceptor, Not working!!)
https://stackoverflow.com/a/48295397/8086424 (Not Working)
Unable to resolve host "jsonplaceholder.typicode.com": No address associated with hostname
Can Retrofit with OKHttp use cache data when offline (TOO confusing!)
Here's my code:
public static Retrofit getRetrofitInstance(Context context) {
if (retrofit == null) {
c = context;
int cacheSize = 10 * 1024 * 1024; // 10 MB
Cache cache = new Cache(context.getCacheDir(), cacheSize);
OkHttpClient okHttpClient = new OkHttpClient.Builder()
.addInterceptor(provideHttpLoggingInterceptor())
.addInterceptor(offlineCacheInterceptor)
.addNetworkInterceptor(provideCacheInterceptor())
.cache(cache)
.build();
//////////////////////////
retrofit = new retrofit2.Retrofit.Builder()
.baseUrl(BASE_URL)
.addConverterFactory(GsonConverterFactory.create())
.client(okHttpClient)
.build();
}
return retrofit;
}
public static Interceptor offlineCacheInterceptor = new Interceptor() {
#Override
public Response intercept(Chain chain) throws IOException {
Request request = chain.request();
Log.e("bbbb", "bbbb");
if (!checkInternetAvailability()) {
Log.e("aaaaa", "aaaaaa");
CacheControl cacheControl = new CacheControl.Builder()
.maxStale(30, TimeUnit.DAYS)
.build();
request = request.newBuilder()
.cacheControl(cacheControl)
.removeHeader("Pragma")
.build();
}
return chain.proceed(request);
}
};
public static Interceptor provideCacheInterceptor() {
return new Interceptor() {
#Override
public Response intercept(Chain chain) throws IOException {
Response response = chain.proceed(chain.request());
// re-write response header to force use of cache
CacheControl cacheControl = new CacheControl.Builder()
.maxAge(2, TimeUnit.MINUTES)
.build();
return response.newBuilder()
.header(CACHE_CONTROL, cacheControl.toString())
.removeHeader("Pragma")
.build();
}
};
}
I am using jsonplaceholder.typicode.com/photos that returns:
content-type: application/json; charset=utf-8
date: Sun, 21 Oct 2018 14:26:41 GMT
set-cookie: __cfduid=d9e935012d2f789245b1e2599a41e47511540132001; expires=Mon, 21-Oct-19 14:26:41 GMT; path=/; domain=.typicode.com; HttpOnly
x-powered-by: Express
vary: Origin, Accept-Encoding
access-control-allow-credentials: true
expires: Sun, 21 Oct 2018 18:26:41 GMT
x-content-type-options: nosniff
etag: W/"105970-HCYFejK2YCxztz8++2rHnutkPOQ"
via: 1.1 vegur
cf-cache-status: REVALIDATED
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 46d466910cab3d77-MXP
Cache-Control: public, max-age=60
June 2021 (Retrofit 2.9.0 or OKHTTP 3.14.9) Complete Solution (Update)
Same approach is still working since: Oct. 2018
Oct. 2018 (Retrofit 2.4 or OKHTTP 3.11) Complete Solution
Ok, so Online & Offline caching using OKHTTP or Retrofit has been causing so many problems for many people on stackoverflow and other forums. There are tons of misleading information and non-working code samples all over the internet.
So, today I will explain how you can implement online & offline caching using Retrofit & OKHTTP with clear steps + How to test and know whether you are getting the data from cache or network.
If you are getting a 504 Unsatisfiable Request (only-if-cached) OR an Unable to resolve host "HOST": No address associated with hostnamethen you can use any of the following solutions.
Before you begin, you must always remember to:
Make sure you are using a GET request and not a POST!
Always make sure you add .removeHeader("Pragma") as shown below (This lets you override the server's caching protocol)
Avoid using the HttpLoggingInterceptor while testing, it can cause some confusion in the beginning. Enable it in the end if you want.
ALWAYS ALWAYS ALWAYS delete your app from the device and reinstall it again upon every change in code, if you want to explore using Interceptors. Otherwise changing code while the old cache data is still on the device will cause you lots of confusion and misleading deductions!
The order of adding Interceptors to OKHTTPClient object matters!
N.B: If you want to depend on your server's caching protocol for online and offline caching, then don't read the 2 solutions. Just read this article. All you need is to create a cache object and attache it to OKHTTPClient object.
Solution 1: (Longer, but you have full control)
Step 1: (Create onlineInterceptor)
static Interceptor onlineInterceptor = new Interceptor() {
#Override
public okhttp3.Response intercept(Chain chain) throws IOException {
okhttp3.Response response = chain.proceed(chain.request());
int maxAge = 60; // read from cache for 60 seconds even if there is internet connection
return response.newBuilder()
.header("Cache-Control", "public, max-age=" + maxAge)
.removeHeader("Pragma")
.build();
}
};
Step 2: (Create Offline Interceptor) (Only if you want cache access when offline)
static Interceptor offlineInterceptor= new Interceptor() {
#Override
public okhttp3.Response intercept(Chain chain) throws IOException {
Request request = chain.request();
if (!isInternetAvailable()) {
int maxStale = 60 * 60 * 24 * 30; // Offline cache available for 30 days
request = request.newBuilder()
.header("Cache-Control", "public, only-if-cached, max-stale=" + maxStale)
.removeHeader("Pragma")
.build();
}
return chain.proceed(request);
}
};
Step 3: (Create a cache object)
int cacheSize = 10 * 1024 * 1024; // 10 MB
Cache cache = new Cache(context.getCacheDir(), cacheSize);
Step 4: (Add interceptors and cache to an OKHTTPClient object)
OkHttpClient okHttpClient = new OkHttpClient.Builder()
// .addInterceptor(provideHttpLoggingInterceptor()) // For HTTP request & Response data logging
.addInterceptor(OFFLINE_INTERCEPTOR)
.addNetworkInterceptor(ONLINE_INTERCEPTOR)
.cache(cache)
.build();
Step 5:(If you are using Retrofit, add the OKHTTPClient object to it)
retrofit = new retrofit2.Retrofit.Builder()
.baseUrl(BASE_URL)
.addConverterFactory(GsonConverterFactory.create())
.client(okHttpClient)
.build();
DONE!
Solution 2: (Just use a library to do all that for you! But deal with the limitations)
Use OkCacheControl library
Step 1 (Create Cache object as shown above)
Step 2 (Create an OKHTTPClient object)
OkHttpClient okHttpClient = OkCacheControl.on(new OkHttpClient.Builder())
.overrideServerCachePolicy(1, MINUTES)
.forceCacheWhenOffline(networkMonitor)
.apply() // return to the OkHttpClient.Builder instance
//.addInterceptor(provideHttpLoggingInterceptor())
.cache(cache)
.build();
Step 3:(Attach the OKHTTPClient object to Retrofit as shown above)
Step 4: (Create a NetworkMonitor Object)
static OkCacheControl.NetworkMonitor networkMonitor=new
OkCacheControl.NetworkMonitor() {
#Override
public boolean isOnline() {
return isInternetAvailable();
}
};
DONE!
Testing:
In order to know whether your device is getting data from the network or from cache, simply add the following code to your onResponse method of Retrofit.
public void onResponse(Call<List<RetroPhoto>> call, Response<List<RetroPhoto>> response) {
if (response.raw().cacheResponse() != null) {
Log.e("Network", "response came from cache");
}
if (response.raw().networkResponse() != null) {
Log.e("Network", "response came from server");
}
}
If the device is using the Network, you will get "response came from server".
If device is using Cache, you will get both of the above responses! For more info about this read this article.
For more info about using OKHTTP interceptors go to this page.
I'm using OKHTTP and Retrofit to handle network calls, I'm not sure how can I clear the cache when user logs out,I'm using MVP/Repository pattern and dagger on my Project
This is the sample project I'm referring
https://github.com/LadwaAditya/DaggerRetrofitOkhttp-Tutorial
OkHttpClient tempClient = new OkHttpClient.Builder()
.readTimeout(60, TimeUnit.SECONDS)// connect timeout
.connectTimeout(60, TimeUnit.SECONDS)// socket timeout
.followRedirects(false)
.cache(provideHttpCache())
.addNetworkInterceptor(new ResponseCacheInterceptor())
.addInterceptor(new AddHeaderAndCookieInterceptor())
.build();
private Cache provideHttpCache() {
Cache cache = new Cache(new File(Application.getAppInstance().getCacheDir(), CACHE_DIR_NAME), CACHE_SIZE);
return cache;
}
OKHTTP has a Cache.evict method but I'm not sure how expose this method
Have you tried referencing your Cache from your OkHttpClient on logout and calling tempClient.cache().delete()? Both delete() and evictAll() appear to be public from the documentation.
I have seen good long discussion on this topic and it is claimed to be fixed in 2.3.0.
Here is the combination I am using
compile 'com.squareup.retrofit2:retrofit:2.0.0-beta4'
compile 'com.squareup.retrofit2:converter-gson:2.0.0-beta4'
compile 'com.squareup.okhttp3:logging-interceptor:3.0.1'
compile 'com.squareup.okhttp3:okhttp:3.2.0'
logs I see against received response, have Etag; but subsequent request I do doesn't have If-None-Match passed in its header.
I tested it by inserting If-None-Match explicitly by my code, caching worked and response was expected one. So there is surely something wrong with version of libraries I am using or something not good about my code.
Here I am setting up okClient.
HttpLoggingInterceptor httpLoggingInterceptor = new HttpLoggingInterceptor();
httpLoggingInterceptor.setLevel(HttpLoggingInterceptor.Level.HEADERS);
okhttp3.OkHttpClient okClient = new okhttp3.OkHttpClient.Builder()
.addInterceptor(new HeaderInterceptor())
.addInterceptor(httpLoggingInterceptor)
.cache(createCacheForOkHTTP())
.connectTimeout(5, TimeUnit.MINUTES)
.readTimeout(5, TimeUnit.MINUTES)
.build();
retrofit = new Retrofit.Builder()
.baseUrl(AppConfig.API_URL)
.addConverterFactory(GsonConverterFactory.create())
.client(okClient)
.build();
My header interceptor contains the logic that is pretty much focused to my API itself. Here it is
private class HeaderInterceptor
implements Interceptor {
private String generateAuthHeader(AuthResponse accessToken) {
if (accessToken == null) {
return "";
}
return String.format("Bearer %s", accessToken.getAccessToken());
}
#Override
public okhttp3.Response intercept(Chain chain)
throws IOException {
Request request = chain.request();
final String authorizationValue = generateAuthHeader(runtime.getPrefAccessToken());
if (!TextUtils.isEmpty(authorizationValue)) {
request = request.newBuilder()
.addHeader(AppConfig.API_KEY_AUTHORIZATION, authorizationValue)
.addHeader(AppConfig.API_KEY_ACCEPT, AppConfig.API_ACCEPT)
.build();
//.addHeader("If-None-Match", "a69385c6d34596e48cdddd3ce475d290")
} else {
request = request.newBuilder()
.addHeader(AppConfig.API_KEY_CONTENT_TYPE, AppConfig.API_CONTENT_TYPE)
.addHeader(AppConfig.API_KEY_ACCEPT, AppConfig.API_ACCEPT)
.build();
}
okhttp3.Response response = chain.proceed(request);
return response;
}
}
And here is the method using which I am setting up cache.
private Cache createCacheForOkHTTP() {
Cache cache = null;
cache = new Cache(App.getInstance().getBaseContext().getCacheDir(), 1024 * 1024 * 10);
return cache;
}
Looking for some quick and effective response as I already have spent reasonable time finding the solution but no luck.
Thanks
Your code seems to be working, I haven't tried it out but i faced the same issue few weeks ago. It turned out that it was because of the log from retrofit did not show the if-none-match header. But when i tried to intercept the request using proxy and redirect the request to my laptop first (i was using mitmproxy app), the if-none-match header appeared.
Anyway, if you look into /okhttp3/internal/http/CacheStrategy.java inside this method private CacheStrategy getCandidate(), you will see that OkHttp3 is actually using the etag & if-none-match header properly.
Hope this clarifies.
I have searched for this but didn't got any appropriate ans. I am developing a social android app in which some feed data need to cache show that when user open this app and there is no network connection won't get blank screen. what is the best and fast access way to cache web data.
You will most likely parse the HTTP data into your own objects. So the question will rather be how to serialize these objects, and save them. These are the most common formats for object serialization:
a plain file, such as in XML or JSON format
a SQLite database
SharedPreferences (as a Set, so this will only work well if the order of the strings is able to be rebuilt later, such as them being in alphabetical order)
(These points look common, eh, #CommonsWare ;)
Facebook, for example, uses an SQLite database, at least for their iOS app.
If you are using retrofit then you can use cache interceptors.
Firstly declare cache memory size
int cacheSize = 10 * 1024 * 1024; // this is 10MB
Then create a cache object
Cache provideCache(MyApplication context) {
Cache cache = null;
try {
cache = new Cache(new File(context.getCacheDir(), "http-cache"), cacheSize);
} catch (Exception e) {
Log.e("Cache", "Error in creating Cache!");
}
return cache;
}
Then create 2 network interceptors for HTTP client ( One for Online, one for No network case)
Interceptor provideOnlineInterceptor(MyApplication context) {
return chain -> {
Response response = chain.proceed(chain.request());
CacheControl cacheControl;
if (Utilities.isNetworkConnected(context)) {
cacheControl = new CacheControl.Builder().maxAge(0, TimeUnit.SECONDS).build();
} else {
cacheControl = new CacheControl.Builder()
.maxStale(7, TimeUnit.DAYS)
.build();
}
return response.newBuilder()
.removeHeader(HEADER_PRAGMA)
.removeHeader(HEADER_CACHE_CONTROL)
.header(HEADER_CACHE_CONTROL, cacheControl.toString())
.build();
};
Interceptor provideOfflineInterceptor(MyApplication context) {
return chain -> {
Request request = chain.request();
if (!Utilities.isNetworkConnected(context)) {
CacheControl cacheControl = new CacheControl.Builder()
.maxStale(7, TimeUnit.DAYS)
.build();
request = request.newBuilder()
.removeHeader(HEADER_PRAGMA)
.removeHeader(HEADER_CACHE_CONTROL)
.cacheControl(cacheControl)
.build();
}
return chain.proceed(request);
};
Create Httpclient
OkHttpClient provideOkHttpClient(MyApplication context) {
OkHttpClient.Builder httpClient = new OkHttpClient.Builder()
.addInterceptor(provideOfflineCacheInterceptor(context))
.addNetworkInterceptor(provideCacheInterceptor(context))
.cache(provideCache(context));
return httpClient.build();
}
Then finally add this httpclient to Retrofit instance
Retrofit provideRetrofit(OkHttpClient client,Gson gson) {
return new Retrofit.Builder().baseUrl(ApiConstants.BASE_URL)
.addCallAdapterFactory(RxJava2CallAdapterFactory.create())
.addConverterFactory(GsonConverterFactory.create(gson))
.client(client)
.build();
}
Hope this will work.
Please go through below link for more detail
Network caching using interceptors