Develop Reference

How to fetch a JSON API response with 6000+ objects in a JSONArray

I have a JSON API which is returning 6000+ objects in a JSONArray.
I've implemented Rxjava calling a retrofit GET API call. There isn't any error, but just a sentence stating "Do partial code cache collection, code=30KB, data=19KB".
How do I get the API data? Can I increase the cache for the retrofit response?

I am not sure configuring your cache size will solve your problem but you can increase cache by configuring your Client. If you are using OkHttpClient, you can do it something like this
val clientBuilder = OkHttpClient.Builder()
//pass context here
val cacheDir = File(context.applicationContext.cacheDir, "someChildName")
//Required Cache size
val DISK_CACHE_SIZE = 50 * 1024 * 1024; // 50MB
val cache = Cache(cacheDir, DISK_CACHE_SIZE)
//You can use .cache() to configure your Client
clientBuilder.cache(cache).build()

In your Manifest file you can set it like below, android:largeHeap="true":
<application
android:icon="#drawable/app_logo4"
android:label="#string/app_name"
android:largeHeap="true"
But a phone with limited RAM capacity or old phones will be slow.

If the API is providing pagination then better use pagination, because 6000+ data is huge. Your app's performance will degrade, and you will probably also receive an out of memory (OOM) exception and in low connection scenarios your app always receives a timeout issue.
One solution is you set your timeout in minutes .connectTimeout(15, TimeUnit.MINUTES);, use #Sagar Vekariya code. And don't call your API in the main thread; better use a coroutine.
fun methodName(callback:(ArrayList<Bean>)->Unit) {
GlobalScope.launch(Dispatchers.IO) {
val list = get6KData()
callback.invoke(list)
}
}
You can do one more thing. Don't convert a response in Objects; just take it as a simple JSON text file. That will solve your OOM issue. Later parse the JSON content and insert in the database directly and fire a query in the database and use pagination to list in the app. For example,
fun methodName() {
GlobalScope.launch(Dispatchers.IO) {
val json = get6KData() // Get JSON text here
parsingNinsertDB(json)
// Here you can return the first batch or use another common method for pagination
}
}
fun pageMethod(pageNo:Int,callback:(ArrayList<Bean>)->Unit) {
GlobalScope.launch(Dispatchers.IO) {
val list = db.get6KData(pageNo)
callback.invoke(list)
}
}

HttpLoggingInterceptor logging = new HttpLoggingInterceptor();
logging.setLevel(HttpLoggingInterceptor.Level.BODY);
OkHttpClient.Builder builder = new OkHttpClient().newBuilder();
builder.addInterceptor(logging);
builder.readTimeout(60, TimeUnit.SECONDS);
builder.connectTimeout(5, TimeUnit.MINUTES);
Gson gson = new GsonBuilder()
.setLenient()
.create();
OkHttpClient httpClient = builder.build();
Retrofit retrofit = new Retrofit.Builder()
.baseUrl(BASE_URL)
.addConverterFactory(GsonConverterFactory.create(gson))
.client(httpClient)
.build();

Retrofit 2 caching - chain of interceptors

I need to implement basic caching of API responses. I've made a little playground project that calls GitHub API and caching was successful (I've used Charles to verify that). However when I transferred this solution to my target project caching didn't work anymore. Could multiple interceptors in the chain be the reason?
Code from playground project (working):
Interceptor (same for target project):
public class CacheControlInterceptor implements Interceptor {
#Override
public Response intercept(Chain chain) throws IOException {
Response response = chain.proceed(chain.request());
return response.newBuilder()
.header("Cache-Control", "only-if-cached")
.build();
}
}
Cache and client declaration:
long SIZE_OF_CACHE = 10 * 1024 * 1024; // 10 MB
final Cache cache = new Cache(new File(getCacheDir(), "retrofit_cache"), SIZE_OF_CACHE);
OkHttpClient.Builder client = new OkHttpClient.Builder().cache(cache);
client.networkInterceptors().add(new CacheControlInterceptor());
Retrofit retrofit = new Retrofit.Builder()
.baseUrl("https://api.github.com/users/")
.addConverterFactory(GsonConverterFactory.create())
.client(client.build())
.build();
Screen from debugging of CacheControlInterceptor:
screen
Code from target project (NOT working):
Cache and client declaration:
private OkHttpClient provideOkHttpClient() {
HttpLoggingInterceptor loggingInterceptor = new HttpLoggingInterceptor();
loggingInterceptor.setLevel(HttpLoggingInterceptor.Level.BODY);
OkHttpClient.Builder okhttpClientBuilder = new OkHttpClient.Builder();
okhttpClientBuilder.connectTimeout(30, TimeUnit.SECONDS);
okhttpClientBuilder.readTimeout(30, TimeUnit.SECONDS);
okhttpClientBuilder.writeTimeout(30, TimeUnit.SECONDS);
okhttpClientBuilder.addInterceptor(loggingInterceptor);
okhttpClientBuilder.addInterceptor(new JwtRenewInterceptor(getUserSession()));
okhttpClientBuilder.addInterceptor(new AutoLoginInterceptor(getUserSession()));
okhttpClientBuilder.addNetworkInterceptor(new CacheControlInterceptor());
long SIZE_OF_CACHE = 10 * 1024 * 1024; // 10 MB
final Cache cache = new Cache(new File(getCacheDir(), "retrofit_cache"), SIZE_OF_CACHE);
okhttpClientBuilder.cache(cache);
return okhttpClientBuilder.build();
}
Screen from debugging of CacheControlInterceptor: screen

If you want apply some headers to all requests using OkHttp cache you should use Application interceptor, not network interceptor. Otherwise, you are not giving cache mechanism a chance to return cached responses.
It's nicely illustrated on OkHttp wiki
So most probably what is happening in your code is that you let Cache to store responses but you never use them since requests going to Cache are missing only-if-cached header.
Try
okhttpClientBuilder.addInterceptor(new CacheControlInterceptor());

Actually the mistake was caused by my poor reasoning about http headers. I thought that method addHeader or header will simply add key Cache-Control and then value only-if-cached. However it adds only value! And since in my target project's API there was no header key Cache-Control (unlike in GitHub API) there was no place for value only-if-cached to be stored.

How to set app level cache directory for OkHttp Client

I am using Retrofit - OkHttp for web service call.
While using OkHttpClient.Builder of OkHttp to get OkHttpClient,
there is parameter for cache.
See below line:
OkHttpClient.Builder builder = new OkHttpClient.Builder()
.cache(?);// This last parameter
How to set this cache parameter ?

Using below code we can able to set Application level cache directory for OkHttpClient:
// Install an HTTP cache in the application cache directory.
File cacheDir = new File(app.getCacheDir(), "http");
// [app] is your Application class object.
// using [app.getCacheDir()], get app cache directory.
Cache cache = new Cache(cacheDir, DISK_CACHE_SIZE);
//DISK_CACHE_SIZE is constant for OkHttpClient cache directory size.
//private int DISK_CACHE_SIZE = 50 * 1024 * 1024; // 50MB
OkHttpClient.Builder builder = new OkHttpClient.Builder()
.cache(cache);
OkHttpClient okHttpClient=builder.build();

How to use disk cache to cache ParseFile

I'm using Parse and Picasso to load images onto ParseImageViews. Is there anything I'm missing to cache the parse files? My listview seems to be fetching the file from server every time and using the disk cache that comes with Picasso.
I don't see cache-control: max-age parameter in the http responses of downloads of parse files(from amazon s3 where parse stores them)
I have the following code,
final ParseImageView pic = viewHolder.img;
pic.setScaleType(ImageView.ScaleType.CENTER_INSIDE);
ParseFile f = parseObject.getParseFile("image");
Picasso.with(mContext).load(f.getUrl()).into(pic);
Any help would be appreciated. Thanks.

Use OkHttp client as http transport for Picasso and specify disk and memory cache size:
OkHttpClient okHttp = new OkHttpClient();
Cache cache = new Cache(ctx.getCacheDir(), cacheSize);
okHttp.setCache(cache);
// Use OkHttp as downloader
Downloader downloader = new OkHttpDownloader(okHttp);
mPicasso = new Picasso.Builder(getApplicationContext())
.downloader(downloader)).memoryCache(new LruCache(size)).build();
Setup request interceptor (example) for OkHttp client:
// Add Cache-Control to origin response (force cache)
client.networkInterceptors().add(new Interceptor() {
private com.squareup.okhttp.Request request;
private Response response;
private String requestUrl;
#Override
public Response intercept(Chain c) throws IOException {
request = c.request();
response = c.proceed(request);
if (!request.cacheControl().noStore()
&& !response.cacheControl().noStore()) {
requestUrl = request.urlString();
// Do not cache keys or playlists
response = response
.newBuilder()
.header("Cache-Control","public, max-age=42000").build();
}
return response;
}
});

In Android okhttpclient not caching public cache-control

I am using below code to cache in Android device to collect the response and get from cache till the max age expires. When i use "public" I was able to see the response cached in my application installed folder(/data/data/app_folder).
I tried all below none of them works for "private" but when I change to "public" all my solutions i tried works.
But I need to make it to work for "private". Am I missing something.
Header I will receive "Cache-Control", "private,max-age=120"
Solution 1:
File httpCacheDirectory = new File(context.getCacheDir(), "responses");
HttpResponseCache httpResponseCache = null;
try {
httpResponseCache = new HttpResponseCache(httpCacheDirectory, 10 * 1024 * 1024);
} catch (IOException e) {
Log.e("Retrofit", "Could not create http cache", e);
}
OkHttpClient okHttpClient = new OkHttpClient();
okHttpClient.setResponseCache(httpResponseCache);
Solution 2:
And I am using Retrofit library, an Android client.
Cache cache = new Cache(cacheDirectory, cacheSize);
client = new OkHttpClient();
client.setCache(cache);