I am working on an Android App for handheld Scan Devices and want to download around 4.500 items from an MySQL database via Retrofit2 into a SQlite Database on the device; hence, when I tried to download all items at once, it slowed down the UI and froze the app up to 5 minutes; I googled a lot on how to solve the problem and couldn´t come up with a fitting solution for now; hence I tried to download the Database with 7 columns for each item - hence, around 31.500 entries in the database - in "Chunks" by iterating in a For-each loop and using .stream() and .limit() in a Background threat, like this:
public static void writeItemsToDatabase(Context mContext, String basic) {
//creating the itemApi interface
ItemApi itemApi = retrofit.create(ItemApi.class);
//making the call object
Call<List<Item>> call = itemApi.checkItems(basic);
call.enqueue(new Callback<List<Item>>() {
#Override
public void onResponse(#NonNull Call<List<Item>> call,
#NonNull Response<List<Item>> response) {
if (response.isSuccess()) {
List<Item> itemList;
itemList = response.body();
int dataSize = response.body().size();
Log.d(TAGGG, String.valueOf(dataSize));
itemList.forEach(List -> Log.d(TAGGG, String.valueOf(List.getEan())));
itemList.forEach(List -> Log.d(TAGGG, String.valueOf(List.getNo())));
class DownloadTask extends AsyncTask<String, Integer, String> {
// Runs in UI before background thread is called
#Override
protected void onPreExecute() {
super.onPreExecute();
// Do something like display a progress bar
}
// This is run in a background thread
#Override
protected String doInBackground(String... params) {
// Do something that takes a long time, for example:
for (int i = 0; i <= 3 ; i++) {
try (DatabaseHandler itemsManager = new DatabaseHandler((XXXXApp)
mContext.getApplicationContext())) {
itemList.stream().limit(1500).forEach(item -> {
itemsManager.addItem(item);
itemsManager.close();
});
}
// Call this to update your progress
publishProgress(i);
}
return "this string is passed to onPostExecute";
}
// This is called from background thread but runs in UI
#Override
protected void onProgressUpdate(Integer... values) {
super.onProgressUpdate(values);
// Do things like update the progress bar
}
// This runs in UI when background thread finishes
#Override
protected void onPostExecute(String result) {
super.onPostExecute(result);
// Do things like hide the progress bar or change a TextView
}
}
new DownloadTask().execute();
}
}
#Override
public void onFailure(Call<List<Item>> call, Throwable t) {}
});
return;
}
however, the result is not satisfying as the Database doesn´t get´s downloaded properly; I changed the values for i to 9 and .limit() to 500 (to achieve the same result, the Download of +4.500 Items) with the same result.
The problem certainly is in this code snippet:
for (int i = 0; i <= 3 ; i++) {
try (DatabaseHandler itemsManager = new DatabaseHandler((XXXApp)
mContext.getApplicationContext()))
{
itemList.stream().limit(1500).forEach(item -> {
itemsManager.addItem(item);
itemsManager.close();
});
}
// Call this to update your progress
publishProgress(i);
}
It is the nearest approach that I´ve found to what I want to achieve after googling a lot; the problem certainly is that it´s a For-Loop that closes the Database each time and reopens it; I am also not sure if the amount of Data is too big for an SQlite database; hence any help or hints on how to solve this properly would be very much appreciated, thanks!
Create once instance of DatabaseHandler(what is it? you can use room with more comfortable API) and reuse it.
Insert many(100-500) items in one transaction.
Or you can create sqlite db file on server side then download it and open as DB in android app.
Related
I have worked with SQLite on android before but this is the first time I'm handling quite a lot of data.
When my activity is opened for the first time it downloads the data from the server. I use google's volley library so I assume I don't need to run it in an async task.
My issue however seems to be with saving the data into sqlite db, rows returned for that single request are between 500-1000. From that point on anytime new data is added on the server I update the app using FCM so no issue with that.
While saving all those records I want to display a circular progress bar and message to the user explaining to them what is happening however I have noticed that my app freezes for 10-15 seconds while the data is being saved. By freezes I mean the progress bar stops moving and activities components and views are not loaded until that process is done.
So my question now is this, what is the right way of doing this so it doesn't interfere with the UI thread allowing me to have the views loaded and a smoother progress bar animation before the operation has completed.
I tried using an async task but noticed no change in performance:
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_bus_schedules);
new SaveDataAsync().executeOnExecutor(AsyncTask.THREAD_POOL_EXECUTOR, (Void[]) null);
}
private class SaveDataAsync extends AsyncTask<Void, Void, Void> {
#Override
protected void onPreExecute() {
// TODO Auto-generated method stub
super.onPreExecute();
}
#Override
protected Void doInBackground(Void... params) {
getSchedules();
return null;
};
}
protected void getSchedules() {
showProgress(true)
mScheduleReq.all(new Response.Listener<JSONArray>() {
#Override
public void onResponse(JSONArray response) {
try {
JSONArray schedules = response;
for (int c = 0; c < schedules.length(); c++) {
JSONObject schedule = schedules.getJSONObject(c);
mSchedulesDat.save(new TravelSchedule(
schedule.getInt("id"),
schedule.getInt("company_id"),
schedule.getInt("route_id"),
schedule.getString("class"),
(float) schedule.getDouble("fare"),
schedule.getString("weekday"),
schedule.getString("reporting_time"),
schedule.getString("departure_time"),
schedule.getInt("by_bus"),
schedule.getInt("by_plane")
));
// Saving Extras
JSONArray extras = schedule.getJSONArray("extras");
for (int k=0; k < extras.length(); k++) {
JSONObject extra = extras.getJSONObject(k);
mExtrasDat.save(new Extra(
extra.getInt("id"),
schedule.getInt("id"),
extra.getString("category"),
extra.getString("name"),
(float) extra.getDouble("price"),
extra.getString("description"),
extra.getString("company")
));
}
// This will prevent fetching all data again from server the next time
mAppDat.save("initialised_travel_schedules", String.valueOf(true));
}
} catch (JSONException e) {
e.printStackTrace();
}
// Hide progress bar
showProgress(false);
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
// Hide progress bar
showProgress(false);
// Parse and display network error response
showNetworkResponse(mContext, error);
}
});
}
You are doing your work on the UI/main thread, which is blocking all other updates to the UI. Any heavy processing that needs to be done should be handled in the background, either with a loader, AsyncTask, or (preferably) RxJava.
If you decide to go the reactive/RxJava route, I would suggest trying out https://github.com/square/sqlbrite. This will allow you to subscribe/unsubscribe to your query and insert operations in the background and post results to the view asynchronously (Activity, Fragment, etc.) as data comes in.
The official docs have a pretty good overview about how threading works on Android if you want to dig deeper.
I am getting a list of installed non-system apps to show to the user and I am using this to do so:
private class getApplications extends AsyncTask<String, Void, String> {
#Override
protected String doInBackground(String... params) {
// perform long running operation operation
for (i = 0; i < list.size(); i++) {
if ((list.get(i).flags & ApplicationInfo.FLAG_SYSTEM) != 1) {
label = (String) pm.getApplicationLabel(list.get(i));
Log.w("Installed Applications", list.get(i).packageName.toString());
}
}
return label;
}
#Override
protected void onPostExecute(String result) {
txtApplications.append(label);
if (i!=list.size()-1) {
txtApplications.append(", ");
}
}
#Override
protected void onPreExecute() {
pm = getPackageManager();
list = pm.getInstalledApplications(PackageManager.GET_META_DATA);
}
};
This works fine as long as it is on the main UI, but it causes the app to lag when loading. I have read these three questions: AsyncTask and getInstalledPackages() fail, PackageManager.getInstalledPackages() returns empty list, Showing ProgressDialog during UI Thread operation in Android and I think understand what they are saying to do, but I am having problems understanding how to make it work. I know the delay is coming from List list = pm.getInstalledApplications(PackageManager.GET_META_DATA); and I know getInstalledApplications(PackageManager.GET_META_DATA); has to be run on the main UI or the app force closes. How do I go about keeping getInstalledApplications(PackageManager.GET_META_DATA); where it needs to be, but populate the list in the background so the app doesn't appear to be frozen? Thank you in advance for any help.
Updated code to show Asynctask. I got the code to run in an Async, but now it only shows one result in the textview instead of the list. I know it has to be some thing simple that I am missing to make it work.
I'd modify your doInBackground() so it looks like this:
#Override
protected String doInBackground(String... params) {
// perform long running operation operation
for (i = 0; i < list.size(); i++) {
if ((list.get(i).flags & ApplicationInfo.FLAG_SYSTEM) != 1) {
//add all the application names to the same String.
label +=", " + (String) pm.getApplicationLabel(list.get(i));
Log.w("Installed Applications", list.get(i).packageName.toString());
}
}
return label;
}
And so, I'd say that your onPostExecute() needs to look just like:
#Override
protected void onPostExecute(String result) {
txtApplications.append(label);
}
}
Since now, label contains all the application names (because of the label += ...);
The goal:
Using Google App Engine server and Android client, I'm trying to put on the Google map at the Android client Users overlays. Every 30 seconds I'm polling the server and getting Vector that contains users and adding it to the map.
Current status:
I'm dong all that using in one new thread, So after running the app I got:
weird behaviors(delayed overlays, multiple overlays) and after that crushed with ConcurrentModificationException.
After reading a bit i figured out that I need to work with AsyncTask.
Correct me if I'm wrong,But I understand that everything done in the Activity at at onCreate is "running" in UIhread so I need to put the "Logic" (All the Network handling) in doInBackground and all the UI Handling like putting overlays on the map in onPostExecute.
My Question are:
1) In the current status I'm doing:
new Thread()
{
#Override
public void run()
{
super.run();
while(true)
{
SystemClock.sleep(30000);
Vector responseFromServer = getUsersVectorFromServer();
putNewOnlineUserOnTheMap();
}
}
}.start();
What is the right way to convert this To AsyncTask?
Do I poll the server still using new thread in the doInBackground or there is right way to do this?
2) Is there a specific list of what counts as UI to put in onPostExecute or any concepts list?
In my case I guess that in need to put putNewOnlineUserOnTheMap() in onPostExecute.
Thanks.
Something similar to the following:
class UpdateTask extends AsyncTask<Void, Vector, Void>{
#Override
protected Void doInBackground(Void... params) {
// this is running in a background thread.
while (!isCancelled()) {
SystemClock.sleep(30000);
Vector responseFromServer = getUsersVectorFromServer();
// send the result back to the UI thread
// onProgressUpdate will be called then
publishProgress(responseFromServer);
}
return null;
}
#Override
protected void onProgressUpdate(Vector... values) {
// this is executed on the UI thread where we can safely touch UI stuff
putNewOnlineUserOnTheMap(values[0]);
}
}
You can't use the result of the task since the task is finished then. But you can use the progress publishing mechanism to get periodic results. If you use it like that and do the modification on the UI thread you should not get ConcurrentModificationException because you do the modifications on the one thread that can safely modify the UI.
One thing to note here: create new instances of your Vector in the background thread and then use it to update the UI. But don't touch the same object afterwards in the backgroundthread. That way you don't need any synchronization since after the background thread sends it away it is only the UI thread that touches it. (and you could use a simple ArrayList instead of a Vector)
AsyncTask uses generics and varargs.The parameters that are passed to the asyntask are . TypeOfVariableArgumentsParameters is passed into the doInBackground(), ProgressParam is used for progress information and ResultParam must be returned from doInBackground() and is passed to onPostExecute() as parameter.
example:--
protected class ParsingTask extends AsyncTask> {
private ProgressDialog loadingDialog = new ProgressDialog(JsonParserActivity.this);
protected void onPreExecute() {
loadingDialog.setMessage("loading app store..");
loadingDialog.show();
}
#Override
protected ArrayList<Items> doInBackground( Context... params ) {
// do ur process here.
return result;
}
if (!this.isCancelled()) {
}
return result;
}
#Override
protected void onProgressUpdate(String... s) {
super.onProgressUpdate(s);
Toast.makeText(getApplicationContext(), s[0], Toast.LENGTH_SHORT).show();
}
#Override
protected void onPostExecute( ArrayList<Items> response ) {
//if u r dealing with list view and adapters set the adapter here at the onPostExecute()
loadingDialog.dismiss();
}
#Override
protected void onCancelled() {
super.onCancelled();
Toast.makeText(getApplicationContext(), "The operation was cancelled", 1).show();
}
}
You can use AsyncTask like below. Hope this will help you..
Class YourClass{
void YourClass(){
NetworkTask nT = new NetworkTasK();
nT.execute();
}
}
protected class NetworkTask extends AsyncTask<Void, String, Boolean>
{
#Override
protected Boolean doInBackground(Void... params)
{
try
{
String response;
while(keepreceiving)
{
response = in.readLine();//Prog Counter stops here until getting i/p.
if(response != null)
yourFunctionForResponse(response);
}
}
catch (Exception ex)
{
}
return null;
}
private void yourFunctionForResponse(String response){
//things to do....
}
}
You may also try runOnUiThread(Runnable action) along with this to implement your work.
Iv'e got an Android app that is using a list activity to display a list of items pulled from the internet. I First use an AsyncTask to load the list and that Async task finishes it calls a different async task to start loading the thumbnail pictures that go along with the list items. The problem I am having is that the user has access to a refresh button that they can press at any time and when it is pressed, the whole list of items is delete and the loading starts over. The Async task that loads the thumbnails could potentially still be running if this happens and may try to add a thumbnail to a non existing list item then. Iv'e tried synchronizing on the list, using a Boolean which after researching I realized would not work. I have also tried using a static atomic boolean to check if refresh has been hit to cancel the thumbnail loader. Any ideas?
public class LoadItems extends AsyncTask<Void, Void, Boolean> {
private Activity activity;
private static boolean loading = false;
public static final AtomicBoolean refreshing = new AtomicBoolean(false);
private static final String TAG = "LoadItems";
private int start;
private List<ListItem> items;
public LoadItems(Activity activity) {
this.activity = activity;
}
#Override
protected void onPreExecute() {
loading = true;
start = ItemViewer.itemList.size();
}
#Override
protected Boolean doInBackground(Void... arg0) {
items = WebFunctions.getMoreItems(activity);
return (items != null);
}
protected void onPostExecute(Boolean success) {
if (success) {
for (ListItem item: items) {
ItemViewer.itemList.add(item);
Log.d(TAG, "added item " + item.getTitle());
}
LoadThumbnails thumbnailLoader = new LoadThumbnails();
thumbnailLoader.execute(start, ItemViewer.itemList.size());
}
loading = false;
}
public void protectedExecute() {
if (!loading)
execute();
}
public void refresh() {
if (!refreshing.getAndSet(true)) {
WebFunctions.reset();
ItemViewer.itemList.removeAllItems();
execute();
}
}
}
public class LoadThumbnails extends AsyncTask<Integer, Void, Drawable> {
private int position;
private int end;
#Override
protected Drawable doInBackground(Integer... params) {
position = params[0];
end = params[1];
Drawable thumbnail = null;
synchronized(ItemViewer.itemList) {
if (LoadItems.refreshing.get())
cancel(true);
String url = ItemViewer.itemList.get(position).getThumbnailUrl();
if (!url.isEmpty())
thumbnail = WebFunctions.loadDrawableFromUrl(ItemViewer.activity, url);
}
return thumbnail;
}
protected void onPostExecute(Drawable d) {
synchronized (ItemViewer.itemList) {
if (LoadItems.refreshing.get())
cancel(true);
if (d != null)
ItemViewer.itemList.setThumbnail(position, d);
position++;
if (position < end) {
LoadThumbnails lt = new LoadThumbnails();
lt.execute(position, end);
}
}
}
}
This is pretty simple to solve. Whenever the user hits the refresh button, make sure you call cancel() on the last async tasks you have created before you create new tasks. For example,
private void onRefreshClick(View v) {
if(mLastLoadItemTask != null) mLastLoadItemTask.cancel(true);
if(mLastLoadThumbnailTask != null) mLastLoadThumbnailTask.cancel(true);
mLastLoadItemTask = new LoadItems(...);
mLastLoadItemTask.execute();
}
Then, in the onPostExecute of each of your async tasks, first check to see if they were cancelled by calling isCancelled(). If they were cancelled, make sure the onPostExecute method does no work by just returning. For example,
protected void onPostExecute(...) {
if(isCancelled()) return;
//Adding items to list
//Or start load thumbnail task
}
As you can see that should prevent any unintentional or stale updates because the onPostExecute methods and your cancel calls will all happen on the main therad. The last thing I would suggest is to alter your loadThumbs task to be able to stop doing work as soon as possibly by checking isCancelled() whenever it makes sense to do so.
The following steps might help:
cache the results, whatever you have previously pulled from the net should be saved and quickly restored back when your application is launched. this way you avoid long delays and empty screens on application startup, which, in turn, stops the user from pressing 'reload'
make a boolean variable reload_in_progress, set it to true when you start pulling data from the net, and set it to false when all thumbnails are ready. 'reload' click handler should ignore clicks when reload_in_progress is true.
show some king of progress bar to the user, so (s)he knows it's already reloading and does not push reload again.
almost forgot, never update data shown to the user "live", this leads to wonderful situations, when the user clicks on item while it's changing and doing something completely different from what (s)he expected. long updates should keep its data to themselves and quickly swap old data for the new one only when everything is ready.
I have listed of products with different category. I have to sort them. Because of the queries, It is taking more time to load. Between two activities, the screen is coming black. I want to run the query in the background. How can I do that and how to use its result in main activity?
private class InsertTask extends AsyncTask {
String cat;
#Override
protected void onPreExecute() {
super.onPreExecute();
}
#Override
protected Boolean doInBackground(String... params) {
Boolean success = false;
try {
category(cat);
success = true;
} catch (Exception e) {
if(e.getMessage()!=null)
e.printStackTrace();
}
return success;
}
#Override
protected void onPostExecute(Boolean success) {
super.onPostExecute(success);
}
private void category(String category) {
try{
Cursor1 = mDbHelper.fetchcategory(category);
}catch(Exception e){
Log.v("Excep", ""+e);
}
}
And when called
InsertTask task = new InsertTask();
task.execute();
I have listed the category in buttons. How can I get the values then?
You should use AsyncTask for that. And some more info.
Its good you have thought of AsyncTask. Firstly, you can declare this class as inner in you class activity (if you haven't previously did) and so you are able to access you view class members.
You can do this also by creating thread and one handler that will be used to update your UI components. Remember that if you use threads you'll need to lock/unlock your database object because of the thread safety(if any other thread is accessing the database for any reason). Read more about thread safety of dbs.
I was doing some searching myself, and I came across this read, its rather long but looks extremely helpful, with lots of code examples. (I bookmarked it for myself).
Threads, Async, and Handlers O MY!
But some form of threading is the ticket.
From Android dev.
(My favorite code snippet)
public void onClick(View v) {
new Thread(new Runnable() {
public void run() {
//Do Work here
}
}).start();
}