For my app (supporting Android 2.2+) I have to check HTML-code of a lot (approx 700) of different web-pages and retrieve a single name from each web-page. I have all the URL's stored in an array.
I now use a single Asynctask and iterate over the array with URLs like this:
(snippet from Asynctask's doinbackground)
publishProgress(urls.size());
int a = 0;
for(String code : urls) {
if(!running) return null;
try {
URL url = new URL(code);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
naam_codes.put(readStream(con.getInputStream(), true).get(0), code);
} catch (Exception e) {
running = false;
}
publishProgress(++a);
and readstream being:
BufferedReader reader = null;
ArrayList<String> html = new ArrayList<String>();
try {
reader = new BufferedReader(new InputStreamReader(in, Charset.forName("ISO-8859-1")));
if (snel){
//reading, matching and stuff
}
else {
//other reading, matching and stuff
}
}
} catch (IOException e) {
//pass
} finally {
if (reader != null) {
try {
reader.close();
} catch (IOException e) {
return null;
}
}
}
return html;
Now my problem is that it has to wait for one download+matching to finish before starting with a new one. It should be possible to speed this up, right? After monitoring for a bit the process doesn't seem to fully use the CPU nor internet-bandwidth(?). Should I instead of iterating inside one Asynctask, iterate on the UI-thread and execute multiple Asynctasks? If so, how?
Multiple AsyncTasks won't take advantage of multiple cores before API 11. After that, you can create one AsyncTask per download/parsing and have them executed parralelly using the executeOnExecutor function with the parameter AsyncTask.THREAD_POOL_EXECUTOR.
From the documentation:
Order of execution
When first introduced, AsyncTasks were executed serially on a single background thread. Starting with DONUT, this was changed to a pool of
threads allowing multiple tasks to operate in parallel. Starting with
HONEYCOMB, tasks are executed on a single thread to avoid common
application errors caused by parallel execution.
If you truly want parallel execution, you can invoke
executeOnExecutor(java.util.concurrent.Executor, Object[]) with
THREAD_POOL_EXECUTOR.
If I were you, I would build my own server (Just a CRON task launching a PHP script somewhere + a MySQL database + a PHP script to serve your data) and I would not let the applications do the processing.
Let your server do the 700 downlaods, parse them, store what you need in a database. And then let your applications access your server script which will pick the required info from your database.
Advantages:
Your server has better bandwidth
It has more processing power
Your apps can request whatever data they need instead of downloading & parsing several hundreds of pages.
Inconvenient:
You may induce a little delay in making new data available (depends on your CRON task's execution period & execution time to update the database)
Related
see the code below, I put network request and local database write operation together under the same method.
ConcurrentAsyncTask.execute(new Runnable() {
#Override
public void run() {
Pair<String, String> rlt = null;
try {
rlt = sc.createNewDir(repoID, parentDir, dirName);
} catch (SeafException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
String newDirID = rlt.first;
String response = rlt.second;
// The response is the dirents of the parentDir after creating
// the new dir. We save it to avoid request it again
mDatabaseHelper.saveDirents(repoID, parentDir, newDirID, response);
}
});
sc.createNewDir() was used to request network request and mDatabaseHelper.saveDirents() was used to write data into database.
I wonder if it breaks the programming rules of AsyncTask.
Waiting for your advice, thanks!
There is nothing wrong with running different background operations in a single AsyncTask, especially when these operations are logically connected - as in your case network operation produces input for database operations, also you always want to store network operation result to database.
I have an app which requests data from server on regular basis (fixed time interval , 20sec) using JSON.
My current implementation includes having a loop inside an async task , with a 20 sec "sleep" condition. This however doesn't seem like a good implmentation. my question is is there a better implmentation. also is there a good way to force stop an async task from its main thread.?
Another option is to change the code , and let the client send one request and for the server to have the server loop the answers back.Which is a better implmentation thank you in advance :)
protected String doInBackground(String... args) {
String RegistrartionID = RegisID();
while (true){
List<NameValuePair> params = new ArrayList<NameValuePair>();
params.add(new BasicNameValuePair("Start_Term", args[0]));
params.add(new BasicNameValuePair("Dest_Term", args[1]));
params.add(new BasicNameValuePair("RegistrartionID",
RegistrartionID));
Log.i("Start_Term", args[0]);
Log.i("Dest_Term", args[1]);
Log.i("RegistrartionID", RegistrartionID);
JSONObject json = jsonParser.makeHttpRequest(
URL_SEND_CHOSEN_TERMINALS, "POST", params);
try {
Thread.sleep(20000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
return null;
}
As you say, it's not a good implementation. Contrary to popular beliefs, AsyncTask is not intended for long/undetermined-in-time processes, it's meant for short tasks so it can exit afterwards and end.
For your purpose, I'd recommend using a Service. Basically because you've the advantage you start it and stop it within the Activity you define it. You may see its documentation here. Also this may help, it's a very good example.
I recommend you use a Service who executes your AsyncTask and for loop it every 20seconds, you can use AlarmManager
I am trying to perform a simple get request using Apache HTTPClient however it seems as if all the code after the HTTPResponse response = client.execute(get); is being skipped. I am not able to access the contents of the response object,they are all null. However when I use debug mode and I explore the object I see all the data. This function is wrapped in an async task so I am wondering the task itself is not waiting on it to be executed or something I am not sure.
Something similar happened here:
Android code after httpclient.execute(httpget) doesn't get run in try (using AsyncTask)
Here is the code.
#Override
public T execute()
{
utils = new GeneralUtils();
if(getURL() == null)
{
throw new NullPointerException("No path specified");
}
HttpClient client = new DefaultHttpClient();
HttpGet get = new HttpGet(getURL());
Log.e(TAG,"client created");
if(getHeaders()!=null)
{
Log.e(TAG,"client created");
for(Map.Entry<String,String> header:getHeaders().entrySet())
{
get.addHeader(header.getKey(),header.getValue());
}
}
try
{
HttpResponse response = client.execute(get);
Log.e(TAG,"executed");
if(response==null)
Log.v(TAG,"its null as heell");
Log.v(TAG,response.getStatusLine().getReasonPhrase());
Log.v(TAG,String.valueOf(response.getStatusLine().getStatusCode()));
Log.e(TAG,getURL());
Log.v(TAG,"everything else is dead");
for(Header header:response.getAllHeaders())
{
Log.v(TAG,header.getName()+" "+header.getValue());
}
if(response.getStatusLine().getStatusCode() == 200)
{
if(response.getEntity().getContent()!=null)
{
try
{
if(utils.isExternalStorageWritable())
{
String path = getContext().getFilesDir().getAbsolutePath()+"/"+getFileCategory()+"/" +getAlarmId()+getFileExtension();
media = new File(path);
/**
* if the directory has not being created this function does the creation.
*/
media.mkdirs();
FileOutputStream fileOutputStream = new FileOutputStream(media);
IOUtils.copy(response.getEntity().getContent(),fileOutputStream);
fileOutputStream.close();
Log.e(TAG,media.getAbsolutePath());
return (T)media;
}
return null;
}
catch (ClientProtocolException e)
{
Log.v(TAG,e.getMessage());
}
catch (IOException e)
{
Log.v(TAG,e.getMessage());
}
}
}
}
catch (IOException e)
{
Log.e(TAG, e.getCause().getMessage());
e.printStackTrace();
}
return null;
}
The code is not throwing any exceptions so I am not sure about what's happening.
All the code after the response object does not work. It just returns null, As in as soon as I try to obtain a value from response like so response.getStatusCode(), it seems as if the code goes dead and just returns null.
Why don't you use a library that will handle all these restful connections?
I would recommend a couple:
https://github.com/darko1002001/android-rest-client (this is mine i have to mention it first :). I have built this library for the projects i build. For your case you would supply a parser which will give you an InputStream which you will just save as a file (as you do it now with IO utils). It handles the Asynchronous part of the whole thing and generally gives you a nice way to organize code.
http://square.github.io/retrofit/ - is another one that i have been playing around with. i think it is pretty well made and should be able to do whatever you want with it.
http://java.dzone.com/articles/android-%E2%80%93-volley-library - Volley is a project that came out straight from Google and it was demoed at the last Google IO conference. It handles all the async operations for you as well and enables you to do all these things. One thing that i am not really sure about is whether or not it will enable you to parse the responses in the background thread.
I would strongly suggest for you to use one of these as they might save you a lot of time.
If you do want to continue with your code then i would suggest to first investigate if some of the "if" blocks you have are skipped, use the debugger or add log messages to see if it enters the blocks. Go step by step and see what goes wrong.
I am doing something similar in my project, check out this file:
https://github.com/darko1002001/android-rest-client/blob/master/android-rest-lib/src/main/java/com/dg/libs/rest/client/BaseRestClient.java
I created an Android project on 2.3.3 and tried it on mobile 2.3.3, everything works OK. It didn't work on mobile 4, so I re-built for Android 4, but I have the same problem.
This is the code:
public void FTP_Download(){
String server = "192.168.1.135";
int port = 21;
String user = "pc1";
String pass = "1551";
FTPClient ftpClient = new FTPClient();
try {
ftpClient.connect(server, port);
ftpClient.login(user, pass);
ftpClient.enterLocalPassiveMode();
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
Toast.makeText(getBaseContext(), "download starting.",Toast.LENGTH_LONG).show();
// APPROACH #1: using retrieveFile(String, OutputStream)
String remoteFile1 = "i.xml";
File downloadFile1 = new File("sdcard/i.xml");
OutputStream outputStream1 = new BufferedOutputStream(new FileOutputStream(downloadFile1));
boolean success = ftpClient.retrieveFile(remoteFile1, outputStream1);
outputStream1.close();
if (success) {
Toast.makeText(getBaseContext(), "File #1 has been downloaded successfully.",Toast.LENGTH_LONG).show();
}
} catch (IOException ex) {
System.out.println("Error: " + ex.getMessage());
ex.printStackTrace();
} finally {
try {
if (ftpClient.isConnected()) {
ftpClient.logout();
ftpClient.disconnect();
}
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
Note:
I use commons-net-3.1 to connect.
In android version 2.3 above you can not start internet connection from main UI thread. Instead you should use AsyncTask. I assumed you are not using AsyncTask. If you are, then post the code and log cat also. Some examples of other operations that ICS and HoneyComb won't allow you to perform on the UI thread are:( from link posted in comment below ) -
Opening a Socket connection (i.e. new Socket()).
HTTP requests (i.e. HTTPClient and HTTPUrlConnection).
Attempting to connect to a remote MySQL database.
Downloading a file (i.e. Downloader.downloadFile()).
You should not use the main UI Thread to start a network connection or read/write data from it as #phazorRise explained it. But I strongly disagree with using an AsyncTask to perform your download. AsyncTask have been designed for short living operations and downloading a file doesn't belong to that category.
The most relevant way to achieve your goal, if your files are big (and I assume it depends on users, so we can say they are big) is to use a service to download the files.
I invite you to have a look at RoboSpice, it will give your app robustness for networking and it's really the most interesting library for network requests on Android.
Here is an inforgraphics to get familiarized with alternatives and understand why using a service is better than any other technology.
When I use "internet conections" programming for andoid 4, I do an Async Task as follows:
You can put this Class code into the same file as principal file to intercatue with global variables or functions.
public class MyAsyncTask extends AsyncTask<String, Void, String> {
#Override
protected String doInBackground(String... urls) {
String url = urls[0]
try {
//Connection request code, in your case ftp
} catch (Exception e) {
//Catch code
}
}
#Override
protected void onPostExecute(String result) {
//Code to de After the connection is done
}
}
Then, in the Activity I call the Asyn Task
String url = "http://...";
new MyAsyncTask().execute(url);
Edit
Here it's explained how to use Async Task, with an example
http://developer.android.com/reference/android/os/AsyncTask.html
I added this code, and all thing OK
StrictMode.ThreadPolicy policy = new StrictMode.ThreadPolicy.Builder().permitAll().build();
StrictMode.setThreadPolicy(policy);
Note: I taked the code from #user1169115 comment in another post.
This isn't the best soluation, but I don't know why asynctask isn't work, so I don't have another choice.
I'm porting some of my Windows Phone 7 apps to Android. When we call services in the WP7 world, the calls are async. We call the service and there is a delegate _completed event that triggers when when the result is returned. Meanwhile we go on about our way.
The java android code pasted below is how I am calling an HTTP service on my cloud server. I developed this code by going through Android tutorials teaching how to call a service in the android world. Apparently, service calls here are synchronus so the instruction starting with InputStream in... doesn't get executed until the result is returned.
Is this how it is supposed to work for Android? If the service does not respond, there is a wait of a couple minutes and then a timeout exception takes place. That's no good. Everything will hang.
What is the reccommended way to call services in android?
Thanks, Gary
public void onCreate(Bundle savedInstanceState)
{
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
HttpURLConnection urlConnection = null;
try
{
URL url = new URL("http://www.deanblakely.com/REST/api/products");
urlConnection = (HttpURLConnection) url.openConnection();
InputStream in = new BufferedInputStream(urlConnection.getInputStream());
String myString = readStream(in);
String otherString = myString;
otherString = otherString + " ";
}
catch (MalformedURLException e)
{
e.printStackTrace();
}
catch (IOException e)
{
e.printStackTrace();
}
finally
{
urlConnection.disconnect();
}
}
Is this how it is supposed to work for Android?
Yes. Do HTTP operations in a background thread, such as the one supplied by an AsyncTask.
If the service does not respond, there is a wait of a couple minutes and then a timeout exception takes place. That's no good. Everything will hang.
You can set a socket timeout to be something shorter than "a couple minutes". For example, you can call setReadTimeout() on your HttpURLConnection to specify a timeout period in milliseconds.