Develop Reference

Create a queue to send data to PHP server from Android

There I am getting data from SQLite and send it to the server using Volley.
for now, I am sending all the data at a time.
I just want to know how can I create a queue that first data of one vehicle, gets its response and then send another one.
cursor=helperClass.readAllData();
if (cursor!=null)
{
while (cursor.moveToNext())
{
modelClass=new ModelClass(cursor.getInt(0),cursor.getString(1),
cursor.getString(2),cursor.getString(3),
cursor.getString(4),cursor.getString(5));
modelClasses.add(modelClass);
}
sizeOfArray=modelClasses.size();
for (int i=0; i<sizeOfArray;i++)
{
name = modelClasses.get(i).getName();
model=modelClasses.get(i).getModelName();
number=modelClasses.get(i).getEngineNumber();
image=modelClasses.get(i).getImageBase64();
hdimage=modelClasses.get(i).getHdimageBase64();
uploadData(name, model, number, image, hdimage);
Toast.makeText(UploadDataServiceClass.this, String.valueOf(sizeOfArray), Toast.LENGTH_SHORT).show();
Toast.makeText(UploadDataServiceClass.this, String.valueOf(i), Toast.LENGTH_SHORT).show();
}
}
uploadData(name,model,number,image,hdimage)
RequestQueue requestQueue=Volley.newRequestQueue(UploadDataServiceClass.this);
StringRequest stringRequest=new StringRequest(Request.Method.POST, showURL, new Response.Listener<String>()
{
#Override
public void onResponse(String response)
{
try
{
Log.d(TAG, "onResponse: " + response);
JSONObject jsonObject = new JSONObject(response);
}
catch (JSONException e)
{
e.printStackTrace();
}
}
}, new Response.ErrorListener()
{
#Override
public void onErrorResponse(VolleyError error)
{ }
}
)
{
#Override
protected Map<String, String> getParams()
{
Map<String, String> parameters = new HashMap<String, String>();
parameters.put("name", name);
parameters.put("model", model);
parameters.put("number", number);
parameters.put("image", image);
parameters.put("hdimage", hdimage);
parameters.put("crud_type", "insert");
return parameters;
}
};
requestQueue.add(stringRequest);

You need a Executor Service with singleThreadExecutor to execute your threads one by one
Creates an Executor that uses a single worker thread operating off an unbounded queue. (Note however that if this single thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.) Tasks are guaranteed to execute sequentially, and no more than one task will be active at any given time. Unlike the otherwise equivalent newFixedThreadPool(1) the returned executor is guaranteed not to be reconfigurable to use additional threads.
but what is Executor Service ?
with a Executor Service you can set the maximum running tasks (1 in your case)
here is a simple tutorials about ThreadPool , Executors and Future

Volley Increasing ThreadPool Size Android

I have this code below which makes 300 http requests and each request returns 10000 rows from database. Total size of 10000 is approximately 0.4mb. So 300*0.4 = 120mb.
Questions:
How increasing the ThreadPool size for handing requests in Volley, can affect the perfomance in app? I change it to 12, but the execution time and size of data was the same as with 4. Is there any difference at all?
When in creasing the number of Volley threads, does the resulted data increase as well? If had 1 thread the maximum returned data each time would be 0.4mb. But if we had 4, the maximum would be 1.6mb.
Emulator: 4 Cores MultiThread
ExecutorService service = Executors.newFixedThreadPool(4);
RequestQueue queue;
AtomicInteger counter = new AtomicInteger(0);
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
File cacheDir = new File(this.getCacheDir(), "Volley");
queue = new RequestQueue(new DiskBasedCache(cacheDir), new BasicNetwork(new HurlStack()), 4);
queue.start();
start();
}
public void start(){
String url ="...";
for(int i =0 ; i<300; i++) {
counter.incrementAndGet();
StringRequest stringRequest = new StringRequest(Request.Method.GET, url,
new Response.Listener<String>() {
#Override
public void onResponse(String response) {
method(response);
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
Log.d("VolleyError", error.toString());
}
});
stringRequest.setTag("a");
queue.add(stringRequest);
}
}
public synchronized void decreased(){
if(counter.decrementAndGet()==0)
start();
}
public void method( String response){
Runnable task = new Runnable() {
#Override
public void run() {
List<Customer> customers= new ArrayList<>();
ObjectMapper objectMapper = new ObjectMapper();
TypeFactory typeFactory = objectMapper.getTypeFactory();
try {
customers= objectMapper.readValue(response, new TypeReference<List<Customer>>() {});
//Simulate database insertion delay
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
decreased();
} catch (IOException e1) {
e1.printStackTrace();
}
}
};
logHeap("");
service.execute(task);
}

Regarding Question 1:
Thread pool with size 4 will be better when compared to 12.
Thread pool size should be in conjunction with number of processors available.
Since the number of processors are limited, app should not spawn unnecessary threads as this may lead to performance problem. As android OS has to manage resource between more threads which will lead to increased wait and actual time for each thread.
Ideally assuming your threads do not have locking such that they do not block each other (independent of each other) and you can assume that the work load (processing) is same, then it turns out that, have a pool size of Runtime.getRuntime().availableProcessors() or availableProcessors() + 1 gives the best results.
Please refer link Setting Ideal size of Thread Pool for more info.
Regarding question 2: If I have understood your question correctly, there should be no change on returned data as thread pool size has no effect on network payload, only wait time and actual time will be changed, when thread pool size value is changed.

Retreiving data from MySQL database using Android Volley library is too slow

I tried to retrieve a list of form data stored in database in my Android application. Each row of database contains some personal information of a person in string format and also an image of that person. I have stored the images in database in MEDIUMBLOB format.
Whenever I try to retrieve data from my server using JSONObjectrequest of Volley Library in my Android app, the response is usually very slow. Sometimes it takes more than 1 minute to view only 10 to 12 rows retrieved from the database.
This is my logcat:
D/Volley: [337] BasicNetwork.logSlowRequests: HTTP response for request=<[ ] http://helpclick.ahsanaasim.me/v1/tasksall/johny 0xa3e020c1 NORMAL 1> [lifetime=14740],
I tried to search for a perfect solution but I couldn't find anything satisfactory. Here is my Java code:
mRequestStartTime = System.currentTimeMillis();
JsonObjectRequest jsonObjectRequest = new JsonObjectRequest(Request.Method.GET,
showUrl2, new Response.Listener<JSONObject>() {
#Override
public void onResponse(JSONObject response) {
long totalRequestTime = System.currentTimeMillis() - mRequestStartTime;
System.out.println(totalRequestTime);
System.out.println(response.toString());
try { c1++;
System.out.println(c1);
JSONArray incidents = response.getJSONArray("incidents");
for (int i = 0; i < incidents.length(); i++) {
e=0;
JSONObject incident = incidents.getJSONObject(i);
c1++;
id= incident.getString("id");
age = incident.getString("age");
gender = incident.getString("gender");
location = incident.getString("location");
//det= incident.getString("det");
String image=new String();
image = incident.getString("bigimage");
if(image.length()!=0) {
myBitmap = ConvertToImage(image);
Bitmap bitmap= Bitmap.createScaledBitmap(myBitmap,500,500,true);
imgs.add(bitmap);
ages.add(age);
genders.add(gender);
locations.add(location);
ids.add(id);
}
}
if(c1>0) {
System.out.println(ages.size());
Intent intent1 = new Intent(search_buttons_page.this, All_Incidents.class);
startActivity(intent1);
}
}
catch (JSONException e) {
e.printStackTrace();
}
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
System.out.append(error.getMessage());
}
});
jsonObjectRequest.setRetryPolicy(new DefaultRetryPolicy(
20000,
DefaultRetryPolicy.DEFAULT_MAX_RETRIES,
DefaultRetryPolicy.DEFAULT_BACKOFF_MULT));
requestQueue.add(jsonObjectRequest);
It would be really helpful if someone can tell me the reason of this slow performance and how to solve it by making minimum modifications in my source code.

Don't keep the images in the db. Keep them in files, and keep the URL of the files in the db. Download the images separately as needed via URL. Then you aren't killing your database access times with what should be file IO.

NodeMCU server bad response, when sending GET request from Android app

I've made little server based on NodeMCU. All works good, when I'm conneting from browser, but problem starts, when I'm trying to connect from Android app uisng OkHttp or Volley, I'm receiving exceptions.
java.io.IOException: unexpected end of stream on Connection using OkHttp,
EOFException using Volley.
Problem is very similar for this
EOFException after server responds, but answer didn't found.
ESP server code
srv:listen(80, function(conn)
conn:on("receive", function(conn,payload)
print(payload)
conn:send("<h1> Hello, NodeMCU.</h1>")
end)
conn:on("sent", function(conn) conn:close() end)
end)
Android code
final RequestQueue queue = Volley.newRequestQueue(this);
final String url = "http://10.42.0.17:80";
final StringRequest request = new StringRequest(Request.Method.GET, url,
new Response.Listener<String>() {
#Override
public void onResponse(String response) {
mTemperatureTextView.setText(response.substring(0, 20));
System.out.println(response);
}
},
new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
System.out.println("Error + " + error.toString());
mTemperatureTextView.setText("That didn't work!");
}
}
);
mUpdateButton.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View view) {
queue.add(request);
}
});

What you're sending back is not HTTP. It's nothing but a protocol-agnostic HTML fragment. Furthermore, there's a memory leak lingering.
Try this instead:
srv:listen(80, function(conn)
conn:on("receive", function(sck,payload)
print(payload)
sck:send("HTTP/1.0 200 OK\r\nServer: NodeMCU on ESP8266\r\nContent-Type: text/html\r\n\r\n<h1> Hello, NodeMCU.</h1>")
end)
conn:on("sent", function(sck) sck:close() end)
end)
you need to send back some HTTP headers, HTTP/1.0 200 OK and the newlines are mandatory
each function needs to use it's own copy of the passed socket instance, see how I renamed conn to sck in the two callback functions, see https://stackoverflow.com/a/37379426/131929 for details
For a more complete send example look at net.socket:send() in the docs. That becomes relevant once you start sending more than just a couple of bytes.

Android volley to handle redirect

I recently started to use Volley lib from Google for my network requests. One of my requests get error 301 for redirect, so my question is that can volley handle redirect somehow automatically or do I have to handle it manually in parseNetworkError or use some kind of RetryPolicyhere?
Thanks.

Replace your url like that url.replace("http", "https");
for example:
if your url looking like that : "http://graph.facebook......." than
it should be like : "https://graph.facebook......."
it works for me

I fixed it catching the http status 301 or 302, reading redirect url and setting it to request then throwing expection which triggers retry.
Edit: Here are the main keys in volley lib which i modified:
Added method public void setUrl(final String url) for class Request
In class BasicNetwork is added check for redirection after // Handle cache validation, if (statusCode == HttpStatus.SC_MOVED_PERMANENTLY) || statusCode == HttpStatus.SC_MOVED_TEMPORARILY), there I read the redirect url with responseHeaders.get("location"), call setUrl with request object and throw error
Error get's catched and it calls attemptRetryOnException
You also need to have RetryPolicy set for the Request (see DefaultRetryPolicy for this)

If you dont want to modify the Volley lib you can catch the 301 and manually re-send the request.
In your GsonRequest class implement deliverError and create a new Request object with the new Location url from the header and insert that to the request queue.
Something like this:
#Override
public void deliverError(final VolleyError error) {
Log.d(TAG, "deliverError");
final int status = error.networkResponse.statusCode;
// Handle 30x
if(HttpURLConnection.HTTP_MOVED_PERM == status || status == HttpURLConnection.HTTP_MOVED_TEMP || status == HttpURLConnection.HTTP_SEE_OTHER) {
final String location = error.networkResponse.headers.get("Location");
Log.d(TAG, "Location: " + location);
final GsonRequest<T> request = new GsonRequest<T>(method, location, jsonRequest, this.requestContentType, this.clazz, this.ttl, this.listener, this.errorListener);
// Construct a request clone and change the url to redirect location.
RequestManager.getRequestQueue().add(request);
}
}
This way you can keep updating Volley and not have to worry about things breaking.

Like many others, I was simply confused about why Volley wasn't following redirects automatically. By looking at the source code I found that while Volley will set the redirect URL correctly on its own, it won't actually follow it unless the request's retry policy specifies to "retry" at least once. Inexplicably, the default retry policy sets maxNumRetries to 0. So the fix is to set a retry policy with 1 retry (10s timeout and 1x back-off copied from default):
request.setRetryPolicy(new DefaultRetryPolicy(10000, 1, 1.0f))
For reference, here is the source code:
/**
* Constructs a new retry policy.
* #param initialTimeoutMs The initial timeout for the policy.
* #param maxNumRetries The maximum number of retries.
* #param backoffMultiplier Backoff multiplier for the policy.
*/
public DefaultRetryPolicy(int initialTimeoutMs, int maxNumRetries, float backoffMultiplier) {
mCurrentTimeoutMs = initialTimeoutMs;
mMaxNumRetries = maxNumRetries;
mBackoffMultiplier = backoffMultiplier;
}
Alternatively, you can create a custom implementation of RetryPolicy that only "retries" in the event of a 301 or 302.
Hope this helps someone!

End up doing a merge of what most #niko and #slott answered:
// Request impl class
// ...
#Override
public void deliverError(VolleyError error) {
super.deliverError(error);
Log.e(TAG, error.getMessage(), error);
final int status = error.networkResponse.statusCode;
// Handle 30x
if (status == HttpURLConnection.HTTP_MOVED_PERM ||
status == HttpURLConnection.HTTP_MOVED_TEMP ||
status == HttpURLConnection.HTTP_SEE_OTHER) {
final String location = error.networkResponse.headers.get("Location");
if (BuildConfig.DEBUG) {
Log.d(TAG, "Location: " + location);
}
// TODO: create new request with new location
// TODO: enqueue new request
}
}
#Override
public String getUrl() {
String url = super.getUrl();
if (!url.startsWith("http://") && !url.startsWith("https://")) {
url = "http://" + url; // use http by default
}
return url;
}
It worked well overriding StringRequest methods.
Hope it can help someone.

Volley supports redirection without any patches, no need for a separate fork
Explanation:
Volley internally uses HttpClient which by default follows 301/302 unless specified otherwise
From: http://hc.apache.org/httpcomponents-client-4.2.x/tutorial/html/httpagent.html
ClientPNames.HANDLE_REDIRECTS='http.protocol.handle-redirects': defines whether redirects should be handled automatically. This parameter expects a value of type java.lang.Boolean. If this parameter is not set HttpClient will handle redirects automatically.

ok, im a bit late to the game here, but i've recently been trying to achieve this same aspect, so https://stackoverflow.com/a/17483037/2423312 is the best one, given that you are willing to fork volley and maintain it and the answer here : https://stackoverflow.com/a/27566737/2423312 - I'm not sure how this even worked.This one is spot on though : https://stackoverflow.com/a/28454312/2423312. But its actually adding a new request object to the NetworkDipatcher's queue, so you'll have to notify the caller as well somehow, there is one dirty way where you can do this by not modifying the request object + changing the field "mURL", PLEASE NOTE THAT THIS IS DEPENDENT ON YOUR IMPLEMENTATION OF VOLLEY'S RetryPolicy.java INTERFACE AND HOW YOUR CLASSES EXTENDING Request.java CLASS ARE, here you go : welcome REFLECTION
Class volleyRequestClass = request.getClass().getSuperclass();
Field urlField = volleyRequestClass.getDeclaredField("mUrl");
urlField.setAccessible(true);
urlField.set(request, newRedirectURL);
Personally I'd prefer cloning volley though. Plus looks like volley's example BasicNetwork class was designed to fail at redirects : https://github.com/google/volley/blob/ddfb86659df59e7293df9277da216d73c34aa800/src/test/java/com/android/volley/toolbox/BasicNetworkTest.java#L156 so i guess they arent leaning too much on redirects, feel free to suggest/edit. Always looking for good way..

I am using volley:1.1.1 with https url though the request was having some issue. On digging deeper i found that my request method was getting changed from POST to GET due to redirect (permanent redirect 301). I am using using nginx and in server block i was having a rewrite rule that was causing the issue.
So in short everything seems good with latest version of volley. My utility function here-
public void makePostRequest(String url, JSONObject body, final AjaxCallback ajaxCallback) {
try {
JsonObjectRequest jsonObjectRequest = new JsonObjectRequest(Request.Method.POST,
url, body, new Response.Listener<JSONObject>() {
#Override
public void onResponse(JSONObject response) {
Log.d(LOG, response.toString());
ajaxCallback.onSuccess(response);
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
Log.e(LOG, error.toString());
ajaxCallback.onError(error);
}
});
singleton.getRequestQueue().add(jsonObjectRequest);
} catch(Exception e) {
Log.d(LOG, "Exception makePostRequest");
e.printStackTrace();
}
}
// separate file
public interface AjaxCallback {
void onSuccess(JSONObject response);
void onError(VolleyError error);
}