[Android]HttpClient post/put StringEntity length limit? - android

I'm seeing a problem with the HttpClient posting/putting a long StringEntity. When the entity is short, there is no problem at all. However, when the length exceeds a value (something around 1400 characters), the http packet can never be sent out (I sniffed the interface using WireShark). Actually, the connection is established, but the data is not transmitted, so the receiver side got a timeout exception.
I'm wondering if these is a lenght limit.
I tried to increase the connection timeout and socket timeout, which only made me wait longer to see the timeout ...
And i also tried to use InputStreamEntity, didn't work either.
[Update]: I tried to used HttpURLConnection directly instead of HttpClient. The same problem still exists. However, I do have some findings. When I forced the packet to be cut into chunks (using HttpURLConnection.setChunkedStreamingMode), WireShare did captured some segment of the packet, with the previous segments missing. I guess this must be a bug in the apache http library.

The issue is resolved. It took me several days to find out that, the problem is with my wireless router. It has some weird settings which truncates large incoming http messages.

It is possible it is timing out on the server side. Also make sure you are using the org.apache.client.httpclient jar. Officially there is no max length for a url. Look here http://www.w3.org/Protocols/rfc2616/rfc2616.html

I think that can help you urllength

Related

HttpURL connection what is the recommended value for setConnectionTimeout and setReadTimeOut?

I Want to know what is the recommended value for methods setConnectionTimeou() and
setReadTimeOut() for HttpURL connection? I know these value depends upon the server and what task server is performing. but still i want to know the recommended values for these method.
It's hard to answer such question without knowing the typical response time. Users are fairly accustomed to wait a few seconds when using mobile devices whilst on mobile networks.
Personally if the timeout is between 10 - 15s I will consider it a normal latency, if it is 20s or more, I will most likely quit the app.
From Default Documentation
Both setConnectTimeout (int timeout) and setReadTimeout (int timeout) From API 1
A SocketTimeoutException is thrown if the connection could not be established in this time. Default is 0 which stands for an infinite timeout.
see this link its give you more idea about this.
https://www.nngroup.com/articles/website-response-times/
You can do it this example:
Your method
How to add parameters to HttpURLConnection using POST
OR
You can follow this other form
https://developer.android.com/reference/java/net/HttpURLConnection.html

Android http json response is coming in pieces

I have a problem - I am trying to make a POST request from my app, but it always drop IOexception error on response(like response code, message etc...).
I made that POST request (what i am trying to make from my app) from my PC and I used WIRESHARK to see the response, but the response comes in multiple PIECES, not in one as usual.
In my app I use httpulrconnection in acync task.
How do I manage to catch all of the response?
I've added a pics of my WIRESHARK file and make it red which contains the response:
Wireshark http post with response in pieces
This is just how anything over a networks is working.
The data is splitted into smaller packets which will get routed through the network. The maximum of the packet size is defined by the different hops in the network.
In most cases those packets are something like 1460bytes of actual payload.
TCP will make this packet-handling transparent, as it is a protocol on top of this raw layer. So normally you shouldn't take care of this.
If you want to see the whole response (like your application will see it), you can view single tcp streams in wireshark:
https://www.wireshark.org/docs/wsug_html_chunked/ChAdvFollowTCPSection.html
Edit
The errors you are seeing don't have anything to do with the packaging.

Android: first http response always takes longer (even without keep-alive)

I am using the class HttpUrlConnection for requesting JSON responses
I realized that no matter if I set or not
System.setProperty("http.keepAlive", "false");
The first response is always going to take longer, while the next responses are very quick, with and without keepAlive. I am not even using SSL.
Notice that, my app doesn't need to perform any authentication with the server, so there isn't any startup call to the webservices. The first request I make to the webservices is actually the very first.
I am also verifying server-side with "netstat", that by setting keepAlive false on the Android client the connections disappear straight away, while without specifying keepAlive false they keep staying as "ESTABLISHED".
How can you explain that subsequent responses are quicker even if the connection doesn't persist?
ANDROID CODE:
line 1) URL url = new URL(stringUrl);
line 2) HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
line 3) InputStream instream = new BufferedInputStream(urlConnection.getInputStream());
Until line 2 everything always gets executed very quickly, either with keepAlive or not. Line 3 in the first request takes around 3 seconds, while in all subsequent always less than 1 second. Each request is about 0.5KB gzipped.
SYSTEM:
I am testing using a Nexus 5, connected via 3G
My webservices are written in Go, running on a CentOS 6.4 linux server
I am using standard tcp v4
UPDATE:
For the moment I have decided to use a trick: when the fragment is resuming, I make a HTTP HEAD request to the server. In this way all subsequent calls in the next 10 seconds are very quick. If the user waits more than 10 seconds then the first one will be slow, and the next ones will be quick again. This is all happening without using KeepAlive.
It's a really big mistery now. It looks like there is some kind of "awake" period which lasts for about 10 seconds. I don't think there is anything strange on my code which can result on that. Also because everything seems to happen during the line 3 I reported above.
SOLVED! thanks to Mark Allison!
Here is a very clear explanation:
http://developer.android.com/training/efficient-downloads/efficient-network-access.html
Also, everything can easily be monitored using Android DDMS's Network Statistics. If you wait some seconds (let's say 20) from last request, you can see that it takes 2 seconds to transmit a new request.
I suspect that the lag that you are seeing is simply down to the cellular radio transitioning from either low power or idle state to full power (which can take over 2 seconds).
Check out Reto Meier's excellent DevBytes series on Efficient Data Transfer for an in-depth explanation of what's going on.
The first request cannot leverage a keep-alive obviously, because thankfully Android doesn't keep the connections alive for minutes or hours. Only subsequent requests can reuse keep-alive connections and only after a short period of time.
It's natural that you have to wait in line 3. Before something like conn.getResponseCode() or conn.getInputStream() the HttpURLConnection is in CREATED state. There is no network activity until it's getting in CONNECTED state. Buffered* shouldn't make any difference here.
I've observed long delays when using SSL and there was a time-shift between server and device. This happens very often when using an emulator which is not cold-booted. For that I've a small script running before test. It's important that PC and emulator are in the same time-zone, otherwise it's very contra-productive: (see below, because it's hard to show the command inline).
I can imagine that Android saves battery in putting 3G into sleep mode when there is no activity. This is just speculation, but you could make a test by creating some other network activity with other apps (browser, twitter, ...) and then see whether your app needs the same long "think time" until first connection.
There are other good options for losing time: DNS resolution, Server-side "sleep" (e.g. a virtual machine loading "memory" from disk).
The command to set time of Android emulator:
adb -e shell date -s `date +"%Y%m%d.%H%M%S"`
Edit
To further analyze the problem, you could run tcpdump on your server. Here is tutorial in case you don't know it well. Store the dumps to files (pcap) and then you can view them with wireshark. Depending on the traffic on your CentOS server you have to set some filters so you only record the traffic from your Android device. I hope that this gives some insight to the problem.
To exclude your server from being the bad guy, you could create a small script with curl commands doing the equivalent as your app.
You could create a super-tiny service without database or other i/o dependencies and measure the performance. I don't know "Go", but the best thing would be a static JSON file delivered by Apache or nginx. If you only have Go, then take something like /ping -> { echo "pong" }. Please tell us your measurements and observations.
Instead of using so many classes I suggest you use this library
you can have a look at here
http://loopj.com/android-async-http/
your code will become very very less , instead of declaring so many classes writing bulk of code , you can just use 4 lines of code
AsyncHttpClient client = new AsyncHttpClient();
client.get("http://www.google.com", new AsyncHttpResponseHandler() {
#Override
public void onSuccess(String response) {
System.out.println(response);
}
});
It is very efficient in geting the response very quickly(1 or 2 secs including parsing).
I hope this will help you out. :)

Getting "SocketException : Connection reset by peer" in Android

My app needs to contact the same device it is working on, via http://127.0.0.1/... (a localhost url).
For some reason, about 50% of the times (and maybe exactly 50%) when I reach a website there with JSON content, I get the exception:
java.net.SocketException: recvfrom failed: ECONNRESET (Connection reset by peer)
For the other 50%, I get perfectly good results. I've tried to do polls (and even large delay between polls), but I keep getting the same weird results.
I've searched the internet and also here, and I'm not sure why it occurs. Does the peer mean that the client has caused it? Why does it happen, and how should i handle it?
Some websites say that it's a common thing, but I didn't find what's the best thing to do in such cases.
Ok, the answer was that it's the server's fault - it had to close the connection after each request.
It might be that Android keeps a pool of connections and use the old one or something like that.
Anyway , now it works.
EDIT: according to the API of HttpURLConnection, this can be solved on the client side too:
The input and output streams returned by this class are not buffered.
Most callers should wrap the returned streams with BufferedInputStream
or BufferedOutputStream. Callers that do only bulk reads or writes may
omit buffering. When transferring large amounts of data to or from a
server, use streams to limit how much data is in memory at once.
Unless you need the entire body to be in memory at once, process it as
a stream (rather than storing the complete body as a single byte array
or string).
To reduce latency, this class may reuse the same underlying Socket for
multiple request/response pairs. As a result, HTTP connections may be
held open longer than necessary. Calls to disconnect() may return the
socket to a pool of connected sockets. This behavior can be disabled
by setting the http.keepAlive system property to false before issuing
any HTTP requests. The http.maxConnections property may be used to
control how many idle connections to each server will be held.
Taken from:
developer.android.com/reference/java/net/HttpURLConnection.html
Try to set this property for your HttpURLConnection before connecting:
conn.setRequestProperty("connection", "close");
This will disable "keep-alive" property which is on by default.
This is an old thread i know. But this might help someone.
In my case this error was caused by the .NET WCF (soap) service. One of the objects in the returning result had a DataMember with get{} property but no set{} property.
For serialization to occur every DataMember should have both get{} & set{} available. I implemented an empty set{} (empty due to my business rules), and problem was solved.
My scenerio is a specific bad server implementation, but maybe it'll help someone saving time when troubleshooting.
I was having a lot of these Connection reset by peer when I was visiting certain web pages or downloading files (from my app or the Android browser).
Turned out it was my 3G carrier that blocked the connections (e.g. downloading an .exe file was forbidden).
Do you have the same problem on Wifi ?
in my situation the problem has been solved by cleaning the proxy address and port from APN which was produced by the operator.
as I have tested, using IP address of remote server instead of domain name also can solve the problem.
In our case the issue is in the Server side (Application Pool configuration in IIS). I resolved it by setting Maximum Worker Processes to 1. Setting it with value more than 1 will enable Web Garden which seems to be the issue.

HttpClient times out when changing from 3G to WiFi

I perform some large downloads. I start a download being connected to 3G, all is fine. Then, I switch to WiFi connection, but the request returns a timeout exception. I have used HttpClient library. I have implemented a retry mechanism, so, when the request returns an exception, it sleeps for 0.5 seconds and tries to execute again and again. I would expect that, after connecting to a WiFi, the Http request could execute. But it seems that the Http execute method returns a null response, all the time after that. Very strange, if I commute again to 3G, the execute method returns again a good response. Can anyone help me please :) ?
First, it seems that it may be more convenient to use DownloadManager for large files - it handles retry and everything.
As for HttpClient - it's known to have some issues, but i'm not sure if you bumped into one of them or just overlooked something. It's been deprecated as of Gingerbread, you may want to try HttpUrlConnection instead, it's said to have less problems than HttpClient.
Also, when switching between WIFI and cell connections, HttpClient may need to be reinitialized completely, there's http range header to tell server which byte you want to continue downloading from. But again, I suggest you give DownloadManager a try, it may save you a lot of time.
This might be a routing problem:
when switching between different network types usually the local ip address and, more important, the local routing table changes, due to a different gateway being used. This means that packages that traveled fine between client and server wont reach any destination after a network change, if they are send the same route. Most likely your client implementation has to be notified of the change or even restartet completely, so that the routing strategy is reinitilized.
If the documentation of the implementation components you use dont reveal anything you could try to track this down using a package sniffer like wireshark. Typically packages running into nirwhana show up easily there.

Categories

Resources