I have a problem - I am trying to make a POST request from my app, but it always drop IOexception error on response(like response code, message etc...).
I made that POST request (what i am trying to make from my app) from my PC and I used WIRESHARK to see the response, but the response comes in multiple PIECES, not in one as usual.
In my app I use httpulrconnection in acync task.
How do I manage to catch all of the response?
I've added a pics of my WIRESHARK file and make it red which contains the response:
Wireshark http post with response in pieces
This is just how anything over a networks is working.
The data is splitted into smaller packets which will get routed through the network. The maximum of the packet size is defined by the different hops in the network.
In most cases those packets are something like 1460bytes of actual payload.
TCP will make this packet-handling transparent, as it is a protocol on top of this raw layer. So normally you shouldn't take care of this.
If you want to see the whole response (like your application will see it), you can view single tcp streams in wireshark:
https://www.wireshark.org/docs/wsug_html_chunked/ChAdvFollowTCPSection.html
Edit
The errors you are seeing don't have anything to do with the packaging.
Related
i'm developping an android app that communicate with a BLE device(in kotlin).
i'm sending a command to the ble device, the device usually give the following response:
-OK(confirming the command was received and valid)
-one or multiple line of data
-Done
in onCharacteristicWrite i tryed to show this.value to read the response but look like it's not the way to do it. i googled a lot without lot of success.
so my question is how could i get the data from the response.the device send?
i try to call a readCharateristic without success either.
thank you for your help
GATT does not have any protocol feature of a request - response transaction where both messages embed data.
For Write Request - Write Response, only the request can contain data. The response can only be empty or an error code.
For Read Request - Read Response, the request contain only what characteristic is to be read and the response contains some data or an error code.
To implement a request - response mechanism using GATT, one usually follows one of the following strategies:
Use Write Request or Write Without Response for the request message and Notification for the response message.
Use Write Request or Write Without Response for the request message. When the GATT server produces a response message, it is put in a characteristic that the client will then read using a Read Request.
The first method has lowest overhead and best throughput.
What you should do is to investigate what method your peripheral actually uses. If it sends its responses in notifications, you must enable these and implement the onCharacteristicChanged callback.
I'm doing a little test where I send out a short string(4-8 bytes) to a client every 0.5 seconds from a Node.js server using ws. The client is either using iOS/Android or a web browser. The client does not send anything back to the server, except for TCP-ACKS I suppose. The weird thing is, when I'm debugging the app in iOS using XCode network report, I can only see that the client sends out some bytes(approx 500) when the connections establishes, probably during the HTTP handshake. The remaining time ZERO data is going out from the device, there is only data coming in. The same results is achieved when receiving data in Chrome and tracking the data using Nettop.
The thing that makes so confused is that on the Android, almost the same amount of data that goes in to the device goes out when inspecting the network usage with Android profiler/Battery Historian/TrafficStats. I have tried using different libraries for the Websocket implementation and using different Android devices.
I have a hard time believing the ACKS sent out by the android is as big as the message received, even though it's just a small string of four characters.
So my questions are:
Could the case be that Nettop/XCode network report is simply ignoring all the ACKS, so in reality as much data is sent out in Chrome/iOS as in Android?
Is there something 'Wrong' with the libraries used in Android or could it be something with its operating system?
Could an ACK be as big as a simple TCP-package with 4 characters in it?
The result below when using Websocket
The data received/transmitted when using Android Battery Historian
The data received/transmitted on iOS using Network Report
Could an ACK be as big as a simple TCP-package with 4 characters in it?
An ACK consists of the IP and the TCP header and no payload. With IPv4 this means at least 20 bytes IP header and 20 bytes TCP header, i.e. 40 bytes. A packet with 4 bytes payload is only larger by 4 bytes, i.e. 44 bytes or just 10%.
The network report in Android shows 68350 in vs. 61370 in bytes, which is a difference of 11%. This matches the expected difference.
I'm not familiar with what iOS measure here, but it probably either measures only the application payload (i.e. the 4 bytes) or simply ignores packets with no payload, i.e. the ACK's.
I am using the class HttpUrlConnection for requesting JSON responses
I realized that no matter if I set or not
System.setProperty("http.keepAlive", "false");
The first response is always going to take longer, while the next responses are very quick, with and without keepAlive. I am not even using SSL.
Notice that, my app doesn't need to perform any authentication with the server, so there isn't any startup call to the webservices. The first request I make to the webservices is actually the very first.
I am also verifying server-side with "netstat", that by setting keepAlive false on the Android client the connections disappear straight away, while without specifying keepAlive false they keep staying as "ESTABLISHED".
How can you explain that subsequent responses are quicker even if the connection doesn't persist?
ANDROID CODE:
line 1) URL url = new URL(stringUrl);
line 2) HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
line 3) InputStream instream = new BufferedInputStream(urlConnection.getInputStream());
Until line 2 everything always gets executed very quickly, either with keepAlive or not. Line 3 in the first request takes around 3 seconds, while in all subsequent always less than 1 second. Each request is about 0.5KB gzipped.
SYSTEM:
I am testing using a Nexus 5, connected via 3G
My webservices are written in Go, running on a CentOS 6.4 linux server
I am using standard tcp v4
UPDATE:
For the moment I have decided to use a trick: when the fragment is resuming, I make a HTTP HEAD request to the server. In this way all subsequent calls in the next 10 seconds are very quick. If the user waits more than 10 seconds then the first one will be slow, and the next ones will be quick again. This is all happening without using KeepAlive.
It's a really big mistery now. It looks like there is some kind of "awake" period which lasts for about 10 seconds. I don't think there is anything strange on my code which can result on that. Also because everything seems to happen during the line 3 I reported above.
SOLVED! thanks to Mark Allison!
Here is a very clear explanation:
http://developer.android.com/training/efficient-downloads/efficient-network-access.html
Also, everything can easily be monitored using Android DDMS's Network Statistics. If you wait some seconds (let's say 20) from last request, you can see that it takes 2 seconds to transmit a new request.
I suspect that the lag that you are seeing is simply down to the cellular radio transitioning from either low power or idle state to full power (which can take over 2 seconds).
Check out Reto Meier's excellent DevBytes series on Efficient Data Transfer for an in-depth explanation of what's going on.
The first request cannot leverage a keep-alive obviously, because thankfully Android doesn't keep the connections alive for minutes or hours. Only subsequent requests can reuse keep-alive connections and only after a short period of time.
It's natural that you have to wait in line 3. Before something like conn.getResponseCode() or conn.getInputStream() the HttpURLConnection is in CREATED state. There is no network activity until it's getting in CONNECTED state. Buffered* shouldn't make any difference here.
I've observed long delays when using SSL and there was a time-shift between server and device. This happens very often when using an emulator which is not cold-booted. For that I've a small script running before test. It's important that PC and emulator are in the same time-zone, otherwise it's very contra-productive: (see below, because it's hard to show the command inline).
I can imagine that Android saves battery in putting 3G into sleep mode when there is no activity. This is just speculation, but you could make a test by creating some other network activity with other apps (browser, twitter, ...) and then see whether your app needs the same long "think time" until first connection.
There are other good options for losing time: DNS resolution, Server-side "sleep" (e.g. a virtual machine loading "memory" from disk).
The command to set time of Android emulator:
adb -e shell date -s `date +"%Y%m%d.%H%M%S"`
Edit
To further analyze the problem, you could run tcpdump on your server. Here is tutorial in case you don't know it well. Store the dumps to files (pcap) and then you can view them with wireshark. Depending on the traffic on your CentOS server you have to set some filters so you only record the traffic from your Android device. I hope that this gives some insight to the problem.
To exclude your server from being the bad guy, you could create a small script with curl commands doing the equivalent as your app.
You could create a super-tiny service without database or other i/o dependencies and measure the performance. I don't know "Go", but the best thing would be a static JSON file delivered by Apache or nginx. If you only have Go, then take something like /ping -> { echo "pong" }. Please tell us your measurements and observations.
Instead of using so many classes I suggest you use this library
you can have a look at here
http://loopj.com/android-async-http/
your code will become very very less , instead of declaring so many classes writing bulk of code , you can just use 4 lines of code
AsyncHttpClient client = new AsyncHttpClient();
client.get("http://www.google.com", new AsyncHttpResponseHandler() {
#Override
public void onSuccess(String response) {
System.out.println(response);
}
});
It is very efficient in geting the response very quickly(1 or 2 secs including parsing).
I hope this will help you out. :)
I'm seeing a problem with the HttpClient posting/putting a long StringEntity. When the entity is short, there is no problem at all. However, when the length exceeds a value (something around 1400 characters), the http packet can never be sent out (I sniffed the interface using WireShark). Actually, the connection is established, but the data is not transmitted, so the receiver side got a timeout exception.
I'm wondering if these is a lenght limit.
I tried to increase the connection timeout and socket timeout, which only made me wait longer to see the timeout ...
And i also tried to use InputStreamEntity, didn't work either.
[Update]: I tried to used HttpURLConnection directly instead of HttpClient. The same problem still exists. However, I do have some findings. When I forced the packet to be cut into chunks (using HttpURLConnection.setChunkedStreamingMode), WireShare did captured some segment of the packet, with the previous segments missing. I guess this must be a bug in the apache http library.
The issue is resolved. It took me several days to find out that, the problem is with my wireless router. It has some weird settings which truncates large incoming http messages.
It is possible it is timing out on the server side. Also make sure you are using the org.apache.client.httpclient jar. Officially there is no max length for a url. Look here http://www.w3.org/Protocols/rfc2616/rfc2616.html
I think that can help you urllength
From my app I upload a file to our server using this basic code (it is a bit more than this of course but this is basically it)
HttpParams params = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(params, 15000);
HttpConnectionParams.setSoTimeout(params, 5 * 60 * 1000);
HttpClient client = new DefaultHttpClient(params);
HttpPost post = new HttpPost("upload url");
HttpEntity requestEntity = (new FileEntity(tmpFile, "multipart/form-data;boundary="+boundary);
post.setEntity(requestEntity);
HttpResponse response = (HttpResponse) client.execute(post);
That works fine MOST of the time.
For some phones running Android 2.2+ the file received on the server
side is not complete. Small portions of the file are simply missing,
and the parts that are missing are at different locations of the file
each time.
We have verified this by comparing the file from the app against what
is received on the server side. On the server side we captured
packets with tcpdump to make sure it wasn't an issue with our web
server or web server code.
We also checked the data with tcpdump from the phone. The tcpdump file from the phone DOES differ from the data we are trying to send. For one case we did analysis on the tcpdump file is missing data between the address 8d68 and 9000 from the file. The packets from tcpdump line up with those addresses (one packet has a portion of the data up until 8d68 and then the next packet has data starting from 9000).
For these phones the problem only happens some of the time. Sometimes
file uploads work and the entire file is received intact on our end.
This is happening ONLY for 2.2+ phones. It happens for a wide variety
of phones, and a variety of carriers, and for hundreds of users. It appears to happen over both wifi and 3g based on the IP addresses seen on the server side.
This is anecdotal but when trying to get this to happen on my Nexus over the past 2 days I have seen it happen 6 times and those times are always right around when I am entering or leaving the room close to a certain wifi router. The rest of the day when I'm in the office on a different wifi router or on a cell network the issue never happens. My theory being that the app is busy sending data and now we move from wifi to cell network or vice versa, is that a dumb idea or a possibility?
I can put the tcpdump files and data files up somewhere if anyone cares to take a look.
What else should I be investigating to figure out the reason for this?
I was facing some similar issue when uploading the binary data. I figured out that it was a problem with the server code.
There were some filters at the server side, which were reading the incoming requests and trying to log them. The code was trying to put the incoming stream in some String type of DS. Its was because of this that the binary images were distorted.
Just check if there are any filters at the server side. Hope this helps