Develop Reference

Android could not receive form submit response

I have a python webserver, which is listening for connections, and responds to them based on the request. The portion of code on python server of interest is POST request (http://192.168.0.1/conf_wifi_app.html). It takes 4 arguments
username (admin)
password (admin)
essid (Home wifi network SSID)
passphrase (Home wifi network password)
On python server, after the post body parameters are validated, a response is to be sent like so (notice I've put logs for debugging):
json_response = '{"error":false,' + '"code":"' + str(activation_code) + '","mac":"' + str(macaddress) + '","message":"Device configured successfully"}'
bytes_sent = client_s.send(json_response)
client_s.close()
print("Bytes sent " + str(bytes_sent))
print("json_response : " + json_response)
where client_s is the client socket. "send" function on socket should send the response (json_response), and then we close the socket. Logs print the number of bytes which is actually sent.
The client responds perfectly well when POST request is done from the web browser or from postman plugin. Just for some reference, I've put the raw request when invoked from postman plugin on chrome browser:
POST /conf_wifi_app.html HTTP/1.1
Host: 192.168.0.1
Connection: keep-alive
Content-Length: 67
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/68.0.3440.75 Chrome/68.0.3440.75 Safari/537.36
Cache-Control: no-cache
Origin: chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop
Postman-Token: 4f4a14a7-857d-666f-a2db-279731c83b4a
Content-Type: application/x-www-form-urlencoded
Accept: /
Accept-Encoding: gzip, deflate
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
essid=NETGEAR-XXXX&passphrase=XXXXXXXX&username=admin&password=admin&submit=submit
The response is received when the POST request is made from web browser (or postman). Now I was trying to create an android app which does the same POST request as follows:
HttpURLConnection urlConnection = null;
try {
Map<String,Object> params = new LinkedHashMap<>();
params.put("user", Constants.DEVICE_DEFAULT_USER);
params.put("username", Constants.DEVICE_DEFAULT_USER);
params.put("password", Constants.DEVICE_DEFAULT_PASSWORD);
params.put("essid", homeWifiSSID.replaceAll("^\"|\"$", ""));
params.put("passphrase", homeWifiPassword);
StringBuilder urlParameters = new StringBuilder();
for (Map.Entry<String,Object> param : params.entrySet()) {
if (urlParameters.length() != 0) urlParameters.append('&');
urlParameters.append(URLEncoder.encode(param.getKey(), "UTF-8"));
urlParameters.append('=');
urlParameters.append(URLEncoder.encode(String.valueOf(param.getValue()), "UTF-8"));
}
byte[] postData = urlParameters.toString().getBytes("UTF-8");
int postDataLength = postData.length;
URL url = new URL(Constants.DEVICE_CONFIG_URL);
urlConnection = (HttpURLConnection) network.openConnection(url);
urlConnection.setDoInput( true );
urlConnection.setDoOutput( true );
urlConnection.setInstanceFollowRedirects( false );
urlConnection.setRequestMethod( "POST" );
urlConnection.setRequestProperty( "Content-Type", "application/x-www-form-urlencoded");
urlConnection.setRequestProperty( "charset", "utf-8");
urlConnection.setRequestProperty( "Content-Length", Integer.toString( postDataLength ));
urlConnection.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8");
urlConnection.setRequestProperty("Accept-Encoding", "gzip, deflate");
urlConnection.setUseCaches( false );
urlConnection.setConnectTimeout(OuroborosAPI.CONNECTION_TIMEOUT);
urlConnection.setReadTimeout(OuroborosAPI.CONNECTION_TIMEOUT);
try( DataOutputStream wr = new DataOutputStream( urlConnection.getOutputStream())) {
wr.write( postData );
wr.flush();
wr.close();
}
Reader in = new BufferedReader(new InputStreamReader(urlConnection.getInputStream(), "UTF-8"));
for (int c; (c = in.read()) >= 0;) {
System.out.print((char) c);
}
}
catch (Exception e) {
}
From android app, the post raw data received is as follows:
POST /conf_wifi.html HTTP/1.1
Content-Type: application/x-www-form-urlencoded
charset: utf-8
Content-Length: 85
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8
User-Agent: Dalvik/2.1.0 (Linux; U; Android 7.1.1; Moto G (5S) Plus Build/NPSS26.116-61-11)
Host: 192.168.0.1
Connection: Keep-Alive
Accept-Encoding: gzip
essid=NETGEAR-XXXX&passphrase=XXXXXXXX&username=admin&password=admin&submit=submit
The python socket in this case does send data (as confirmed from the logs), but the android errors out saying unexpected end of string.
I've literally tried every thing (like adding extra headers, etc) but to no avail. Please help or suggest where I may be going wrong.

The problem was not that I was not sending \n ended lines, or not using readline()
I was not sending the HTML raw headers like so
HTTP/1.1 200 OK
Content-Length: 9328
Content-Type: text/html
... Actual content/payload....
Once I sent the headers also, the code worked without any problems.

Received length differs from sent length

I am using an Android app to send a base64 encoded string to a CherryPy server. The Android code works like this:
URL url = new URL("http://foo.bar/blabla");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestProperty("Content-Type", "application/octet-stream");
conn.setDoOutput(true);
conn.setFixedLengthStreamingMode(base64s.length());
OutputStream out = new BufferedOutputStream(conn.getOutputStream());
out.write(base64s.getBytes());
So, you'd say the amount of bytes sent equals the amount of bytes in the Content-Length header. However, in Python, when I run this:
cl = cherrypy.request.headers['Content-Length']
rawbody = cherrypy.request.body.read()
print "{} bytes, {}".format(len(rawbody), cl)
The numbers cl and len(rawbody) are different.
How can that be?

Maybe you forgot to close the stream with out.close(); ?

your server sould send a "close header" so the client will no its end stream for him.

Caching of responses in HttpUrlConnection of android

In my app , i have implemented HttpResponseCache to cache the responses so that it could be used instead of hitting the server. For a particular api , the server returns the header Cache-Control as no-cache;must-revalidate. It has the header ETag also. The problem is the response of this api is not cached. As a result each time I request the api , server returns 200.
Does no-cache,must-revalidate mean the response won't/shouldn't be cached ?
Please find below the request and response headers of the http request :
Request Headers :
GET HTTP/1.1
User-Agent
Accept application/json
Cache-Control max-age=0
Cookie
Host
Connection Keep-Alive
Accept-Encoding gzip
Response Headers :
HTTP/1.1 200 OK
Server Apache-Coyote/1.1
Cache-Control no-cache, must-revalidate
ETag "c683a0301c68c566fcc706f5cd82f1f8"
Content-Type application/json;charset=UTF-8
Transfer-Encoding chunked
Content-Encoding gzip
Vary Accept-Encoding
Date Mon, 24 Feb 2014 04:44:03 GMT
Sending HTTP_GET request :
URL url = new URL(this.url);
HttpURLConnection conn = null;
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setReadTimeout(timeout);
conn.setConnectTimeout(timeout);
conn.setInstanceFollowRedirects(true);
conn.setRequestProperty("User-Agent", this.userAgent);
conn.setUseCaches(true);
conn.setDefaultUseCaches(true);
conn.connect();
this.responseCode = conn.getResponseCode();
this.extractResponseHeaders(conn.getHeaderFields());
InputStream inputStream = null;
if (this.responseCode >= 400 ) {
inputStream = conn.getErrorStream();
} else {
inputStream = conn.getInputStream();
}
try {
if(null != inputStream){
this.response = convertToString(inputStream);
}
} catch (JSONException e) {
e.printStackTrace();
}

Getting Bad Request only on Gingerbread phones and simulator

I am facing a weird problem. I've a piece of code that makes a call to a server. The code works fine when I use Kitkat emulator. However, the same code does not work on Gingerbread phones or emulator. I always get 400 Bad Request from the server. I checked on the server. The error is:
client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23):
Apache is not configured as virtual host.
here is the code I am using:
public void makeServerCall(String serverUrl, String postString) {
URL url = new URL(serverURL + ESS_AUTHENTICATE_URL);
connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setRequestMethod(HTTP_POST);
//forcing user agent to a well known user agent with a hope it will work :-)
connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:1.3.1)");
String base64Tenant = "Basic " + Base64.encodeToString( auth.getBytes(), Base64.DEFAULT);
connection.setRequestProperty(AUTHENTICATE_HEADER, base64Tenant);
connection.addRequestProperty("Cache-Control", "no-cache");
connection.addRequestProperty("Pragma", "no-cache");
connection.setUseCaches(false);
connection.setRequestProperty( "Content-Type", "application/x-www-form-urlencoded");
connection.setRequestProperty("Accept-Charset", "UTF-8");
System.setProperty("http.keepAlive", "false");
connection.setFixedLengthStreamingMode(postString.getBytes().length);
// Send request
PrintWriter out = new PrintWriter(connection.getOutputStream());
out.print(postString);
out.close();
int respCode = connection.getResponseCode();
Log.i(TAG, "Response code = " + respCode);
if (respCode != 200) {
is = connection.getErrorStream();
if (is != null) {
String response = getStringFromStream(is);
Log.d(TAG, "Error occured: " + response);
is.close();
}
}
else {
// process success Response
}
}
Here is the wireshark capture I got when I run the code on Kitkat simulator
POST /removed_path HTTP/1.1
Accept-Charset: UTF-8
User-Agent: Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:1.3.1)
Authorization: Basic ASVGQVVMVEASDFw==
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/x-www-form-urlencoded
Host: correcthost.myserver.com
Connection: Keep-Alive
Accept-Encoding: gzip
Content-Length: 49
this is the post stringHTTP/1.1 200 OK
Date: Tue, 04 Feb 2014 07:52:42 GMT
Here is the wireshark capture from Gingerbread
POST /removed_path HTTP/1.1
Accept-Charset: UTF-8
User-Agent: Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:1.3.1)
Authorization: Basic ASVGQVVMVEASDFw==
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/x-www-form-urlencoded
Content-Length: 49
Host: correcthost.myserver.com
Connection: Keep-Alive
Accept-Encoding: gzip
The only difference between the two wiresharks difference is the order of content-length.
I've access to server too. I am not sure, I can change anything there because it is working for recent versions of Android and iOS. Any ideas on what is going wrong?

Android socket & HTTP response headers

The funny thing is: I may have accidentally found the solution for Is it possible to send HTTP request using plain socket connection and receive response without headers?. I hope I'm overlooking something.
Anyway, I'm connecting to a web server, send a HTTP request, receive the response and close the connection. The actual HTTP conversation (tcpdump on the server side) looks like this:
GET /blah.php?param1=test1t&param2=test2 HTTP/1.0
HTTP/1.1 200 OK
Date: Sat, 22 Sep 2012 10:16:46 GMT
Server: Apache/2.2.17 (Ubuntu)
X-Powered-By: PHP/5.3.5-1ubuntu7.8
Vary: Accept-Encoding
Content-Length: 17
Connection: close
Content-Type: text/html
<pre>173364</pre>
Cool. It works... so far. Now this is the code. The string szRetval contains only "<pre>173364</pre>".
Socket s = new Socket("1.2.3.4", 80);
//DataInputStream in = new DataInputStream(s.getInputStream());
BufferedReader bin = new BufferedReader(new InputStreamReader(s.getInputStream()));
DataOutputStream out = new DataOutputStream(s.getOutputStream());
out.writeBytes(szRequest);
String szRetval = "";
String szLine = "";
while((szLine=bin.readLine())!=null) {
szRetval += szLine;
}
s.close();
return szRetval;
As you can see from the code sample, I've already switched from DataInputStream to BufferedReader. It makes no difference, so I guess this has to do with the Socket class itself.
Why o why are the http response headers not returned by a (raw) socket??
Android 2.3.3, Network I/O in AsyncTask, not using proxy.