Develop Reference

Android could not receive form submit response

I have a python webserver, which is listening for connections, and responds to them based on the request. The portion of code on python server of interest is POST request (http://192.168.0.1/conf_wifi_app.html). It takes 4 arguments
username (admin)
password (admin)
essid (Home wifi network SSID)
passphrase (Home wifi network password)
On python server, after the post body parameters are validated, a response is to be sent like so (notice I've put logs for debugging):
json_response = '{"error":false,' + '"code":"' + str(activation_code) + '","mac":"' + str(macaddress) + '","message":"Device configured successfully"}'
bytes_sent = client_s.send(json_response)
client_s.close()
print("Bytes sent " + str(bytes_sent))
print("json_response : " + json_response)
where client_s is the client socket. "send" function on socket should send the response (json_response), and then we close the socket. Logs print the number of bytes which is actually sent.
The client responds perfectly well when POST request is done from the web browser or from postman plugin. Just for some reference, I've put the raw request when invoked from postman plugin on chrome browser:
POST /conf_wifi_app.html HTTP/1.1
Host: 192.168.0.1
Connection: keep-alive
Content-Length: 67
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/68.0.3440.75 Chrome/68.0.3440.75 Safari/537.36
Cache-Control: no-cache
Origin: chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop
Postman-Token: 4f4a14a7-857d-666f-a2db-279731c83b4a
Content-Type: application/x-www-form-urlencoded
Accept: /
Accept-Encoding: gzip, deflate
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
essid=NETGEAR-XXXX&passphrase=XXXXXXXX&username=admin&password=admin&submit=submit
The response is received when the POST request is made from web browser (or postman). Now I was trying to create an android app which does the same POST request as follows:
HttpURLConnection urlConnection = null;
try {
Map<String,Object> params = new LinkedHashMap<>();
params.put("user", Constants.DEVICE_DEFAULT_USER);
params.put("username", Constants.DEVICE_DEFAULT_USER);
params.put("password", Constants.DEVICE_DEFAULT_PASSWORD);
params.put("essid", homeWifiSSID.replaceAll("^\"|\"$", ""));
params.put("passphrase", homeWifiPassword);
StringBuilder urlParameters = new StringBuilder();
for (Map.Entry<String,Object> param : params.entrySet()) {
if (urlParameters.length() != 0) urlParameters.append('&');
urlParameters.append(URLEncoder.encode(param.getKey(), "UTF-8"));
urlParameters.append('=');
urlParameters.append(URLEncoder.encode(String.valueOf(param.getValue()), "UTF-8"));
}
byte[] postData = urlParameters.toString().getBytes("UTF-8");
int postDataLength = postData.length;
URL url = new URL(Constants.DEVICE_CONFIG_URL);
urlConnection = (HttpURLConnection) network.openConnection(url);
urlConnection.setDoInput( true );
urlConnection.setDoOutput( true );
urlConnection.setInstanceFollowRedirects( false );
urlConnection.setRequestMethod( "POST" );
urlConnection.setRequestProperty( "Content-Type", "application/x-www-form-urlencoded");
urlConnection.setRequestProperty( "charset", "utf-8");
urlConnection.setRequestProperty( "Content-Length", Integer.toString( postDataLength ));
urlConnection.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8");
urlConnection.setRequestProperty("Accept-Encoding", "gzip, deflate");
urlConnection.setUseCaches( false );
urlConnection.setConnectTimeout(OuroborosAPI.CONNECTION_TIMEOUT);
urlConnection.setReadTimeout(OuroborosAPI.CONNECTION_TIMEOUT);
try( DataOutputStream wr = new DataOutputStream( urlConnection.getOutputStream())) {
wr.write( postData );
wr.flush();
wr.close();
}
Reader in = new BufferedReader(new InputStreamReader(urlConnection.getInputStream(), "UTF-8"));
for (int c; (c = in.read()) >= 0;) {
System.out.print((char) c);
}
}
catch (Exception e) {
}
From android app, the post raw data received is as follows:
POST /conf_wifi.html HTTP/1.1
Content-Type: application/x-www-form-urlencoded
charset: utf-8
Content-Length: 85
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8
User-Agent: Dalvik/2.1.0 (Linux; U; Android 7.1.1; Moto G (5S) Plus Build/NPSS26.116-61-11)
Host: 192.168.0.1
Connection: Keep-Alive
Accept-Encoding: gzip
essid=NETGEAR-XXXX&passphrase=XXXXXXXX&username=admin&password=admin&submit=submit
The python socket in this case does send data (as confirmed from the logs), but the android errors out saying unexpected end of string.
I've literally tried every thing (like adding extra headers, etc) but to no avail. Please help or suggest where I may be going wrong.

The problem was not that I was not sending \n ended lines, or not using readline()
I was not sending the HTML raw headers like so
HTTP/1.1 200 OK
Content-Length: 9328
Content-Type: text/html
... Actual content/payload....
Once I sent the headers also, the code worked without any problems.

Getting content from website with HttpURLConnection (Android API 18)

How to get content from website with HttpURLConnection for android APIs 18 and above? The code is working fine for API 23, but inputstream is returning odd values for API 18. This is what I get when I try to read data from URL with API 18:
TTP/1.1 200 OK Content-Type: application/json; charset=utf-8
Access-Control-Allow-Origin: * Cache-Control: no-cache, no-store,
max-age=0, must-revalidate Pragma: no-cache Expires: Mon, 01 Jan 1990
00:00:00 GMT Date: Wed, 03 Aug 2016 19:01:33 GMTContent-Encoding: gzip
P3P: CP="This is not a P3P policy! See
https://support.google.com/accounts/answer/151657?hl=en for more
info." P3P: CP="This is not a P3P policy! See
https://support.google.com/accounts/answer/151657?hl=en for more
info." X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block Server: GSE Set-Cookie:
NID=83=Nb29w9eQo3Fdx_bQuj6YbdLwSfxjuQT4f1Lcb87IbTXQqdGGh6OyuxxB0XGWxNIiAfMdCePDtDb5P9vMYQvbln7svacSJjFkWnU6-B4AN9vLHHY4RUdL3Xny7zSmE8Lm;Domain=.googleusercontent.com;Path=/;Expires=Thu,
02-Feb-2017 19:01:33 GMT;HttpOnly Set-Cookie:
NID=83=Z9EmVPVCfKYu4FrAHTVHDPMNM80s23cO6P1VqJAocZHnrQb8QFPKW9BLjQGu5xKOwtqNaT38gTZVJm1zmbT7tVhZAYCQlaSb7dRiSTcqQ71a41cIs4l67RxEkOjXfttC;Domain=.googleusercontent.com;Path=/;Expires=Thu,
02-Feb-2017 19:01:33 GMT;HttpOnly Alternate-Protocol: 443:quic
Alt-Svc: quic=":443"; ma=2592000; v="36,35,34,33,32,31,30"
Transfer-Encoding: chunked 00000001 00000001 � 00000001 00000001 ��
00000001 �� 00000001 �� 00000001 �� �� ��
What would be the reasoning behind this? I can provide the code if needed. I'm fairly new with this and can't find the answer anywhere.
To get response from URL I use
private String downloadUrl(String urlString) throws IOException {
InputStream is = null;
try {
URL url = new URL(urlString);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setReadTimeout(10000 /* milliseconds */);
conn.setConnectTimeout(15000 /* milliseconds */);
conn.setRequestMethod("GET");
conn.setDoInput(true);
// Starts the query
conn.connect();
int responseCode = conn.getResponseCode();
is = conn.getInputStream();
String contentAsString = convertStreamToString(is);
return contentAsString;
} finally {
if (is != null) {
is.close();
}
}
}
and the method which converts the stream into string
private String convertStreamToString(InputStream is) {
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}

I have had a similar problem.. The problem is that you are using an HttpUrlConnection on an https request. So all of your data is going to be encrypted. I would recommend using some other HttpRequest library like http-request by kevinsawicki: https://github.com/kevinsawicki/http-request. This will have easy support for getting headers, bodies (Json, XML, Plaintext) and other meta data about the request.
How to add the maven
In android studio open up the project viewer and open build.gradle (app)
and add this line in the dependencies list
compile 'com.github.kevinsawicki:http-request:6.0'
Now open up the (project) build.gradle and make sure under repositories you have this
mavenCentral()
Example get request
HttpRequest req = HttpRequest.get("https://google.com");
req.trustAllCerts();
req.trustAllHosts(); //If you are having certificate problems
int code = req.code();
String body = req.body();
Log.d("CODE:", String.valueOf(code));
Log.d("BODY:", body);
Best regards, David

you might need
in = new InputStreamReader(
httpResponseCode == HttpURLConnection.HTTP_OK ? conn.getInputStream() : conn.getErrorStream(),
"UTF-8");

Getting Bad Request only on Gingerbread phones and simulator

I am facing a weird problem. I've a piece of code that makes a call to a server. The code works fine when I use Kitkat emulator. However, the same code does not work on Gingerbread phones or emulator. I always get 400 Bad Request from the server. I checked on the server. The error is:
client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23):
Apache is not configured as virtual host.
here is the code I am using:
public void makeServerCall(String serverUrl, String postString) {
URL url = new URL(serverURL + ESS_AUTHENTICATE_URL);
connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setRequestMethod(HTTP_POST);
//forcing user agent to a well known user agent with a hope it will work :-)
connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:1.3.1)");
String base64Tenant = "Basic " + Base64.encodeToString( auth.getBytes(), Base64.DEFAULT);
connection.setRequestProperty(AUTHENTICATE_HEADER, base64Tenant);
connection.addRequestProperty("Cache-Control", "no-cache");
connection.addRequestProperty("Pragma", "no-cache");
connection.setUseCaches(false);
connection.setRequestProperty( "Content-Type", "application/x-www-form-urlencoded");
connection.setRequestProperty("Accept-Charset", "UTF-8");
System.setProperty("http.keepAlive", "false");
connection.setFixedLengthStreamingMode(postString.getBytes().length);
// Send request
PrintWriter out = new PrintWriter(connection.getOutputStream());
out.print(postString);
out.close();
int respCode = connection.getResponseCode();
Log.i(TAG, "Response code = " + respCode);
if (respCode != 200) {
is = connection.getErrorStream();
if (is != null) {
String response = getStringFromStream(is);
Log.d(TAG, "Error occured: " + response);
is.close();
}
}
else {
// process success Response
}
}
Here is the wireshark capture I got when I run the code on Kitkat simulator
POST /removed_path HTTP/1.1
Accept-Charset: UTF-8
User-Agent: Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:1.3.1)
Authorization: Basic ASVGQVVMVEASDFw==
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/x-www-form-urlencoded
Host: correcthost.myserver.com
Connection: Keep-Alive
Accept-Encoding: gzip
Content-Length: 49
this is the post stringHTTP/1.1 200 OK
Date: Tue, 04 Feb 2014 07:52:42 GMT
Here is the wireshark capture from Gingerbread
POST /removed_path HTTP/1.1
Accept-Charset: UTF-8
User-Agent: Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:1.3.1)
Authorization: Basic ASVGQVVMVEASDFw==
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/x-www-form-urlencoded
Content-Length: 49
Host: correcthost.myserver.com
Connection: Keep-Alive
Accept-Encoding: gzip
The only difference between the two wiresharks difference is the order of content-length.
I've access to server too. I am not sure, I can change anything there because it is working for recent versions of Android and iOS. Any ideas on what is going wrong?

After sending HTTP request to server sometimes I get status 302 (redirect)

I try to send POST request to some server with json parameters and I wait json response.
From Android it works fine but from Linux I get status 302 redirect. I really don't know where is my problem.
Here is command from linux:
curl -i -X POST --data "id=105&json={\"orderBy\":0\"maxResults\":50}" "http://mysite.com/ctlClient/"
And I get response:
HTTP/1.1 302 Found
Date: Thu, 04 Jul 2013 12:22:06 GMT
Server: Apache
X-Powered-By: PHP/5.3.19
Set-Cookie: PHPSESSID=dqblvijvttgsdrv2u5tn72p9d6; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
location: http://mysite.com/fwf/online/
Content-Length: 0
Connection: close
Content-Type: text/html
From Server access log:
"POST /ctlClient/ HTTP/1.1" 302 - "-" "curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5"
Android:
String data = "{\"orderBy\":0\"maxResults\":50}";
String WEFI_MAIL_URL = "http://mysite.com/ctlClient/";
DefaultHttpClient httpclient = null;
String out = null;
try {
httpclient = new DefaultHttpClient();
HttpPost httpost = new HttpPost(WEFI_MAIL_URL);
httpost.setHeader("User-Agent", "Apache-HttpClient/4.1 (java 1.5)");
List <NameValuePair> nvps = new ArrayList <NameValuePair>();
nvps.add(new BasicNameValuePair("id", "105"));
nvps.add(new BasicNameValuePair("json", data));
httpost.setEntity(new UrlEncodedFormEntity(nvps, HTTP.UTF_8));
HttpResponse response = httpclient.execute(httpost);
HttpEntity entity = response.getEntity();
if (entity != null) {
InputStream is = entity.getContent();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line = null;
while ( (line = br.readLine()) != null) {
out = line;
}
is.close();
if(isTimeOut == false){
_loadActual();
}
else{
return;
}
} else {
return;
}
} catch (Exception e) {
}
if (httpclient != null) {
httpclient.getConnectionManager().shutdown();
}
}
access log on Android response:
"POST /ctlClient/ HTTP/1.1" 302 - "-" "Apache-HttpClient/4.1 (java 1.5)"
"GET /fwf/online/ HTTP/1.1" 200 13981 "-" "Apache-HttpClient/4.1 (java 1.5)"
We can see that on POST server redirects me to /fwf/online/
From Wireshark i get the same results from both methods:
POST /ctlClient/ HTTP/1.1
User-Agent: Apache-HttpClient/4.1 (java 1.5)
Content-Type: application/x-www-form-urlencoded
Content-Length: 301
Host: mysite.com
Connection: Keep-Alive
Why it works from Android but not from Linux with CURL?
Can anybody help me or give me direction how can i solve it?
Thank you

You want to add -L to your curl command. See http://curl.haxx.se/docs/faq.html#How_do_I_tell_curl_to_follow_HTT.

How can I get a missing cookie with android ? In Firefox yes appear

I can get a cookie with firefox, but not with android.
This is the code:
HttpParams httpparams = new BasicHttpParams();
httpparams.setParameter(ClientPNames.COOKIE_POLICY,CookiePolicy.BROWSER_COMPATIBILITY);
HttpGet httpget = new HttpGet(sURL);
httpget.setParams(httpparams);
httpget.setHeader("User-Agent","Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2");
httpget.setHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
httpget.setHeader("Accept-Language", "es-mx,es;q=0.8,en-us;q=0.5,en;q=0.3");
httpget.setHeader("Accept-Encoding", "gzip,deflate");
httpget.setHeader("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.7");
httpget.setHeader("Keep-Alive", "115");
BasicHttpResponse response = (BasicHttpResponse) httpclient.execute(httpget);
Using wireshrack I saw:
GET /login.php HTTP/1.1
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: es-mx,es;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Host: www.conquerclub.com
Connection: Keep-Alive
and the response is using my code:
HTTP/1.1 200 OK
Date: Mon, 09 Aug 2010 05:41:38 GMT
Server: Apache/2.0.52 (Red Hat)
X-Powered-By: PHP/5.2.13
Set-Cookie: referer=%5Bdirect%5D; path=/
Set-Cookie: referer60=%5Bdirect%5D; expires=Wed, 08-Sep-2010 05:41:38 GMT; path=/
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html
but whith Firefox the response is
HTTP/1.1 200 OK
Date: Mon, 09 Aug 2010 05:11:13 GMT
Server: Apache/2.0.52 (Red Hat)
X-Powered-By: PHP/5.2.13
Set-Cookie: referer=%5Bdirect%5D; path=/
Set-Cookie: referer60=%5Bdirect%5D; expires=Wed, 08-Sep-2010 05:11:13 GMT; path=/
Set-Cookie: PHPSESSID=sv8f6ro571t9rv999mu6jtkbu3; path=/; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Encoding: gzip
Vary: Accept-Encoding
Content-Length: 4324
Connection: close
Content-Type: text/html
again using wireshark to see firefox I got this:
GET / HTTP/1.1
Host: www.conquerclub.com
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: es-mx,es;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
It is the same, but the server don't respond this line
Set-Cookie: PHPSESSID=sv8f6ro571t9rv999mu6jtkbu3; path=/; HttpOnly
why? any idea would be very well receive.

I came across a similar problem with android. This happens because of the cookie origin policy in Browser compatibility mode. The HTTP client in android is rejecting the cookie because the cookie's path does not match the current requested document's path.
e.g. if your cookie path is: /mypath, and your cookie is originating from: /mypath/myAdditionalPath, the http client will reject the cookie (internally throwing a MalformedCookieException). But browsers will accept such cookies.
To workaround this problem in your code, you will have to implement your own cookie spec the one that extends from BrowserCompatSpec. Here's some sample skeleton code:
public class MyCookieSpec extends BrowserCompatSpec {
#Override
public void validate(Cookie cookie, CookieOrigin origin) throws
MalformedCookieException {
if(cookie == null) {
throw new IllegalArgumentException("Cookie cannot be null");
}
if(origin == null) {
throw new IllegalArgumentException("Cookie origin cannot be null");
}
// use these logs to see what is the difference between paths of the
// cookieOrigin and the cookie.
String pth = cookie.getPath();
Log.i(TAG, "Cookie ====================================> " + cookie);
Log.i(TAG, "CookieOrigin ====================================> " + origin);
// Check if the cookie is from the same domain, if so return silently
// or else throw a MalformedCookieException
}
#Override
public boolean match(Cookie cookie, CookieOrigin origin) {
if(cookie == null) {
throw new IllegalArgumentException("Cookie cannot be null");
}
if(origin == null) {
throw new IllegalArgumentException("Cookie origin cannot be null");
}
if(Log.isLoggable(TAG, Log.DEBUG)) {
Log.d(TAG, "Matching cookie " + cookie + " with origin " + origin);
}
// if the cookie is originating from the same domain as of the origin
// return true or return false. Be careful here and only return true if
// the cookie is originating from the same domain as that of what is in the
// cookie's path
return true;
}
}
To register and use this cookie spec with your http client, use the following:
DefaultHttpClient httpClient = new DefaultHttpClient(cm, params);
httpClient.getCookieSpecs().register("myspec", new CookieSpecFactory() {
public CookieSpec newInstance(HttpParams hp) {
return new MyCookieSpec();
}
});
httpClient.getParams().setParameter(ClientPNames.COOKIE_POLICY, "myspec");

Sorry guys. I realized that i made a mistake
With firefox, I request www.conquerclub.com with path "/"
BUT using my code I request www.conquerclub.com with path "/login.php"
After I deleted it "login.php", the request return 3 cookies