Develop Reference

Android could not receive form submit response

I have a python webserver, which is listening for connections, and responds to them based on the request. The portion of code on python server of interest is POST request (http://192.168.0.1/conf_wifi_app.html). It takes 4 arguments
username (admin)
password (admin)
essid (Home wifi network SSID)
passphrase (Home wifi network password)
On python server, after the post body parameters are validated, a response is to be sent like so (notice I've put logs for debugging):
json_response = '{"error":false,' + '"code":"' + str(activation_code) + '","mac":"' + str(macaddress) + '","message":"Device configured successfully"}'
bytes_sent = client_s.send(json_response)
client_s.close()
print("Bytes sent " + str(bytes_sent))
print("json_response : " + json_response)
where client_s is the client socket. "send" function on socket should send the response (json_response), and then we close the socket. Logs print the number of bytes which is actually sent.
The client responds perfectly well when POST request is done from the web browser or from postman plugin. Just for some reference, I've put the raw request when invoked from postman plugin on chrome browser:
POST /conf_wifi_app.html HTTP/1.1
Host: 192.168.0.1
Connection: keep-alive
Content-Length: 67
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/68.0.3440.75 Chrome/68.0.3440.75 Safari/537.36
Cache-Control: no-cache
Origin: chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop
Postman-Token: 4f4a14a7-857d-666f-a2db-279731c83b4a
Content-Type: application/x-www-form-urlencoded
Accept: /
Accept-Encoding: gzip, deflate
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
essid=NETGEAR-XXXX&passphrase=XXXXXXXX&username=admin&password=admin&submit=submit
The response is received when the POST request is made from web browser (or postman). Now I was trying to create an android app which does the same POST request as follows:
HttpURLConnection urlConnection = null;
try {
Map<String,Object> params = new LinkedHashMap<>();
params.put("user", Constants.DEVICE_DEFAULT_USER);
params.put("username", Constants.DEVICE_DEFAULT_USER);
params.put("password", Constants.DEVICE_DEFAULT_PASSWORD);
params.put("essid", homeWifiSSID.replaceAll("^\"|\"$", ""));
params.put("passphrase", homeWifiPassword);
StringBuilder urlParameters = new StringBuilder();
for (Map.Entry<String,Object> param : params.entrySet()) {
if (urlParameters.length() != 0) urlParameters.append('&');
urlParameters.append(URLEncoder.encode(param.getKey(), "UTF-8"));
urlParameters.append('=');
urlParameters.append(URLEncoder.encode(String.valueOf(param.getValue()), "UTF-8"));
}
byte[] postData = urlParameters.toString().getBytes("UTF-8");
int postDataLength = postData.length;
URL url = new URL(Constants.DEVICE_CONFIG_URL);
urlConnection = (HttpURLConnection) network.openConnection(url);
urlConnection.setDoInput( true );
urlConnection.setDoOutput( true );
urlConnection.setInstanceFollowRedirects( false );
urlConnection.setRequestMethod( "POST" );
urlConnection.setRequestProperty( "Content-Type", "application/x-www-form-urlencoded");
urlConnection.setRequestProperty( "charset", "utf-8");
urlConnection.setRequestProperty( "Content-Length", Integer.toString( postDataLength ));
urlConnection.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8");
urlConnection.setRequestProperty("Accept-Encoding", "gzip, deflate");
urlConnection.setUseCaches( false );
urlConnection.setConnectTimeout(OuroborosAPI.CONNECTION_TIMEOUT);
urlConnection.setReadTimeout(OuroborosAPI.CONNECTION_TIMEOUT);
try( DataOutputStream wr = new DataOutputStream( urlConnection.getOutputStream())) {
wr.write( postData );
wr.flush();
wr.close();
}
Reader in = new BufferedReader(new InputStreamReader(urlConnection.getInputStream(), "UTF-8"));
for (int c; (c = in.read()) >= 0;) {
System.out.print((char) c);
}
}
catch (Exception e) {
}
From android app, the post raw data received is as follows:
POST /conf_wifi.html HTTP/1.1
Content-Type: application/x-www-form-urlencoded
charset: utf-8
Content-Length: 85
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8
User-Agent: Dalvik/2.1.0 (Linux; U; Android 7.1.1; Moto G (5S) Plus Build/NPSS26.116-61-11)
Host: 192.168.0.1
Connection: Keep-Alive
Accept-Encoding: gzip
essid=NETGEAR-XXXX&passphrase=XXXXXXXX&username=admin&password=admin&submit=submit
The python socket in this case does send data (as confirmed from the logs), but the android errors out saying unexpected end of string.
I've literally tried every thing (like adding extra headers, etc) but to no avail. Please help or suggest where I may be going wrong.

The problem was not that I was not sending \n ended lines, or not using readline()
I was not sending the HTML raw headers like so
HTTP/1.1 200 OK
Content-Length: 9328
Content-Type: text/html
... Actual content/payload....
Once I sent the headers also, the code worked without any problems.

Getting content from website with HttpURLConnection (Android API 18)

How to get content from website with HttpURLConnection for android APIs 18 and above? The code is working fine for API 23, but inputstream is returning odd values for API 18. This is what I get when I try to read data from URL with API 18:
TTP/1.1 200 OK Content-Type: application/json; charset=utf-8
Access-Control-Allow-Origin: * Cache-Control: no-cache, no-store,
max-age=0, must-revalidate Pragma: no-cache Expires: Mon, 01 Jan 1990
00:00:00 GMT Date: Wed, 03 Aug 2016 19:01:33 GMTContent-Encoding: gzip
P3P: CP="This is not a P3P policy! See
https://support.google.com/accounts/answer/151657?hl=en for more
info." P3P: CP="This is not a P3P policy! See
https://support.google.com/accounts/answer/151657?hl=en for more
info." X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block Server: GSE Set-Cookie:
NID=83=Nb29w9eQo3Fdx_bQuj6YbdLwSfxjuQT4f1Lcb87IbTXQqdGGh6OyuxxB0XGWxNIiAfMdCePDtDb5P9vMYQvbln7svacSJjFkWnU6-B4AN9vLHHY4RUdL3Xny7zSmE8Lm;Domain=.googleusercontent.com;Path=/;Expires=Thu,
02-Feb-2017 19:01:33 GMT;HttpOnly Set-Cookie:
NID=83=Z9EmVPVCfKYu4FrAHTVHDPMNM80s23cO6P1VqJAocZHnrQb8QFPKW9BLjQGu5xKOwtqNaT38gTZVJm1zmbT7tVhZAYCQlaSb7dRiSTcqQ71a41cIs4l67RxEkOjXfttC;Domain=.googleusercontent.com;Path=/;Expires=Thu,
02-Feb-2017 19:01:33 GMT;HttpOnly Alternate-Protocol: 443:quic
Alt-Svc: quic=":443"; ma=2592000; v="36,35,34,33,32,31,30"
Transfer-Encoding: chunked 00000001 00000001 � 00000001 00000001 ��
00000001 �� 00000001 �� 00000001 �� �� ��
What would be the reasoning behind this? I can provide the code if needed. I'm fairly new with this and can't find the answer anywhere.
To get response from URL I use
private String downloadUrl(String urlString) throws IOException {
InputStream is = null;
try {
URL url = new URL(urlString);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setReadTimeout(10000 /* milliseconds */);
conn.setConnectTimeout(15000 /* milliseconds */);
conn.setRequestMethod("GET");
conn.setDoInput(true);
// Starts the query
conn.connect();
int responseCode = conn.getResponseCode();
is = conn.getInputStream();
String contentAsString = convertStreamToString(is);
return contentAsString;
} finally {
if (is != null) {
is.close();
}
}
}
and the method which converts the stream into string
private String convertStreamToString(InputStream is) {
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}

I have had a similar problem.. The problem is that you are using an HttpUrlConnection on an https request. So all of your data is going to be encrypted. I would recommend using some other HttpRequest library like http-request by kevinsawicki: https://github.com/kevinsawicki/http-request. This will have easy support for getting headers, bodies (Json, XML, Plaintext) and other meta data about the request.
How to add the maven
In android studio open up the project viewer and open build.gradle (app)
and add this line in the dependencies list
compile 'com.github.kevinsawicki:http-request:6.0'
Now open up the (project) build.gradle and make sure under repositories you have this
mavenCentral()
Example get request
HttpRequest req = HttpRequest.get("https://google.com");
req.trustAllCerts();
req.trustAllHosts(); //If you are having certificate problems
int code = req.code();
String body = req.body();
Log.d("CODE:", String.valueOf(code));
Log.d("BODY:", body);
Best regards, David

you might need
in = new InputStreamReader(
httpResponseCode == HttpURLConnection.HTTP_OK ? conn.getInputStream() : conn.getErrorStream(),
"UTF-8");

Android HttpUrlConnection Cannot Post - Error 403

I'm trying to issue a HttpPost against my webservice with spring security csrf.
first, I'm trying to recover the XSRF TOKEN through a GET request, like this
public static CookieManager xcsrfToken() throws IOException{
String token;
URL url = new URL(urlBase);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
CookieManager cookieManager = new CookieManager();
con.connect();
List<String> cookieHeader = con.getHeaderFields().get("Set-Cookie");
if (cookieHeader != null) {
for (String cookie : cookieHeader) {
cookieManager.getCookieStore().add(null, HttpCookie.parse(cookie).get(0));
}
}
System.out.println(con.getHeaderFields());
con.disconnect();
return cookieManager;
}
This is what i get from the con.getHeaderFields()
{null=[HTTP/1.1 200 OK], Cache-Control=[no-cache, no-store, max-age=0, must-revalidate], Content-Language=[pt-BR], Content-Length=[973], Content-Type=[text/html;charset=ISO-8859-1], Date=[Wed, 19 Aug 2015 10:40:18 GMT], Expires=[0], Pragma=[no-cache], Server=[Apache-Coyote/1.1], Set-Cookie=[JSESSIONID=6C9326FBEEA14752068720006F2B5EAA; Path=/webapi/; HttpOnly, XSRF-TOKEN=07cbed7f-834e-4146-8537-0a6b5669f223; Path=/], X-Android-Received-Millis=[1439980819720], X-Android-Response-Source=[NETWORK 200], X-Android-Sent-Millis=[1439980819693], X-Content-Type-Options=[nosniff], X-Frame-Options=[DENY], X-XSS-Protection=[1; mode=block]}
The XSRF-TOKEN is in my cookie, ok!
If I print then with
System.out.println(cookieManager.getCookieStore().getCookies());
I got this
[JSESSIONID=5B1D3E2D3E7B3E1E6572A3839BFF3741, XSRF-TOKEN=4d4048bd-f21c-48c6-895e-5f67523ad963]
Now, I'm trying to issue a POST against the server, like this
public static HttpURLConnection makeRequest(String metodo, String uri, String requestBody) throws IOException{
URL url = new URL(urlBase + uri);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("POST");
con.setDoInput(true);
con.setDoOutput(!metodo.equals("GET"));
con.setRequestProperty("Content-Type", "application/json");
con.setRequestProperty("Cookie", TextUtils.join("," , xcsrfToken().getCookieStore().getCookies()));
con.connect();
InputStream is = con.getErrorStream();
System.out.println(IOUtils.toString(is, "UTF-8"));
System.out.println(con.getHeaderFields());
return con;
}
But the header is comming back without the cookies
{null=[HTTP/1.1 403 Forbidden], Cache-Control=[no-cache, no-store, max-age=0, must-revalidate], Content-Language=[en], Content-Length=[1149], Content-Type=[text/html;charset=utf-8], Date=[Wed, 19 Aug 2015 10:42:18 GMT], Expires=[0], Pragma=[no-cache], Server=[Apache-Coyote/1.1], X-Android-Received-Millis=[1439980939827], X-Android-Response-Source=[NETWORK 403], X-Android-Sent-Millis=[1439980939811], X-Content-Type-Options=[nosniff], X-Frame-Options=[DENY], X-XSS-Protection=[1; mode=block]}
And it says that don't have a CSRF valid token
Invalid CSRF Token 'null' was found on the request parameter '_csrf' or header 'X-XSRF-TOKEN'.
In my webservice, the tokens is configured to rename to XSRF-TOKEN because of angularJs.
SOLUTION
public static void getTokens() throws IOException{
URL url = new URL(urlBase);
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
con.connect();
cookieManager = new CookieManager();
List<String> cookieHeader = con.getHeaderFields().get("Set-Cookie");
if (cookieHeader != null) {
for (String cookie : cookieHeader) {
String[] tokens = TextUtils.split(cookie, "=");
if (tokens[0].equals("JSESSIONID"))
cookieManager.getCookieStore().add(null, HttpCookie.parse(cookie).get(0));
if (tokens[0].equals("XSRF-TOKEN")) {
String[] tokenValue = TextUtils.split(tokens[1],";");
xsrfTOKEN = tokenValue[0];
}
}
}
con.disconnect();
}
Then, attach it to HttpUrlConnection
con.setRequestProperty("X-XSRF-TOKEN", xsrfTOKEN);

As I have experienced, we would need to submit the token as a request header. Spring expects its name to be X-CSRF-TOKEN by default. But people using AngularJS normally alter it to X-XSRF-TOKEN in Spring Security configuration.
But looking at your code, I couldn't figure out if you are sending that header.
If it would help, here is a snippet from my one project (using RestAssured):
if (xsrfToken != null && !ctx.getRequestMethod().equals(Method.GET))
requestSpec.header("X-XSRF-TOKEN", xsrfToken);

Getting Bad Request only on Gingerbread phones and simulator

I am facing a weird problem. I've a piece of code that makes a call to a server. The code works fine when I use Kitkat emulator. However, the same code does not work on Gingerbread phones or emulator. I always get 400 Bad Request from the server. I checked on the server. The error is:
client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23):
Apache is not configured as virtual host.
here is the code I am using:
public void makeServerCall(String serverUrl, String postString) {
URL url = new URL(serverURL + ESS_AUTHENTICATE_URL);
connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setRequestMethod(HTTP_POST);
//forcing user agent to a well known user agent with a hope it will work :-)
connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:1.3.1)");
String base64Tenant = "Basic " + Base64.encodeToString( auth.getBytes(), Base64.DEFAULT);
connection.setRequestProperty(AUTHENTICATE_HEADER, base64Tenant);
connection.addRequestProperty("Cache-Control", "no-cache");
connection.addRequestProperty("Pragma", "no-cache");
connection.setUseCaches(false);
connection.setRequestProperty( "Content-Type", "application/x-www-form-urlencoded");
connection.setRequestProperty("Accept-Charset", "UTF-8");
System.setProperty("http.keepAlive", "false");
connection.setFixedLengthStreamingMode(postString.getBytes().length);
// Send request
PrintWriter out = new PrintWriter(connection.getOutputStream());
out.print(postString);
out.close();
int respCode = connection.getResponseCode();
Log.i(TAG, "Response code = " + respCode);
if (respCode != 200) {
is = connection.getErrorStream();
if (is != null) {
String response = getStringFromStream(is);
Log.d(TAG, "Error occured: " + response);
is.close();
}
}
else {
// process success Response
}
}
Here is the wireshark capture I got when I run the code on Kitkat simulator
POST /removed_path HTTP/1.1
Accept-Charset: UTF-8
User-Agent: Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:1.3.1)
Authorization: Basic ASVGQVVMVEASDFw==
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/x-www-form-urlencoded
Host: correcthost.myserver.com
Connection: Keep-Alive
Accept-Encoding: gzip
Content-Length: 49
this is the post stringHTTP/1.1 200 OK
Date: Tue, 04 Feb 2014 07:52:42 GMT
Here is the wireshark capture from Gingerbread
POST /removed_path HTTP/1.1
Accept-Charset: UTF-8
User-Agent: Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:1.3.1)
Authorization: Basic ASVGQVVMVEASDFw==
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/x-www-form-urlencoded
Content-Length: 49
Host: correcthost.myserver.com
Connection: Keep-Alive
Accept-Encoding: gzip
The only difference between the two wiresharks difference is the order of content-length.
I've access to server too. I am not sure, I can change anything there because it is working for recent versions of Android and iOS. Any ideas on what is going wrong?

After sending HTTP request to server sometimes I get status 302 (redirect)

I try to send POST request to some server with json parameters and I wait json response.
From Android it works fine but from Linux I get status 302 redirect. I really don't know where is my problem.
Here is command from linux:
curl -i -X POST --data "id=105&json={\"orderBy\":0\"maxResults\":50}" "http://mysite.com/ctlClient/"
And I get response:
HTTP/1.1 302 Found
Date: Thu, 04 Jul 2013 12:22:06 GMT
Server: Apache
X-Powered-By: PHP/5.3.19
Set-Cookie: PHPSESSID=dqblvijvttgsdrv2u5tn72p9d6; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
location: http://mysite.com/fwf/online/
Content-Length: 0
Connection: close
Content-Type: text/html
From Server access log:
"POST /ctlClient/ HTTP/1.1" 302 - "-" "curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5"
Android:
String data = "{\"orderBy\":0\"maxResults\":50}";
String WEFI_MAIL_URL = "http://mysite.com/ctlClient/";
DefaultHttpClient httpclient = null;
String out = null;
try {
httpclient = new DefaultHttpClient();
HttpPost httpost = new HttpPost(WEFI_MAIL_URL);
httpost.setHeader("User-Agent", "Apache-HttpClient/4.1 (java 1.5)");
List <NameValuePair> nvps = new ArrayList <NameValuePair>();
nvps.add(new BasicNameValuePair("id", "105"));
nvps.add(new BasicNameValuePair("json", data));
httpost.setEntity(new UrlEncodedFormEntity(nvps, HTTP.UTF_8));
HttpResponse response = httpclient.execute(httpost);
HttpEntity entity = response.getEntity();
if (entity != null) {
InputStream is = entity.getContent();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line = null;
while ( (line = br.readLine()) != null) {
out = line;
}
is.close();
if(isTimeOut == false){
_loadActual();
}
else{
return;
}
} else {
return;
}
} catch (Exception e) {
}
if (httpclient != null) {
httpclient.getConnectionManager().shutdown();
}
}
access log on Android response:
"POST /ctlClient/ HTTP/1.1" 302 - "-" "Apache-HttpClient/4.1 (java 1.5)"
"GET /fwf/online/ HTTP/1.1" 200 13981 "-" "Apache-HttpClient/4.1 (java 1.5)"
We can see that on POST server redirects me to /fwf/online/
From Wireshark i get the same results from both methods:
POST /ctlClient/ HTTP/1.1
User-Agent: Apache-HttpClient/4.1 (java 1.5)
Content-Type: application/x-www-form-urlencoded
Content-Length: 301
Host: mysite.com
Connection: Keep-Alive
Why it works from Android but not from Linux with CURL?
Can anybody help me or give me direction how can i solve it?
Thank you

You want to add -L to your curl command. See http://curl.haxx.se/docs/faq.html#How_do_I_tell_curl_to_follow_HTT.