I am writing a small app that retrieves some html from a web server based on some variables in the http POST. The HTML data that comes back has a <pre> section in it with some words that are spaced out nicely using newline and tab characters but my app does not receive them. The code is as follows
HttpPost post = new HttpPost("REMOVED FOR PRIVACY");
try {
// Add your data
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>();
//REMOVED FOR PRIVACY
post.setEntity(new UrlEncodedFormEntity(nameValuePairs));
// Execute HTTP Post Request
HttpResponse response = httpClient.execute(post, httpContext);
String htmlBrief = inputStreamToString(response.getEntity().getContent()).toString();
return htmlBrief;
}
I think it might be how I am reading the response by putting it through a BufferedReader like so
private StringBuilder inputStreamToString(InputStream is) throws IOException {
int c;
StringBuilder total = new StringBuilder();
// Wrap a BufferedReader around the InputStream
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
// Read response until the endIndex
while ((c = rd.read()) != -1) {
total.append((char)c);
}
// Return full string
return total;
}
I thought it might be because I was reading in line by line in the buffered reader so I switched to one character at a time but it didn't help the problem.
Any help would be greatly appreciated.
Thanks, Ben
Use Charles or Fiddler to inspect what is actually sent in the HTTP request body.
Most likely problems:
Mismatching character sets for client & server;
Failure to decode the URL encoded body.
Related
I'm developing an application in Android where I can insert new comments (like in facebook or any other social network).
The function that does it is the following :
public static String putNewComment(String comment, int userId, int trackId) throws IOException,JSONException {
HttpPost post = new HttpPost(Globals.PUBLISH_NEW_COMMENT);
StringEntity input = new StringEntity("{\"comment\": \""+comment+"\",\"trackId\":"+trackId+",\"userId\":"+userId+"}");
input.setContentType("application/json;charset=UTF-8");
post.setEntity(input);
input.setContentEncoding(new BasicHeader(HTTP.CONTENT_TYPE,"application/json;charset=UTF-8"));
post.setHeader("Accept", "application/json");
post.setEntity(input);
HttpResponse response = client.execute(post,Globals.localContext);
System.out.println("Response Code : "
+ response.getStatusLine().getStatusCode());
BufferedReader rd = new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
String resultMy="";
String line;
while ((line = rd.readLine()) != null) {
resultMy+=line;
System.out.println("Server says: "+line);
}
return resultMy;
}
Every time I try to put a new comment that contains unicode characters like é,ò, and several others I get this error:
com.fasterxml.jackson.databind.JsonMappingException: Invalid UTF-8 middle byte 0x22
Now, I am aware of the fact that the error is due to the fact that the content encoding is UTF-8 and I'm trying to insert an unicode character.
The problem is how to fix this? I tried by changing in this way :
input.setContentType("application/json;charset=UTF-16");
post.setEntity(input);
input.setContentEncoding(new BasicHeader(HTTP.CONTENT_TYPE,"application/json;charset=UTF-16"));
or
input.setContentType("application/json;charset=UTF-32");
post.setEntity(input);
input.setContentEncoding(new BasicHeader(HTTP.CONTENT_TYPE,"application/json;charset=UTF-32"));
But it doesn't work !
Please help me
I have made a HTTP-post inside my android application. Values are sent as strings from my app to my webserver. Problem is, the values are not in UTF-8 as I want them to be. My webserver has UTF-8 encoding so I know that there is code inside my app that I need to change.
See my snippet below:
private void sendPostRequest(String facebookId, String name, String email) {
class SendPostReqAsyncTask extends AsyncTask<String, Void, String>{
#Override
protected String doInBackground(String... bcs) {
String bcFacebookId = bcs[0];
String bcName = bcs[1];
String bcEmail = bcs[2];
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost("URL");
BasicNameValuePair facebookIdBasicNameValuePair = new BasicNameValuePair("bcFacebookId", bcFacebookId);
BasicNameValuePair nameBasicNameValuePair = new BasicNameValuePair("bcName", bcName);
BasicNameValuePair emailBasicNameValiePair = new BasicNameValuePair("bcEmail", bcEmail);
List<NameValuePair> nameValuePairList = new ArrayList<NameValuePair>();
nameValuePairList.add(facebookIdBasicNameValuePair);
nameValuePairList.add(nameBasicNameValuePair);
nameValuePairList.add(emailBasicNameValiePair);
try {
UrlEncodedFormEntity urlEncodedFormEntity = new UrlEncodedFormEntity(nameValuePairList);
httpPost.setEntity(urlEncodedFormEntity);
try {
HttpResponse httpResponse = httpClient.execute(httpPost);
InputStream inputStream = httpResponse.getEntity().getContent();
InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
StringBuilder stringBuilder = new StringBuilder();
String bufferedStrChunk = null;
while((bufferedStrChunk = bufferedReader.readLine()) != null){
stringBuilder.append(bufferedStrChunk);
}
return stringBuilder.toString();
} catch (ClientProtocolException cpe) {
cpe.printStackTrace();
} catch (IOException ioe) {
System.out.println("Second Exception caz of HttpResponse :" + ioe);
ioe.printStackTrace();
}
} catch (UnsupportedEncodingException uee) {
System.out.println("An Exception given because of UrlEncodedFormEntity argument :" + uee);
uee.printStackTrace();
}
return null;
}
For an example, the letter 'ö' becomes a '?'. How do I fix this?
Cheers!
The biggest single reason that characters get converted into question marks is the conversion of characters to bytes, and then back into characters, not matching.
The code you have supplied has this line:
InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
This is problematic because you are not specifying how to convert the bytes into characters. Instead you probably want this:
InputStreamReader inputStreamReader = new InputStreamReader(inputStream, "UTF-8");
What you specify for the character encoding will depend upon the character encoding that you have specified elsewhere. Without specifying the character encoding, you will get the "default" character encoding, and that depends upon settings in both the client and the server. Java uses Unicode, and UTF-8 is the only encoding that will preserve all the characters that Java allows.
For debugging, you may want to use the InputStream and retrieve bytes from that, and print out the byte values, in order to verify that they are indeed UTF-8 encoded representations of the original character values. The proper encoding of 'ö' (x00F6) is 'ö' (x00C3 x00B6).
You will also need to assure that the original POST request is properly UTF-8 encoded. The UrlEncodedFormEntity class also uses the default character encoding, which might not be UTF-8. Change this:
UrlEncodedFormEntity uefe = new UrlEncodedFormEntity(nameValuePairList);
to
UrlEncodedFormEntity uefe = new UrlEncodedFormEntity(nameValuePairList, "UTF-8");
if database coding is set properly + table coding is set properly + columns coding set properly, then all data are stored properly. That's the first part. Now the second, important part - make sure you have this command after your mysql connection : SET NAMES utf8
This was my case for the same issue. Hope this this will work for you as well.
I am sending an HttpPost request and getting the response in JSON format. But, as the response is so huge, I receive only a small part of the response compared to what I can see it in the browser. My code is below:
StringBuilder builder = new StringBuilder();
HttpPost httpPost = new HttpPost(uri);
HttpResponse response = client.execute(httpPost);
HttpEntity entity = response.getEntity();
InputStream content = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(content));
String line;
while ((line = reader.readLine()) != null) {
builder.append(line);
}
As the default buffer size is 8192 characters. I just tried with giving more value but it didn't matter. So, any suggestions on this...
I think that +Che Jami is onto something in the OP.
Manjunath, logcat will only output 1024 characters at a time. Have you tried outputting the String builder 1024 characters at a time? Did you check the length of the string after reading into it?
Send only chunks (batches) of your full response in your responses and while your response is not finished use polling.
Okay, so I was trying to send Http Post Requests to this one site, and I sniffed the sent request with wireshark thus getting the text data from the post request of this site. I used this in a stock Java application, and it worked perfectly fine. I could use the post method regularly with no problem whatsoever, and it would return the appropriate website. Then I tried doing this with Android. Instead of returning the actual html data after executing the post request, it returns the regular page html data untouched. It DOES send a post request (sniff with wireshark again), it just doesn't seem to get the appropriate response. I took the exact same method used from another one of my projects, which worked perfectly fine in that project, and pasted it into my new project. I added the INTERNET user permission in Android, so there's nothing wrong with that. The only visible difference is that I used NameValuePairs in the other one (the one that worked) and in this one I'm directly putting the string into a StringEntity without encoding (using UTF-8 encoding screws up the String though). I used this exact same line of text in regular Java like I said, and it worked fine with no encoding. So what could be the problem? This is the code:
public static String sendNamePostRequest(String urlString) {
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(urlString);
StringBuffer sb = new StringBuffer();
try {
post.setEntity(new StringEntity(
"__EVENTTARGET=&__EVENTARGUMENT=&__VIEWSTATE=%2FwEPDwULLTE3NDM5MzMwMzRkZA%3D%3D&__EVENTVALIDATION=%2FwEWBAL%2B%2B4CfBgK52%2BLYCQK1gpH7BAL0w%2FPHAQ%3D%3D&_nameTextBox=John&_zoekButton=Zoek&numberOfLettersField=3"));
HttpResponse response = client.execute(post);
HttpEntity entity = response.getEntity();
BufferedReader br = new BufferedReader(new InputStreamReader(
entity.getContent()));
String in = "";
while ((in = br.readLine()) != null) {
sb.append(in + "\n");
}
br.close();
} catch (Exception e) {
e.printStackTrace();
}
return sb.toString();
}
Can you see what's wrong here?
I'm getting back names of (Foursquare) venues from a server call where the names of the venues returned can be in English or non-English.
Assume the venue name is in a JSON object as follows:
{...
"name":"venue name which can be in any language"
...}
I'm creating a JSONObject from this response and then pulling out the name of the venue as follows:
String name = jsonObject.getString("name");
Lastly, I'm setting the TextView's text to show the name of the venue as follows:
myTextView.setText(name);
I'm finding however for Arabic names that where the Arabic characters are joined in the original JSON object (as they should be), the characters that show in the app (i.e. in the TextView) are disjoint. (I'm not too familiar with other languages so can't really tell if they're showing incorrectly too.)
Is there something additional I should be doing to pull out non-English names correctly from the JSON object and setting it as the text of a TextView, or is it down to the phone to decide how the text will be displayed?
Edit: I've tried parsing the server response (as suggested by #bbedward) explicitly specifying the content encoding as UTF-8 as follows...
HttpEntity httpEntity = httpResponse.getEntity();
String responseMessage = EntityUtils.toString(myHttpEntity, "UTF-8");
JSONObject jsonObject = new JSONObject(responseMessage);
... but still no joy. (Arabic characters appear, as before, disjoint in words where they should be joint up.) Could it be a phone thing or is there something extra needing to be done myself to get the words/characters to show proper in non-English languages? Perhaps the server needs to explicitly specify a "Content-Type" header with value "UTF-8"?
I'm going to answer anyway, I'm guessing you aren't getting your json in UTF-8 as i had a similar problem, I believe json won't come any other way.
Complete Example
The only things to concern yourself with this is setting the encoding for the InputStreamReader and creating the JSONObject
private DefaultHttpClient httpclient = new DefaultHttpClient(new BasicHttpParams());
HttpPost httppost = new HttpPost("http://myjsonurl.com/search?type=json");
// Depending on your web service
httppost.setHeader("Content-type", "application/json");
try
{
String result = null;
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity = response.getEntity();
InputStream inputStream = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream, "utf-8"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null)
{
sb.append(line + "\n");
}
JSONObject myJObject = new JSONObject(sb.toString();
}
catch (Exception e)
{
}
finally
{
try{if(inputStream != null)inputStream.close();}catch(Exception none){}
}
add this line when you connect to mysql:
mysql_set_charset('utf8', $con);
ex:
$con = mysql_connect(DB_HOST, DB_USER, DB_PASSWORD);
mysql_set_charset('utf8', $con);
mysql_select_db(DB_DATABASE);