Android client, using Springs resttemplace and the Apache common HTTP client to make requests.
I'm working against a server, that sometimes returns a 401 error, with a http header string, "ERROR" that contains a user-info string. The string is language dependent, so it might contain, for example, Scandinavian characters.
The string looks fine in my IOS app, aswell as when i examine it in the Firefox RESTclient plugin.
However, in my Android app, i cannot for the life of me get the chars to come out right. I'd very much appreciate it if someone could think of a way i can make the data come out right.
The server sends content-type UTF-8, and its a regular .setHeader() on the httpservletresponse that sets the parameter i try to retrieve.
Here's the creation of my resttemplate in my Android client (ive tried most methods as you can see):
HttpClient httpClient = new HttpClient();
Credentials defaultcreds = new UsernamePasswordCredentials(msisdn, password);
httpClient.getState().setCredentials(new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, AuthScope.ANY_REALM), defaultcreds);
httpClient.getParams().setSoTimeout(prefs.getServerTimeout());
httpClient.getParams().setConnectionManagerTimeout(3000);
httpClient.getParams().setContentCharset("utf-8");
httpClient.getParams().setCredentialCharset(prefs.getCredentialsEncoding());
httpClient.getParams().setHttpElementCharset("utf-8");
httpClient.getParams().setUriCharset("utf-8");
CommonsClientHttpRequestFactory requestFactory = new CommonsClientHttpRequestFactory(httpClient);
requestFactory.setReadTimeout(prefs.getServerTimeout());
RestTemplate restTemplate = new RestTemplate(requestFactory);
// Add message converters
List<HttpMessageConverter<?>> mc = restTemplate.getMessageConverters();
MappingJacksonHttpMessageConverter json = new MappingJacksonHttpMessageConverter();
List<MediaType> supportedMediaTypes = new ArrayList<MediaType>();
supportedMediaTypes.add(MediaType.APPLICATION_JSON);
json.setSupportedMediaTypes(supportedMediaTypes);
mc.add(json);
restTemplate.setMessageConverters(mc);
// Set our specific error handler
restTemplate.setErrorHandler(new MyErrorHandler());
this is my http response, copied from restclient in Firefox if i run the same request there:
Status Code: 401 Unauthorized
Content-Length: 954
Content-Type: text/html;charset=utf-8
Date: Sat, 19 Jan 2013 23:53:10 GMT
ERROR: För att <contents cut out but as you see Scandinavian char looks fine here >
Server: Apache-Coyote/1.1
WWW-Authenticate: Basic realm="rest"
HTTP headers use ASCII (or, in the older specs, Latin-1). Putting other encodings into headers might or might not work.
Your Response Content-Type appears to be text/html. Did you try setting Content-Type and charset in the request header like below? As per documentation here
Documents transmitted with HTTP that are of type text, such as text/html, text/plain, etc., can send a charset parameter in the HTTP header to specify the character encoding of the document.
LinkedMultiValueMap<String, String> headers = new LinkedMultiValueMap<String, String>();
headers.set("Content-Type", "text/html;charset=utf-8");
HttpEntity request = new HttpEntity(null, headers);
ResponseEntity<Person> response = template.exchange(url,
HttpMethod.GET, request, Person.class);
Related
When posting the same basic JSON from Android or via curling, the result is different.
I'm trying to understand why? I'm assuming there is something about Rails that I don't understand.
Command Line HTTP POST
curl 'http://localhost:3000/mobile/register' -X POST -H 'Content-Type:
application/json' -d '{"user": {"email": "awesome#example.com",
"password":"helloworld", "password_confirmation":"helloworld"}}'
Server logs
Parameters: {"user"=>{"email"=>"awesome#example.com",
"password"=>"[FILTERED]", "password_confirmation"=>"[FILTERED]"},
"registration"=>{"user"=>{"email"=>"awesome#example.com",
"password"=>"[FILTERED]", "password_confirmation"=>"[FILTERED]"}}}
Android HTTP POST
OkHttpClient client = new OkHttpClient();
JSONObject userObject = new JSONObject();
userObject.put("email", mUserEmail);
userObject.put("password", mUserPassword);
userObject.put("password_confirmation", mUserPasswordConfirmation);
String userString = userObject.toString();
RequestBody requestBody = new MultipartBody.Builder()
.setType(MultipartBody.FORM)
.addFormDataPart("user", userString)
.build();
Request request = new Request.Builder()
.url(REGISTER_API_ENDPOINT_URL)
.method("POST", RequestBody.create(null, new byte[0]))
.post(requestBody)
.build();
Call call = client.newCall(request);
call.enqueue(new Callback()...
Server logs
Parameters:
{"user"=>"{\"email\":\"awesome#example.com\",\"password\":\"helloworld\",\"password_confirmation\":\"helloworld\"}"}
Rails Routes:
devise_scope :user do
namespace :mobile do
post '/register', to: 'registrations#create', as: :register
end
end
Differences:
curling hides the password but Android does not
curling adds an additional nested JSON with all the data repeated
I'm not sure what causes these differences?
UPDATE
I checked request.env["CONTENT_TYPE"] in the controller action and I am seeing differences in Content-Type.
Curl -> application/json
Android -> multipart/form-data; boundary=...
Could this cause the issue?
Is it easy to change from the Android side? I added .header("Content-Type", "application/json") to the request but it makes no difference?
I think the Android HTTP POST lacks the Content-Type: application/json header because it sends multipart form data . So the rails app logs it as plain string data instead of parsing it, registering user and filtering password.
Also, in case of curl command, the parsed JSON user object is used to register a user. The repeated log entry is perhaps done during this user registration.
To make both requests equivalent, try using the POST TO A SERVER example given at http://square.github.io/okhttp/.
Http request header:
Host: www.mysite.com
Content-Type: application/x-www-form-urlencoded
Cookie: bbuserid=XXX; bbpassword=YYY; bbsessionhash=ZZZ
Content-Length: 252
Http request body:
message=%E4%F6%F5%FC%E4%F6%F5%FC%E4%F6%F5%FC%E4%F6%F5%FC&securitytoken=XXX&do=postreply&t=483553
Working fine! Data posted to server gets decoded on the other end and user sees orginal message which is äöõüäöõüäöõüäöõü.
Now lets try to implement this excact example with JSoup:
//request body
Map<String, String> datamap = new HashMap<String, String>();
datamap.put(Session.SESSION_SECURITYTOKEN,"XXX");
datamap.put("message", URLEncoder.encode(finalText, "ISO-8859-1"));
datamap.put("do", "postreply");
datamap.put("t", "483553");
//make a post
Jsoup.connect(url)
.header("Content-Type","application/x-www-form-urlencoded")
.timeout(10000)
.cookie(Session.COOKIE_HASH_KEY,session.bbsessionhash)
.cookie(Session.COOKIE_PASSWORD_KEY,session.bbpassword)
.cookie(Session.COOKIE_USERID_KEY,session.bbuserid)
.data(datamap).post();
My message gets posted BUT it is not decoded by the server. So when user views the message he/she sees: %E4%F6%F5%FC%E4%F6%F5%FC%E4%F6%F5%FC%E4%F6%F5%FC
Note: I am doing the post request from Android and posting data to vBulletin forum software (replay to thread).
The problem: When I send the message with JSoup, server sees it like a plain text not a encoded text. How can I make the server to understand that the message parameter holds encoded text, not plain text?
Jsoup uses UTF-8 by default to URL-encode the query string. With the current API version, you cannot change it without rebuilding the source (it's the org.jsoup.helper.DataUtil#defaultCharset constant which is been used in org.jsoup.helper.HttpConnection class). Best what you can do is to post an issue report requesting the ability to preset the charset beforehand.
Until then, you could use HttpClient or URLConnection instead which allows for a more finer grained control over sending HTTP requests. You could finally feed its response as an InputStream to Jsoup#parse() method.
Update: if the target website supports it, you could try explicitly specifying the client's used charset in the Content-Type request header:
.header("Content-Type","application/x-www-form-urlencoded;charset=UTF-8")
Note that you should not use URLEncoder#encode() yourself; let Jsoup do its job.
I want to access a Restful web service. I want the request should be in the following format.
GET /API/Contacts/username HTTP/1.1
HOST: $baseuri:port
Accept: text/xml
Authorization: Basic ZmF0aWdhYmxlIGdlbmVyYXR=
And also I am calling the web service though HTTPS protocol.
The folowing is the code I am using :
HttpGet get = new HttpGet("https://secure.myapp.com/MyApp/API/Contacts/myname");
get.addHeader("Accept","text/xml");
get.addHeader("Authorization","Basic ZmF0aWdhYmxlIGdlbmVyYXR=");
get.addHeader("Host","https://secure.myapp.com");
get.addHeader("Connection Use","HTTP 1.1");
DefaultHttpClient client = new DefaultHttpClient();
ResponseHandler objHandler = new BasicResponseHandler();
String getResponse = client.execute(get,objHandler);
But I am getting an Error : 400 Bad request.
I am not sure whether my code is correct. Is it necessary to specify the method (GET, POST or PUT) explicitly in the header?
Please help me...
Thnking You....
Shouldn't you put 'Basic ZmF0aWdhYmxlIGdlbmVyYXR=' in double quotes?
Also check web service example request and response and make sure that everything is specified.
Ohh...
That was my mistake.
I did not keep the order of adding headers.
When I changed its order according to the Request it is working fine.
In the request the HOST should be the first parameter.
The following is the corrected code.
get.addHeader("Host","secure.myapp.com");
get.addHeader("Accept","text/xml");
get.addHeader("Authorization","Basic ZmF0aWdhYmxlIGdlbmVyYXR=");
get.addHeader("Connection Use","HTTP 1.1");
Sorry to disturb you all...
On Android phone, I used setEntity() to put the FileEntity to the POST request.
HttpPost post = new HttpPost(uri);
FileEntity reqEntity = new FileEntity(f, "application/x-gzip");
reqEntity.setContentType("binary/octet-stream");
reqEntity.setChunked(true);
post.addHeader("X-AethersNotebook-Custom", configuration.getCustomHeader());
post.setEntity(reqEntity);
When using bottle, tried this but it is not working
f = request.body
gzipper = gzip.GzipFile( fileobj= f )
content = gzipper.read()
The content will be an empty string. So I tried to look at request.forms and request.files. Both of them have no key and value.
request.files.keys()
request.forms.keys()
When searching, I read about the entity: "a request MAY transfer entity" and the entity has entity-header and entity-value. So it may be something like file-content = e.get(entity-header).
Using that code, the phone send file using chunked encoding. Because py-bottle does not support chunked enconding, the solution here is rewrite the android to send file as body of POST request.
I have configured the apache httpClient like so:
HttpProtocolParams.setContentCharset(httpParameters, "UTF-8");
HttpProtocolParams.setHttpElementCharset(httpParameters, "UTF-8");
I also include the http header "Content-Type: application/json; charset=UTF-8" for all http post and put requests.
I am trying to send http post/put requests with a json body that contains special characters (ie. chinese characters via the Google Pinyin keyboard, symbols, etc.) The characters appear as gibberish in the logs but I think this is because DDMS does not support UTF-8, as descibed in this issue.
The problem is when the server receives the request, it sometimes doesn't see the characters at all (especially the Chinese characters), or it becomes meaningless garbage when we retrieve it through a GET request.
I also tried putting 250 non-ascii characters in a single field because that particular field should be able to take up to 250 characters. However, it fails to validate at the server side which claims that the 250 character limit has been exceeded. 250 ASCII characters work just fine.
The server dudes claim that they support UTF-8. They even tried simulating a post request that contains Chinese characters, and the data was received by the server just fine. However, the guy (a Chinese guy) is using a Windows computer with the Chinese language pack installed (I think, because he can type Chinese characters on his keyboard).
I'm guessing that the charsets being used by the Android client and the server (made by Chinese guys btw) are not aligned. But I do not know which one is at fault since the server dudes claim that they support UTF-8, and our rest client is configured to support UTF-8.
This got me wondering on what charset Android uses by default on all text input, and if it can be changed to a different one programatically. I tried to find resources on how to do this on input widgets but I did not find anything useful.
Is there a way to set the charset for all input widgets in Android? Or maybe I missed something in the rest client configuration? Or maybe, just maybe, the server dudes are not using UTF-8 at their servers and used Windows charsets instead?
Apparently, I forgot to set the StringEntity's charset to UTF-8. These lines did the trick:
httpPut.setEntity(new StringEntity(body, HTTP.UTF_8));
httpPost.setEntity(new StringEntity(body, HTTP.UTF_8));
So, there are at least two levels to set the charset in the Android client when sending an http post with non-ascii characters.
The rest client itself itself
The StringEntity
UPDATE: As Samuel pointed out in the comments, the modern way to do it is to use a ContentType, like so:
final StringEntity se = new StringEntity(body, ContentType.APPLICATION_JSON);
httpPut.setEntity(se);
I know this post is a bit old but nevertheless here is a solution:
Here is my code for posting UTF-8 strings (it doesn't matter if they are xml soap or json) to a server. I tried it with cyrillic, hash values and some other special characters and it works like a charm. It is a compilation of many solutions I found through the forums.
HttpParams httpParameters = new BasicHttpParams();
HttpProtocolParams.setContentCharset(httpParameters, HTTP.UTF_8);
HttpProtocolParams.setHttpElementCharset(httpParameters, HTTP.UTF_8);
HttpClient client = new DefaultHttpClient(httpParameters);
client.getParams().setParameter("http.protocol.version", HttpVersion.HTTP_1_1);
client.getParams().setParameter("http.socket.timeout", new Integer(2000));
client.getParams().setParameter("http.protocol.content-charset", HTTP.UTF_8);
httpParameters.setBooleanParameter("http.protocol.expect-continue", false);
HttpPost request = new HttpPost("http://www.server.com/some_script.php?sid=" + String.valueOf(Math.random()));
request.getParams().setParameter("http.socket.timeout", new Integer(5000));
List<NameValuePair> postParameters = new ArrayList<NameValuePair>();
// you get this later in php with $_POST['value_name']
postParameters.add(new BasicNameValuePair("value_name", "value_val"));
UrlEncodedFormEntity formEntity = new UrlEncodedFormEntity(postParameters, HTTP.UTF_8);
request.setEntity(formEntity);
HttpResponse response = client.execute(request);
in = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer sb = new StringBuffer("");
String line = "";
String lineSeparator = System.getProperty("line.separator");
while ((line = in.readLine()) != null) {
sb.append(line);
sb.append(lineSeparator);
}
in.close();
String result = sb.toString();
I hope that someone will find this code helpful. :)
You should set charset of your string entity to UTF-8:
StringEntity stringEntity = new StringEntity(urlParameters, HTTP.UTF_8);
You can eliminate the server as the problem by using curl to send the same data.
If it works with curl use --trace to check the output.
Ensure you are sending the content body as bytes. Compare the HTTP request from Android with the output from the successful curl request.