I'm a novice with Java and Android, but not to programming and HTTP. This HTTP GET method, mostly copied from other examples using the Apache HTTP classes, only retrieves the first few K of a large webpage. I checked that the webpage does not have lines longer than 8192 bytes (is that possible?), but out of webpages around 40K I get back maybe 6K, maybe 20K. The number of bytes read does not seem to have a simple realtionship with the total webpage size, or the webpage modulus 8192, or with the webpage content.
Any ideas folks?
Thanks!
public static String myHttpGet(String url) throws Exception {
BufferedReader in = null;
try {
HttpClient client = getHttpClient();
HttpGet request = new HttpGet();
request.setURI(new URI(url));
HttpResponse response = client.execute(request);
in = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer sbuffer = new StringBuffer("");
String line = "";
while ((line = in.readLine()) != null) {
sbuffer.append(line + "\n");
}
in.close();
String result = sbuffer.toString();
return result;
} finally {
if (in != null) {
try {
in.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
No need to write you own HttpEntity-to-String code, try EntityUtils instead:
// this uses the charset the server encoded the entity in
String result = EntityUtils.toString(entity);
It looks as if the problem is with pages from a certain website starting Goo... I'm not having this problem with large pages from other sites. So the code is probably OK.
Related
I am using HttpClient 4.3.6 to perform http GET and POST requests. Right now I am using multipartentity to send a few string parameters and an image in the form of a file. I am able to successfully post the data but my problem comes in when I get the HTTP response. The response contains json data.
What happens is the HTTP response is incomplete and when i try to create a json object with the data i get jsonexception error saying:
Unterminated object at character 407.
I noticed that the response does not contain closed braces. Is this a problem on android or should I check the server? Because I am able to see the data properly on postman and on ios. I have never faced this issue before and don't know how to solve this.
This is my code to post and get the response:
#Override
protected String doInBackground(String... params) {
try {
String url = params[0];
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(url);
MultipartEntity entity = new MultipartEntity();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, baos);
byte[] imageBytes = baos.toByteArray();
ByteArrayBody bab = new ByteArrayBody(imageBytes, "image.jpg");
entity.addPart("image_data", bab);
entity.addPart("action", new StringBody("1", "text/plain", Charset.forName("UTF-8")));
entity.addPart("name", new StringBody("asdfg", "text/plain", Charset.forName("UTF-8")));
entity.addPart("user_id", new StringBody("157", "text/plain", Charset.forName("UTF-8")));
entity.addPart("birthday", new StringBody("18-04-1995", "text/plain", Charset.forName("UTF-8")));
entity.addPart("gender", new StringBody("male", "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_jlpt_student", new StringBody(String.valueOf(0), "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship", new StringBody("Father", "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship_id", new StringBody(String.valueOf(10002), "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_creator", new StringBody(String.valueOf(1), "text/plain", Charset.forName("UTF-8")));
entity.addPart("email", new StringBody(email, "text/plain", Charset.forName("UTF-8")));
httppost.setEntity(entity);
HttpResponse resp = httpclient.execute(httppost);
String response = EntityUtils.toString(resp.getEntity());
Log.i("HttpResponse", response);
return response;
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
#Override
protected void onPostExecute (String result) {
super.onPostExecute(result);
JSONObject jsonObject = null;
try {
jsonObject = new JSONObject(result);
JSONObject json_data = jsonObject.getJSONObject("data");
String json_userid = json_data.getString("user_id");
String json_username = json_data.getString("name");
String json_email = json_data.getString("email");
String json_country = json_data.getString("country_code");
String json_imagefilename = json_data.getString("image_filename");
String json_imgurl = json_data.getString("image_url");
Toast.makeText(ParentGuardianProfile.this, "ImageFile " + json_imagefilename, Toast.LENGTH_SHORT).show();
User new_user = userdao.createUser(json_userid, json_username, json_email,json_imagefilename,json_country,selectedImageUri.toString(), 1);
Log.i("SQLITE", "added user : " + new_user.getmUserName() + new_user.getmId());
} catch (JSONException e) {
e.printStackTrace();
}
}
And my json response is :
{"status":1,"message":"success","data":{"child_id":"381","name":"asdfg","image_filename":"C201603021734476.jpg","image_url":"https:\/\/innokid.blob.core.windows.net\/media\/child\/381.jpg","birthday":"18-04-1995","gender":"male","is_jltp_student":"0","relationship":"Father","relationship_id":"10002","is_creator":1,"rank":1,"qrcode_url":"http:\/\/innokid.azurewebsites.net\/uploads\/qrcode\/child_381.png"
I tried using String buffer as suggested in this post String is being truncated when its too long . But i still get the same result.
Code looks ok at first glance.
How do you got know that the json data is cut? Logcat can truncate text. Debugger should be more reliable in this case.
Try to generate this same request with some tools like curl / SoapUI and validate JSON you got with some formatter / validator (you'll easily find a few of such tools).
It's beyond the range of question, but using raw Android built-in communication libraries seems to be a little bit masochistic. Have you ever consider to use Retrofit?
I think this code is problematic String response = EntityUtils.toString(resp.getEntity());
may be you should use some other function to convert response toString...
Apparently the json is missing two curly brackets '}}' at the end, which can happen due to some bug in the toString code.
I pulled up an old project that was using the org.apache.http stuff and below is how I was parsing the response. As you can see it is rather cumbersome. There are many tested and maintained libraries out there that are better suited to this kind of heavy-lifting.
// Get hold of the response entity (-> the data):
HttpEntity entity = response.getEntity();
if (entity != null) {
// Read the content stream
InputStream instream = entity.getContent();
Header contentEncoding = response.getFirstHeader("Content-Encoding");
if (contentEncoding != null && contentEncoding.getValue().equalsIgnoreCase("gzip")) {
instream = new GZIPInputStream(instream);
}
// Convert content stream to a String
resultString = convertStreamToString(instream);
instream.close();
// Do stuff with resultString here
// Consume Content
entity.consumeContent();
}
And the convertStreamToString() method:
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the
* BufferedReader.readLine() method. We iterate until the BufferedReader
* return null which means there's no more data to read. Each line will
* appended to a StringBuilder and returned as String.
*
* (c) public domain:
* http://senior.ceng.metu.edu.tr/2009/praeda/2009/01/
* 11/a-simple-restful-client-at-android/
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is), 8192);
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
I finally solved this issue by replacing httpclient library with Android Asynchronous Http Client. Now it works fine. Thanks a lot for your help!
However, I still dont understand why the response was truncated when i used httpclient.
Require your Help for fetching the json response from the httpclient in Android since the following code mentioned is making the Application to crash on android Devices particularly on the GingerBread device , since the JSON Response is very huge in size (may be 7 MB) .
So I wanted to know any alternative way for reading the JSONresponse from the Httpclient, since the present implementation is consuming too much of memory and making my application to crash on lower end devices.
It would be very greatful for any suggestions or help for solving this problem .
HttpClient httpClient = new DefaultHttpClient(ccm, params);
HttpGet httpGet = new HttpGet(url);
httpGet.setHeader("Cache-Control", "no-cache; no-store");
HttpResponse httpResponse = httpClient.execute(httpGet);
response = Utils.convertStreamToString(httpResponse.getEntity().getContent());
public static String convertStreamToString(InputStream is)
{
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
//System.gc();
sb.append(line).append("\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
You can use Google Volley for you networking. Among a lot of other things, it has a built in method to retrieve JSON objects, regardless of size.
Give it a try.
You could try this:
public static String convertStreamToString(InputStream is)
{
try
{
final char[] buffer = new char[0x10000];
StringBuilder out = new StringBuilder();
Reader in = new InputStreamReader(is, "UTF-8");
int read;
do
{
read = in.read(buffer, 0, buffer.length);
if (read > 0)
{
out.append(buffer, 0, read);
}
} while (read >= 0);
in.close();
return out.toString();
} catch (IOException ioe)
{
throw new IllegalStateException("Error while reading response body", ioe);
}
}
Google Android ships with an extremely outdated fork of Apache HttpClient. However, the base principles still apply. The most efficient way of processing HTTP responses with Apache HttpClient is by using a ResponseHandler. See my answer to a similar question for details
I make a GET request to a server using HttpUrlConnection.
After connecting:
I get response code: 200
I get response message: OK
I get input stream, no exception thrown but:
in a standalone program I get the body of the response, as expected:
{"name":"my name","birthday":"01/01/1970","id":"100002215110084"}
in a android activity, the stream is empty (available() == 0), and thus I can't get
any text out.
Any hint or trail to follow? Thanks.
EDIT: here it is the code
Please note: I use import java.net.HttpURLConnection; This is the standard
http Java library. I don't want to use any other external library. In fact
I did have problems in android using the library httpclient from apache (some of their anonymous .class can't be used by the apk compiler).
Well, the code:
URLConnection theConnection;
theConnection = new URL("www.example.com?query=value").openConnection();
theConnection.setRequestProperty("Accept-Charset", "UTF-8");
HttpURLConnection httpConn = (HttpURLConnection) theConnection;
int responseCode = httpConn.getResponseCode();
String responseMessage = httpConn.getResponseMessage();
InputStream is = null;
if (responseCode >= 400) {
is = httpConn.getErrorStream();
} else {
is = httpConn.getInputStream();
}
String resp = responseCode + "\n" + responseMessage + "\n>" + Util.streamToString(is) + "<\n";
return resp;
I see:
200
OK
the body of the response
but only
200
OK
in android
Trying the code of Tomislav I've got the answer.
My function streamToString() used .available() to sense if there is any data received,
and it returns 0 in Android. Surely, I called it too soon.
If I rather use readLine():
class Util {
public static String streamToString(InputStream is) throws IOException {
StringBuilder sb = new StringBuilder();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
String line;
while ((line = rd.readLine()) != null) {
sb.append(line);
}
return sb.toString();
}
}
then, it waits for the data to arrive.
Thanks.
You can try with this code that will return response in String:
public String ReadHttpResponse(String url){
StringBuilder sb= new StringBuilder();
HttpClient client= new DefaultHttpClient();
HttpGet httpget = new HttpGet(url);
try {
HttpResponse response = client.execute(httpget);
StatusLine sl = response.getStatusLine();
int sc = sl.getStatusCode();
if (sc==200)
{
HttpEntity ent = response.getEntity();
InputStream inpst = ent.getContent();
BufferedReader rd= new BufferedReader(new InputStreamReader(inpst));
String line;
while ((line=rd.readLine())!=null)
{
sb.append(line);
}
}
else
{
Log.e("log_tag","I didn't get the response!");
}
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return sb.toString();
}
The Stream data may not be ready, so you should check in a loop that the data in the stream is available before attempting to access it.
Once the data is ready, you should read it and store in another place like a byte array; a binary stream object is a nice choice to read data as a byte array. The reason that a byte array is a better choice is because the data may be binary data like an image file, etc.
InputStream is = httpConnection.getInputStream();
byte[] bytes = null;
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] temp = new byte[is.available()];
while (is.read(temp, 0, temp.length) != -1) {
baos.write(temp);
temp = new byte[is.available()];
}
bytes = baos.toByteArray();
In the above code, bytes is the response as byte array. You can convert it to string if it is text data, for example data as utf-8 encoded text:
String text = new String(bytes, Charset.forName("utf-8"));
I wish to settle my long term problem by this question and hope you guys would help, but firstly; I have been having issues to connect to a HTTPS self-signed certificate server for almost 3 weeks. Despite the multiple solutions here, I cannot seem to resolve my problem. Probably I did not know how to use it properly or did not have some files or imported the correct libraries.
I came across some websites that requires me to download a certificate from the https site that I am trying to connect into, and when I did that. I have to do the some steps before I can use the certificate or keystore that I created. I got this solution from this website:
Android: Trusting SSL certificates
// Instantiate the custom HttpClient
DefaultHttpClient client = new MyHttpClient(getApplicationContext());
HttpGet get = new HttpGet("https://www.mydomain.ch/rest/contacts/23");
// Execute the GET call and obtain the response
HttpResponse getResponse = client.execute(get);
HttpEntity responseEntity = getResponse.getEntity();
I have a problem, after the last line, as stated above. What do I do with the responseEntity? How do I use it if I wish to display the https website on a WebView? Some help and explanation would be nice :)
If you want the content from the HttpEntity the correct way does not include calling HttpEntity#getContent() to retrieve a stream and doing tons of pointless stuff already available in the Android SDK.
Try this instead.
// Execute the GET call and obtain the response
HttpResponse getResponse = client.execute(get);
HttpEntity responseEntity = getResponse.getEntity();
// Retrieve a String from the response entity
String content = EntityUtils.toString(responseEntity);
// Now content will contain whatever the server responded with and you
// can pass it to your WebView using #loadDataWithBaseURL
Consider using WebView#loadDataWithBaseURL when displaying content - it behaves a lot nicer.
You need to call responseEntity.getContent() to get response in InputStream against your requested URL. Use that stream in your way to present data as you want. For example, if the expected data is String, so you may simply convert this stream into string with the following method:
/**
* Converts InputStream to String and closes the stream afterwards
* #param is Stream which needs to be converted to string
* #return String value out form stream or NULL if stream is null or invalid.
* Finally the stream is closed too.
*/
public static String streamToString(InputStream is) {
try {
StringBuilder sb = new StringBuilder();
BufferedReader tmp = new BufferedReader(new InputStreamReader(is),65728);
String line = null;
while ((line = tmp.readLine()) != null) {
sb.append(line);
}
//close stream
is.close();
return sb.toString();
}
catch (IOException e) { e.printStackTrace(); }
catch (Exception e) { e.printStackTrace(); }
return null;
}
InputStream is = responseEntity.getContent();
try{
BufferedReader reader = new BufferedReader(new InputStreamReader(is,"iso-8859-1"),8);
StringBuilder sb = new StringBuilder();
sb.append(reader.readLine() + "\n");
String line="0";
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
String result=sb.toString();
is.close();
}catch(Exception e){
Log.e("log_tag", "Error converting result "+e.toString());
}
you will have all the content in the String "result"
i have this high score board in http://drymvizion.atwebpages.com/ and i want a simple way to click a button and show this content in my android application without opening browser. I am a bit new in android developing and i hope it's easy. Thanks ;)
You could use a HttpGet request to get the contents, or better use a xml parser, such as JSoup. For example:
HttpClient client = new DefaultHttpClient();
HttpGet get = new HttpGet("site.name");
try
{
get.addHeader("Accept-Charset","utf-8");
HttpResponse response = client.execute(get);
BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
String result = "";
while ((result = rd.readLine()) != null)
{
Log.e("HttpResponse", result);
if (result.length() > 0)
{
// do something with the source
}
}
}
catch (IOException e)
{
e.printStackTrace();
}
Have a look at WebView's
http://developer.android.com/reference/android/webkit/WebView.html