Develop Reference

Can't read from buffered reader in android

I am trying to get data by using API. I get data in bufferedreader which contains multiple lines. But I am not able to extract that in a String. I do know that bufferedreader has data, i have checked by displaying it using multiple .readLine(); but the loop doesn't work.
protected String[][] doInBackground(Cursor[] cursors)
{
String list[][] = new String[cursors[0].getCount()][2];
//while(cursors[0].moveToNext())
{
String DATA="",temp;
String baseAddress="http://www.alphavantage.co/query?function=TIME_SERIES_INTRADAY";
String apiKey="J63P";
Uri Url = Uri.parse(baseAddress)
.buildUpon()
.appendQueryParameter("symbol","SBIN")
.appendQueryParameter("interval","1min")
.appendQueryParameter("apikey",apiKey)
.build();
Log.d("built URL",Url.toString());
try
{
url= new URL(Url.toString());
urlconnection= (HttpURLConnection) url.openConnection();
urlconnection.setRequestMethod("GET");
urlconnection.connect();
InputStream inputStream = urlconnection.getInputStream();
if (inputStream==null)
{
Log.d("inputstream","empty");
return null;
}
BufferedReader reader= new BufferedReader(new InputStreamReader(inputStream));
while((temp = reader.readLine())!=null)
{
DATA.concat(temp);
}
Log.d("unedited",DATA);
}
catch(IOException e)
{
Log.e("createList", "Error in url ", e);
return null;
}
try {
String List = "Time Series (1min)";
JSONObject full = new JSONObject(DATA);
JSONArray permin = full.getJSONArray(List);
for (int i=0;i<permin.length();i++)
{
String open,high,low,close;
JSONObject current = permin.getJSONObject(i);
open = current.getString("1. open");
high = current.getString("2. high");
low = current.getString("3. low");
close = current.getString("4. close");
Log.d("Extracted ",open + " " + high + " " + low + " " + close +"\n");
}
}
catch (JSONException e)
{
Log.e("createList","Error in json",e);
}
}
return list;
}
and Log is:
04-24 14:34:52.009 3574-3619/lcukerd.com.stocknotifier D/built URL: http://www.alphavantage.co/query?function=TIME_SERIES_INTRADAY&symbol=SBIN&interval=1min&apikey=J63P
04-24 14:34:53.723 3574-3619/lcukerd.com.stocknotifier E/createList: Error in json
org.json.JSONException: End of input at character 0 of
at org.json.JSONTokener.syntaxError(JSONTokener.java:450)
at org.json.JSONTokener.nextValue(JSONTokener.java:97)
at org.json.JSONObject.<init>(JSONObject.java:156)
at org.json.JSONObject.<init>(JSONObject.java:173)
at lcukerd.com.stocknotifier.MainActivity$createList$override.doInBackground(MainActivity.java:123)
at lcukerd.com.stocknotifier.MainActivity$createList$override.access$dispatch(MainActivity.java)
at lcukerd.com.stocknotifier.MainActivity$createList.doInBackground(MainActivity.java:0)
at lcukerd.com.stocknotifier.MainActivity$createList.doInBackground(MainActivity.java:69)
at android.os.AsyncTask$2.call(AsyncTask.java:288)
at java.util.concurrent.FutureTask.run(FutureTask.java:237)
at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:231)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
at java.lang.Thread.run(Thread.java:818)
Also, note that "Log" statement after inner while loop is also not working.
Pls, Help me Solve it.

You got exception in the Json data, it is very clear from the log, it says:
04-24 14:34:53.723 3574-3619/lcukerd.com.stocknotifier E/createList:
Error in json
Oh yeah:
Change this line:
DATA.concat(temp);
To
data = DATA.concat(temp);
Also in java it's better to write it this way:
data += temp;

Android Http Response is incomplete. Returns an unterminated json object

I am using HttpClient 4.3.6 to perform http GET and POST requests. Right now I am using multipartentity to send a few string parameters and an image in the form of a file. I am able to successfully post the data but my problem comes in when I get the HTTP response. The response contains json data.
What happens is the HTTP response is incomplete and when i try to create a json object with the data i get jsonexception error saying:
Unterminated object at character 407.
I noticed that the response does not contain closed braces. Is this a problem on android or should I check the server? Because I am able to see the data properly on postman and on ios. I have never faced this issue before and don't know how to solve this.
This is my code to post and get the response:
#Override
protected String doInBackground(String... params) {
try {
String url = params[0];
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost(url);
MultipartEntity entity = new MultipartEntity();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, baos);
byte[] imageBytes = baos.toByteArray();
ByteArrayBody bab = new ByteArrayBody(imageBytes, "image.jpg");
entity.addPart("image_data", bab);
entity.addPart("action", new StringBody("1", "text/plain", Charset.forName("UTF-8")));
entity.addPart("name", new StringBody("asdfg", "text/plain", Charset.forName("UTF-8")));
entity.addPart("user_id", new StringBody("157", "text/plain", Charset.forName("UTF-8")));
entity.addPart("birthday", new StringBody("18-04-1995", "text/plain", Charset.forName("UTF-8")));
entity.addPart("gender", new StringBody("male", "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_jlpt_student", new StringBody(String.valueOf(0), "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship", new StringBody("Father", "text/plain", Charset.forName("UTF-8")));
entity.addPart("relationship_id", new StringBody(String.valueOf(10002), "text/plain", Charset.forName("UTF-8")));
entity.addPart("is_creator", new StringBody(String.valueOf(1), "text/plain", Charset.forName("UTF-8")));
entity.addPart("email", new StringBody(email, "text/plain", Charset.forName("UTF-8")));
httppost.setEntity(entity);
HttpResponse resp = httpclient.execute(httppost);
String response = EntityUtils.toString(resp.getEntity());
Log.i("HttpResponse", response);
return response;
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
#Override
protected void onPostExecute (String result) {
super.onPostExecute(result);
JSONObject jsonObject = null;
try {
jsonObject = new JSONObject(result);
JSONObject json_data = jsonObject.getJSONObject("data");
String json_userid = json_data.getString("user_id");
String json_username = json_data.getString("name");
String json_email = json_data.getString("email");
String json_country = json_data.getString("country_code");
String json_imagefilename = json_data.getString("image_filename");
String json_imgurl = json_data.getString("image_url");
Toast.makeText(ParentGuardianProfile.this, "ImageFile " + json_imagefilename, Toast.LENGTH_SHORT).show();
User new_user = userdao.createUser(json_userid, json_username, json_email,json_imagefilename,json_country,selectedImageUri.toString(), 1);
Log.i("SQLITE", "added user : " + new_user.getmUserName() + new_user.getmId());
} catch (JSONException e) {
e.printStackTrace();
}
}
And my json response is :
{"status":1,"message":"success","data":{"child_id":"381","name":"asdfg","image_filename":"C201603021734476.jpg","image_url":"https:\/\/innokid.blob.core.windows.net\/media\/child\/381.jpg","birthday":"18-04-1995","gender":"male","is_jltp_student":"0","relationship":"Father","relationship_id":"10002","is_creator":1,"rank":1,"qrcode_url":"http:\/\/innokid.azurewebsites.net\/uploads\/qrcode\/child_381.png"
I tried using String buffer as suggested in this post String is being truncated when its too long . But i still get the same result.

Code looks ok at first glance.
How do you got know that the json data is cut? Logcat can truncate text. Debugger should be more reliable in this case.
Try to generate this same request with some tools like curl / SoapUI and validate JSON you got with some formatter / validator (you'll easily find a few of such tools).
It's beyond the range of question, but using raw Android built-in communication libraries seems to be a little bit masochistic. Have you ever consider to use Retrofit?

I think this code is problematic String response = EntityUtils.toString(resp.getEntity());
may be you should use some other function to convert response toString...
Apparently the json is missing two curly brackets '}}' at the end, which can happen due to some bug in the toString code.

I pulled up an old project that was using the org.apache.http stuff and below is how I was parsing the response. As you can see it is rather cumbersome. There are many tested and maintained libraries out there that are better suited to this kind of heavy-lifting.
// Get hold of the response entity (-> the data):
HttpEntity entity = response.getEntity();
if (entity != null) {
// Read the content stream
InputStream instream = entity.getContent();
Header contentEncoding = response.getFirstHeader("Content-Encoding");
if (contentEncoding != null && contentEncoding.getValue().equalsIgnoreCase("gzip")) {
instream = new GZIPInputStream(instream);
}
// Convert content stream to a String
resultString = convertStreamToString(instream);
instream.close();
// Do stuff with resultString here
// Consume Content
entity.consumeContent();
}
And the convertStreamToString() method:
private static String convertStreamToString(InputStream is) {
/*
* To convert the InputStream to String we use the
* BufferedReader.readLine() method. We iterate until the BufferedReader
* return null which means there's no more data to read. Each line will
* appended to a StringBuilder and returned as String.
*
* (c) public domain:
* http://senior.ceng.metu.edu.tr/2009/praeda/2009/01/
* 11/a-simple-restful-client-at-android/
*/
BufferedReader reader = new BufferedReader(new InputStreamReader(is), 8192);
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}

I finally solved this issue by replacing httpclient library with Android Asynchronous Http Client. Now it works fine. Thanks a lot for your help!
However, I still dont understand why the response was truncated when i used httpclient.

Converting String to Android JSONObject loses utf-8

I am trying to get a (JSON formatted) String from a URL and consume it as a Json object. I lose UTF-8 encoding when I convert the String to JSONObject.
This is The function I use to connect to the url and get the string:
private static String getUrlContents(String theUrl) {
StringBuilder content = new StringBuilder();
try {
URL url = new URL(theUrl);
URLConnection urlConnection = url.openConnection();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
String line;
while ((line = bufferedReader.readLine()) != null) {
content.append(line + "\n");
}
bufferedReader.close();
} catch(Exception e) {
e.printStackTrace();
}
return content.toString();
}
When I get data from server, the following code displays correct characters:
String output = getUrlContents(url);
Log.i("message1", output);
But when I convert the output string to JSONObject the Persian characters becomes question marks like this ??????. (messages is the name of array in JSON)
JSONObject reader = new JSONObject(output);
String messages = new String(reader.getString("messages").getBytes("ISO-8859-1"), "UTF-8");
Log.i("message2", messages);

You're telling Java to convert the string (with key message) to bytes using ISO-8859-1 and than to create a new String from these bytes, interpreted as UTF-8.
new String(reader.getString("messages").getBytes("ISO-8859-1"), "UTF-8");
You could simply use:
String messages = reader.getString("messages");

You can update your code as the following:
private static String getUrlContents(String theUrl) {
StringBuilder content = new StringBuilder();
try {
URL url = new URL(theUrl);
URLConnection urlConnection = url.openConnection();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream(), "utf-8"));
String line;
while ((line = bufferedReader.readLine()) != null) {
content.append(line).append("\n");
}
bufferedReader.close();
} catch(Exception e) {
e.printStackTrace();
}
return content.toString().trim();
}

You've got two encoding issues:
The server sends text encoded in a character set. When you setup your InputStreamReader, you need to pass the encoding the server used so it can be decoded properly. The character encoding is usually given in the Content-type HTTP response, in the charset field. JSON is typically UTF-8 encoded, but can also be legally UTF-16 and UTF-32, so you need to check. Without a specified encoding, your system environment will be used when marshalling bytes to Strings, and vice versa . Basically, you should always specify the charset.
String messages = new String(reader.getString("messages").getBytes("ISO-8859-1"), "UTF-8"); is obviously going to cause issues (if you have non-ascii characters) - it's encoding the string to ISO-8995-1 and then trying to decode it as UTF-8.
A simple regex pattern can be used to extract the charset value from the Content-type header before reading the inputstream. I've also included a neat InputStream -> String converter.
private static String getUrlContents(String theUrl) {
try {
URL url = new URL(theUrl);
URLConnection urlConnection = url.openConnection();
InputStream is = urlConnection.getInputStream();
// Get charset field from Content-Type header
String contentType = urlConnection.getContentType();
// matches value in key / value pair
Pattern encodingPattern = Pattern.compile(".*charset\\s*=\\s*([\\w-]+).*");
Matcher encodingMatcher = encodingPattern.matcher(contentType);
// set charsetString to match value if charset is given, else default to UTF-8
String charsetString = encodingMatcher.matches() ? encodingMatcher.group(1) : "UTF-8";
// Quick way to read from InputStream.
// \A is a boundary match for beginning of the input
return new Scanner(is, charsetString).useDelimiter("\\A").next();
} catch(Exception e) {
e.printStackTrace();
}
return null;
}

Not sure if this will help, but you might be able to do something like this:
JSONObject result = null;
String str = null;
try
{
str = new String(output, "UTF-8");
result = (JSONObject) new JSONTokener(str).nextValue();
}
catch (Exception e) {}
String messages = result.getString("messages");

Android HttpClient response

I'm trying to send json data to a php script from my Android application with HttpClient, and get the response.
Android Code
private void sendPurchase(String SKU) throws IOException{
Log.e("sendPurchase","Inside sendPurchase");
final SharedPreferences prefs = getGCMPreferences(getApplicationContext());
int pur_user = prefs.getInt("C_user", Integer.MIN_VALUE);
InputStream inputStream = null;
String result = "";
HttpClient httpclient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost("http://www.*.com/includes/purchase.php");
JSONObject json = new JSONObject();
try {
json.put("PUR_sku", SKU);
json.put("PUR_user", pur_user);
} catch (JSONException e) { Log.e("SendPurchase","Problem with Json Object"); }
Log.i("JSONObject", json.toString());
StringEntity se = new StringEntity(json.toString(), HTTP.UTF_8);
httpPost.setEntity(se);
httpPost.setHeader("Accept", "application/json");
httpPost.setHeader("Content-type", "application/json");
HttpResponse httpResponse = httpclient.execute(httpPost);
inputStream = httpResponse.getEntity().getContent();
if(inputStream != null){ result = convertInputStreamToString(inputStream); }
else{result = "Did not work!"; }
Log.e("RESULT",result);
}
private static String convertInputStreamToString(InputStream inputStream) throws IOException{
BufferedReader bufferedReader = new BufferedReader( new InputStreamReader(inputStream));
String line = "";
String result = "";
while((line = bufferedReader.readLine()) != null)
result += line;
inputStream.close();
return result;
}
And the PHP script
<?
$auth=0;
require('./connexion.php');
$data = file_get_contents('php://input');
//$data = '{"PUR_sku":"singleone","PUR_user":"3"}';
$json = json_decode($data,true);
/* Some database stuff ... */
echo "Retour ".print_r($json)." et ".$json['PUR_sku']." et ".$json['PUR_user'];
?>
When i launch the app and execute sendPurchase function, it seems to be ok until the execution of the HttpPost. In the logcat i get all the logs with correct params, except the last log "RESULT" that does not appear.
That's why i guess something is going wrong with the HttpPost execution, but actually i don't know if the problem comes from the application side or the php script side...
When i execute the php script alone in a web browser, replacing first $data line by the second one, everything is ok. But when it comes from the application it's not ok...
The Json Object sent (i hope) to the script seems ok too : {"PUR_user":3,"PUR_sku":"singleone"}
(the sendPurchase function is executed in Background).
Any idea about what i'm doing wrong ? Thanks !
/EDIT/
Here is the logcat for #RyuZz solution.
My code is about purchasing an item, consume it and send new value to my database on a web server. The purchase & consume are ok, but i can't send the values to the web server.
And again, when i execute the php script alone in a web browser, replacing first $data line by the second one, everything is ok.
Note that i have another similar code to register user to GCM, using HttpClient, and that code works fine.
06-25 14:07:12.968: D/IabHelper(21833): Successfully consumed sku: singleconf
06-25 14:07:12.968: D/IabHelper(21833): Ending async operation: consume
06-25 14:07:12.979: D/CONSUME(21833): Consumption finished. Purchase: PurchaseInfo(type:inapp):{"orderId":"12999763169054705758.1353445524837889","packageName":"com.*.*","productId":"singleconf","purchaseTime":1435234296875,"purchaseState":0,"purchaseToken":"bohbcbiigcbidfficbikebnk.AO-J1OzuQ_SsNTG1h9MtUvbaPc3PeN9nBHG-qBOE82ao1rTDFNrgA7tYQcMdECxCVFrrZEn_QifQ28OcIupyesZI-5cjDILFODYpBEaeqMfE0wCAeMFkJLfNUK_TsKPMj7F2sBDdgOYx"}, result: IabResult: Successful consume of sku singleconf (response: 0:OK)
06-25 14:07:12.979: D/CONSUME(21833): You bought & consumed a single conf
06-25 14:07:12.979: D/CONSUME(21833): End consumption flow.
06-25 14:07:12.979: E/Purchase Background(21833): Inside doInBackground
06-25 14:07:12.979: E/sendPurchase(21833): Failed to send HTTP POST request due to: java.lang.NullPointerException

You can try the following instead of HttpClient which is anyway deprecated:
try{
int pur_user = prefs.getInt("C_user", Integer.MIN_VALUE);
URL url = new URL("http://www.*.com/includes/purchase.php");
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setDoInput(true);
connection.setRequestProperty("Content-Type", "application/json");
connection.setRequestProperty("Accept", "application/json");
connection.setRequestMethod("POST");
JSONObject jsonObject = new JSONObject();
jsonObject.put("PUR_sku", SKU);
jsonObject.put("PUR_user", pur_user);
//convert JSONObject to JSON to String
json = jsonObject.toString();
OutputStreamWriter writer = new OutputStreamWriter(connection.getOutputStream());
writer.write(json);
writer.close();
responseCode = connection.getResponseCode();
if(responseCode == 200) {
InputStream content = connection.getInputStream();
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(content, "UTF-8"), 8);
StringBuilder sb = new StringBuilder();
String line;
while ((line = reader.readLine()) != null)
{
sb.append(line).append("\n");
}
result = sb.toString();
//TODO get your stuff from result
content.close();
} catch (Exception ex) {
Log.e(TAG, "Failed to parse JSON due to: " + ex);
} finally {
connection.disconnect();
}
} else {
Log.e(TAG, "Server responded with status code: " + responseCode);
}
} catch(Exception ex) {
Log.e(TAG, "Failed to send HTTP POST request due to: " + ex);
}
if this isn't working, please post the logcat.
Don't forget to implement the required permissions in your manifest:
<uses-permission
android:name="android.permission.INTERNET" />

HttpUrlConnection.getInputStream returns empty stream in Android

I make a GET request to a server using HttpUrlConnection.
After connecting:
I get response code: 200
I get response message: OK
I get input stream, no exception thrown but:
in a standalone program I get the body of the response, as expected:
{"name":"my name","birthday":"01/01/1970","id":"100002215110084"}
in a android activity, the stream is empty (available() == 0), and thus I can't get
any text out.
Any hint or trail to follow? Thanks.
EDIT: here it is the code
Please note: I use import java.net.HttpURLConnection; This is the standard
http Java library. I don't want to use any other external library. In fact
I did have problems in android using the library httpclient from apache (some of their anonymous .class can't be used by the apk compiler).
Well, the code:
URLConnection theConnection;
theConnection = new URL("www.example.com?query=value").openConnection();
theConnection.setRequestProperty("Accept-Charset", "UTF-8");
HttpURLConnection httpConn = (HttpURLConnection) theConnection;
int responseCode = httpConn.getResponseCode();
String responseMessage = httpConn.getResponseMessage();
InputStream is = null;
if (responseCode >= 400) {
is = httpConn.getErrorStream();
} else {
is = httpConn.getInputStream();
}
String resp = responseCode + "\n" + responseMessage + "\n>" + Util.streamToString(is) + "<\n";
return resp;
I see:
200
OK
the body of the response
but only
200
OK
in android

Trying the code of Tomislav I've got the answer.
My function streamToString() used .available() to sense if there is any data received,
and it returns 0 in Android. Surely, I called it too soon.
If I rather use readLine():
class Util {
public static String streamToString(InputStream is) throws IOException {
StringBuilder sb = new StringBuilder();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
String line;
while ((line = rd.readLine()) != null) {
sb.append(line);
}
return sb.toString();
}
}
then, it waits for the data to arrive.
Thanks.

You can try with this code that will return response in String:
public String ReadHttpResponse(String url){
StringBuilder sb= new StringBuilder();
HttpClient client= new DefaultHttpClient();
HttpGet httpget = new HttpGet(url);
try {
HttpResponse response = client.execute(httpget);
StatusLine sl = response.getStatusLine();
int sc = sl.getStatusCode();
if (sc==200)
{
HttpEntity ent = response.getEntity();
InputStream inpst = ent.getContent();
BufferedReader rd= new BufferedReader(new InputStreamReader(inpst));
String line;
while ((line=rd.readLine())!=null)
{
sb.append(line);
}
}
else
{
Log.e("log_tag","I didn't get the response!");
}
} catch (ClientProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return sb.toString();
}

The Stream data may not be ready, so you should check in a loop that the data in the stream is available before attempting to access it.
Once the data is ready, you should read it and store in another place like a byte array; a binary stream object is a nice choice to read data as a byte array. The reason that a byte array is a better choice is because the data may be binary data like an image file, etc.
InputStream is = httpConnection.getInputStream();
byte[] bytes = null;
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] temp = new byte[is.available()];
while (is.read(temp, 0, temp.length) != -1) {
baos.write(temp);
temp = new byte[is.available()];
}
bytes = baos.toByteArray();
In the above code, bytes is the response as byte array. You can convert it to string if it is text data, for example data as utf-8 encoded text:
String text = new String(bytes, Charset.forName("utf-8"));