I have a big JSON input (download the file) API and I don´t know how to parse this data. I need:
Save this data (entire JSON input) to text file or database. What is the best way for this?
Load this data from text file or database and create JSONArray from JSON tag "list" (first tag)
The solution should be fast and support Android 2.3. What you have recomend for this? Any ideas?
My code:
HttpClient httpClient = new DefaultHttpClient();
HttpGet httpGet = new HttpGet(urls[0]);
HttpResponse httpResponse;
httpResponse = httpClient.execute(httpGet);
HttpEntity httpEntity = httpResponse.getEntity();
... and what next ?...
FYI:
EntityUtils throws OutOfMemoryException
EDIT:
I try to save data to file like this:
InputStream inputStream = httpEntity.getContent();
FileOutputStream output = new FileOutputStream(Globals.fileNews);
int bufferSize = 1024;
byte[] buffer = new byte[bufferSize];
int len = 0;
while ((len = inputStream.read(buffer)) != -1) {
output.write(buffer, 0, len);
}
And it´s OK. I load data:
FileInputStream fis = null;
StringBuffer fileContent = new StringBuffer("");
fis = new FileInputStream(Globals.fileNews);
byte[] buffer = new byte[1024];
while (fis.read(buffer) != -1) {
fileContent.append(new String(buffer));
}
But how convert StringBuffer to JSONObject? fileContent.ToString() is not ideal, sometimes I get OutOfMemoryException.
First of all: Dispose the HttpClient. Google discourages it:
Unfortunately, Apache HTTP Client does not, which is one of the many
reasons we discourage its use.
Source: developer.android.com
A good replacement is Google Volley. You have to build the JAR yourself but it just works like charm. I use for my setups Google Volley with OkHttp-Stack and GSON requests.
In your case you would write another Request which just writes the response out to the SD-card chunk by chunk. You don't buffer the string before! And some logic to open an input-stream from the file you wrote and give it to your JSON Deserializer. Jackson and GSON are able to handle streams out of the box.
Of course everything works with Android 2.3.
Don't, I repeat, don't try to dump the whole serialized stuff into a string or something. That's almost a OutOfMemoryException guarantee.
Related
I receive a file using the following code:
byte[] fileBytes;
....
JSONObject postJSON = new JSONObject();
postJSON.put("file_name", filename);
postJSON.put("client_id", clientID);
HttpPost post = new HttpPost(fileURL);
StringEntity se = new StringEntity( postJSON.toString(), "UTF-8");
se.setContentType(new BasicHeader(HTTP.CONTENT_TYPE, "application/json"));
post.setEntity(se);
response = httpClient.execute(post);
fileBytes = EntityUtils.toByteArray(response.getEntity());
Using the debugger, I see that the response gets an entity 27136 bytes in length, which is the correct length of the test file, but the fileBytes array is only 11470 bytes long. Can anyone tell my why this truncation is taking place? When I try to get other files, a similar truncation takes place, so it is not a function of the specific file or a specific file length.
Using the following code, I get 11997 bytes for the same file:
StringBuilder stringBuilder = new StringBuilder("");
stringBuilder.append(EntityUtils.toString(response.getEntity()));
fileBytes = stringBuilder.toString().getBytes();
Reading from an InputStream, I get 12288 bytes:
fileBytes = new byte[1024];
InputStream inputStream = response.getEntity().getContent();
int bytesRead = 0;
while(true){
bytesRead = inputStream.read(fileBytes);
if (bytesRead <= 0)
break;
....
}
Changing the encoding to UTF-16 gets me an internal server error.
I also tried the following:
InputStream inputStream = response.getEntity().getContent();
response.getEntity().getContentLength()];
while ((getByte = inputStream.read()) != -1) {
bos.write(getByte);
}
bos.close();
This also gave me a file of 11470.
In all cases, the files are corrupted, and cannot be opened. When compared in a binary file viewer, the firs 11 bytes match, and then the files diverge. I could not find any pattern in the corrupted file.
OK, the answer is apparently that all of the above are fine. The problem was with the server, which was not configuring the data stream correctly: Content-type was text/plain for all files, rather than application/pdf, and so on as appropriate.
My first clue was when we put a text file on the server, and it came over successfully. At that point I started working with the server side, and we figured it out pretty quickly.
Bottom line, if you are working on a server/client application, the problem might not be on your side.
I should have mentioned various posts which helped my construct the various versions that I collected above:
including this
and this
My apologies to various other helpful people whose posts I also looked at and up-voted.
I'm relatively new to Android (I'm an iOS-Developer) and I want to call a Webservice like I'm used to in iOS with NSURLConnectionDelegate's method
didReceiveData:(NSData *)data
I need to get the data incrementally because I'm building a streaming API that gets a lot of JSON data in response and needs to check the data for complete blocks.
Would be great if someone could help me, I've been searching for a while and didn't find a satisfying solution so far.
If you try to call web services in Android you should use the AsyncTask where the request would be made asynchronously. Have a look at the documentation. Every time you're request would be finished the method onPostExecute(Object result) would be called. Thats the method where you can go on with further processes.
The URLConnection documentation contain following example:
URL url = new URL("ftp://mirror.csclub.uwaterloo.ca/index.html");
URLConnection urlConnection = url.openConnection();
InputStream in = new BufferedInputStream(urlConnection.getInputStream());
try {
readStream(in);
finally {
in.close();
}
If i right understood your question, just implement readStream function as you need.
I found out how to do this with the help of a friend and some links.
You need to implement an own ResponseHandler like this:
class ChunkedResponseHandler implements ResponseHandler<String> {
#Override
public String handleResponse(HttpResponse response) throws ClientProtocolException, IOException {
HttpEntity entity = response.getEntity();
InputStream stream = entity.getContent();
StringBuffer output = new StringBuffer();
byte[] b = new byte[4096];
int n;
while ((n = stream.read(b)) != -1) {
output.append(new String(b, 0, n));
// do something while input is streaming
}
return output.toString();
}
}
Now you simply have to assign the response handler when starting the request:
HttpClient client = new DefaultHttpClient();
HttpGet getRequest = new HttpGet("someURL");
ResponseHandler<String> responseHandler = new ChunkedResponseHandler();
String responseBody = client.execute(postRequest, responseHandler);
I have a servlet that has the following purpose:
Receive data via the URL (that is, using get). Then returns a message, based on this input, back to the caller. I am new to this stuff, but have come to learn that using json (actually, Gson) is suitable for this.
My question now is, how do I retrieve this json message? What URL do I target? The relevant lines in the servlet are:
String json = new Gson().toJson(thelist);
response.setContentType("application/json");
response.setCharacterEncoding("UTF-8");
response.getWriter().println(json);
This is how I try to retrieve the json:
try{
DefaultHttpClient defaultClient = new DefaultHttpClient();
HttpGet httpGetRequest = new HttpGet("http://AnIPno:8181/sample/response?first=5&second=92866");
HttpResponse httpResponse = defaultClient.execute(httpGetRequest);
BufferedReader reader = new BufferedReader(new InputStreamReader(httpResponse.getEntity().getContent(), "UTF-8"));
String json = reader.readLine();
JSONObject jsonObject = new JSONObject(json);
} catch(Exception e){
e.printStackTrace();
}
But apparently this does not work, as I have found jsonObject has a size of 0 (it should be an array with three elements).
Previously, I had a write() instead of println() in the servlet. I'm not sure if that matters in this case. But I'm assuming I've misunderstood something about how the json object is retrieved. Is it not enough to point it towards the URL of the servlet?
Reading an InputStream whether from a File on the file system or from an HTTP request is, in most cases, the same.
What you have is correct only if your servlet wrote a single line. If the Gson object toString() method returns multiple lines, you're going to have to read multiple lines from the InputStream. I like to use the Scanner class for reading from an InputStream.
try {
DefaultHttpClient defaultClient = new DefaultHttpClient();
HttpGet httpGetRequest = new HttpGet("http://localhost:8080/cc/jsonyeah");
HttpResponse httpResponse = defaultClient.execute(httpGetRequest);
Scanner scanner = new Scanner(httpResponse.getEntity().getContent(), "UTF-8");
while(scanner.hasNextLine()) { // scanner looks ahead for an end-of-line
json += scanner.nextLine() + "\n"; // read the full line, you can append a \n
}
// do your serialization
} catch(Exception e) {
e.printStackTrace();
}
So we've done the same thing we would've done if we were reading from a file. Now the json object contains the json you received from the servlet, as a String.
For the serialization, you have a few options.
A Gson object has an overloaded method fromJson() that can take a String or a Reader, among other things.
From where we are with the code above, you can do
MyClass instance = new Gson().fromJson(json, MyClass.class);
where MyClass is the type you are trying to create. You will have to use a TypeToken for generic classes (such as a list). TypeToken is an abstract class, so generate an anonymous class and call getType()
Type type = new com.google.gson.reflect.TypeToken<List<String>>(){}.getType();
List<MyClass> list = new Gson().fromJson(json, type);
Another option is to use the overloaded method that takes a Reader directly instead of reading line by line from the InputStream:
BufferedReader reader = new BufferedReader(new InputStreamReader(httpResponse.getEntity().getContent(), "UTF-8"));
MyClass instance = new Gson().fromJson(reader , MyClass.class);
You'll get to skip a step.
Don't forget to close your streams.
I have this function to readJsonData from a a request to a JSON String. You can use this function to retrieve the JSON, then use GSON to parse it to the object that you like. It works for my application. Hope it works for you too.
protected String readJson(HttpResponse resp)
throws IOException {
BufferedReader reader = null;
try {
reader = new BufferedReader(new InputStreamReader(
resp.getEntity().getContent()));
StringBuffer buffer = new StringBuffer();
int read;
char[] chars = new char[1024];
while ((read = reader.read(chars)) != -1)
buffer.append(chars, 0, read);
} finally {
if (reader != null)
reader.close();
}
return buffer.toString();
}
So based on your code. I guess this should work:
String jsonData = readJson(httpResponse);
YourObject obj = new Gson().fromJson(jsonData, YourObject.class);
Before trying this, make sure your servlet prints out the JSON data that you want. I suggest using these Chrome Extensions: Postman - REST Client and JSON Formatter, to test your data from servlet. It's pretty helpful.
I am trying to get compressed data from server. The guy that programmed the server told me, that he uses ZLIB library on his iPhone, and the gzcompress on server. I was trying to find any suitable way to get that data, but it ends up with info "java.io.IOException: unknown format (magic number 9c78)" while creating GZIPInputStream object. Finally I've reached point, where I had data as a String. It was compressed, so I used that answer to decompress: https://stackoverflow.com/a/6963668/419308 . But that code doesn't work. "in.read()" returns -1 at the beginning.
Anyone has any idea why there's -1 ? Or maybe a better way to get the compressed data?
EDIT:
I tried adding file to project and reading from that file. in.read() didn't return -1
EDIT2: According to jJ's answer I've tried this code:
HttpGet request = new HttpGet( urlTeam );
HttpResponse response = new DefaultHttpClient().execute( request );
HttpEntity entity = response.getEntity();
InputStream stream = AndroidHttpClient.getUngzippedContent( entity );
InputStreamReader reader = new InputStreamReader( stream );
BufferedReader buffer = new BufferedReader( reader );
StringBuilder sb = new StringBuilder();
sb.delete( 0, sb.length() );
String input;
while ( ( input = buffer.readLine() ) != null )
{
sb.append( input );
}
But the answer is still compressed (or unreadable)
on android you should use either HttpURLConnection which handles decompression (and http encoding headers) for you from GingerBread on or AndroidHttpClient (for older android versions) that has helper methods like getUngzippedContent and modifyRequestToAcceptGzipResponse.
This is a good summary Android HTTP clients
I have a RESTful WCF service that I am using to retrieve encoded photos and display them in android (trying to anyway). The problem I am having is that the InputStream or possibly something else stops reading the characters before the end.
The response is just an XML string, I intend to parse it myself so no need to worry about that. What I need to know is what in the following code is stopping the input stream from reading characters into my buffer.
HttpEntity responseEntity = response.getEntity();
char[] buffer = new char[(int)responseEntity.getContentLength()];
InputStream stream = responseEntity.getContent();
InputStreamReader reader = new InputStreamReader(stream);
reader.read(buffer);
stream.close();
Have you implemented the HttpURLConnection class.. this would cause this behavior.
http://developer.android.com/reference/java/net/HttpURLConnection.html