I'm following the tutorial to download content from webpage. http://developer.android.com/training/basics/network-ops/connecting.html#download (code is copied below so you don't have to go to this link)
It use len = 500 in this example and I change it to big value such as 50000 but while experimenting I realize this method will only download the first 4048 characters of a webpage no matter how large I set len to be. So I'm wondering if I should use another method to download web content.
Actually I'm not downloading normal webpage, I've put a php script on my server to search in my database then encode a json array as the content of the page, it's not very large, about 20,000 characters..
Main codes from the above link:
// Given a URL, establishes an HttpUrlConnection and retrieves
// the web page content as a InputStream, which it returns as
// a string.
private String downloadUrl(String myurl) throws IOException {
InputStream is = null;
// Only display the first 500 characters of the retrieved
// web page content.
int len = 500; // I've change this to 50000
try {
URL url = new URL(myurl);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setReadTimeout(10000 /* milliseconds */);
conn.setConnectTimeout(15000 /* milliseconds */);
conn.setRequestMethod("GET");
conn.setDoInput(true);
// Starts the query
conn.connect();
int response = conn.getResponseCode();
Log.d(DEBUG_TAG, "The response is: " + response);
is = conn.getInputStream();
// Convert the InputStream into a string
String contentAsString = readIt(is, len);
return contentAsString;
// Makes sure that the InputStream is closed after the app is
// finished using it.
} finally {
if (is != null) {
is.close();
}
}
}
// Reads an InputStream and converts it to a String.
public String readIt(InputStream stream, int len) throws IOException, UnsupportedEncodingException {
Reader reader = null;
reader = new InputStreamReader(stream, "UTF-8");
char[] buffer = new char[len];
reader.read(buffer);
return new String(buffer);
}
Are you sure it's not just LogCat truncating the message?
(Android - Set max length of logcat messages)
Try:
Printing out line by line in your readIt method
Doing this (Displaying More string on Logcat)
Saving to SD card and looking at the file
Actually doing what you want to do with it (put it in a TextView or whatever)
Related
So I'm trying to load this JSON in Android from here, and have tried both Volley and regular HTTP requests. The page (eventually) loads fine as UTF-8 JSON and it looks fine. However, in android, I get garbage like this:
Checked the document.characterSet, it's UTF-8.
Example as Volley (trimmed out some code, so brackets may not be exact):
final JsonObjectRequest jsonObjectRequest = new JsonObjectRequest(url, new Response.Listener<JSONObject>() {
#Override
public void onResponse(JSONObject response) {
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError ex) {
Log.e("LOG", ex.toString());
}
});
requestQueue.add(jsonObjectRequest);
Example as regular HTTP GET:
HttpURLConnection urlConnection = null;
URL url = new URL(urlString);
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setRequestMethod("GET");
urlConnection.setReadTimeout(10000 /* milliseconds */);
urlConnection.setConnectTimeout(2500 /* milliseconds */);
urlConnection.setDoOutput(true);
urlConnection.connect();
String UTF8 = "UTF-8";
int BUFFER_SIZE = 8192;
BufferedReader br = new BufferedReader(new InputStreamReader(url.openStream(), UTF8), BUFFER_SIZE);
StringBuilder sb = new StringBuilder();
String line;
while ((line = br.readLine()) != null) {
sb.append(line + "\n");
}
br.close();
String jsonString = sb.toString();
urlConnection.disconnect();
urlConnection = null;
br = null;
sb = null;
if (jsonString != null &&
jsonString.length() > 0) {
return new JSONObject(jsonString);
}
Both give garbage responses. What am I missing? I'm able to access other data on other sites.
The content is compressed using zlib. See header:
Content-Encoding: deflate
You'll have to read the raw bytes and decompress them before attempting to parse as JSON. Looks like Android provides native support for zlib via the Deflater class.
Note that further, readers by default use the system default character encoding. Unless your system default happens to match that of the delivered content, you'll need to tell the system how to decode the charset. The correct thing is to read raw bytes from a stream, then turn the bytes into a string using the proper character encoding,
byte[] buffer = new byte[1024];
int c;
ByteArrayOutputStream baos = new ByteArrayOutputStream();
while ((c = inputStream.read(buffer)) > 0) {
baos.write(buffer, 0, c);
}
String json = new String(baos.toByteArray(), "UTF-8"); // assuming the encoding if UTF-8
You can either know the encoding ahead of time, or parse it from the Content-Type header. I looked at the response from the provided URL and it does not specify a charset, so you'll have to hardcode the known value.
EDIT: apparently you can do this with a reader, although I haven't tried it:
Reader r = new InputStreamReader(inputStream, "UTF-8");
String thisurl ="http://songolum.com/file/ppdVkTxqhwcJu-CAzCgtNeICMi8mHZnKQBgnksb5o2Q/Ed%2BSheeran%2B-%2BPerfect.mp3?r=idz&dl=311&ref=ed-sheran-perfect";
url = null;
try {
url = new URL(thisurl);
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
try {
// urlConnection.setRequestProperty("Transfer-Encoding", "chunked");
urlConnection.setRequestProperty("Accept-Encoding", "identity");
urlConnection.setDoOutput(true);
urlConnection.setChunkedStreamingMode(0);
int l=0;
InputStream in = new BufferedInputStream(urlConnection.getInputStream());
while(in.read()!=-1)
{
l=l+in.read();
}
System.out.println("Content-length" +l);
**I checked with other software and I found it's gzip compressed file and its with 10mb and I'm getting almost 1mb **
To answer your question directly, you were going wrong because you were calling read() twice, and also because you were adding together the values of each byte read, instead of counting them. InputStream.read() reads one byte and returns its value, or -1 on EOF. You need to read a number of bytes into a buffer and count how many bytes each read() call returned:
InputStream in = urlConnection.getInputStream();
byte[] buffer = new byte[4096];
int countBytesRead;
while((countBytesRead = in.read(buffer)) != -1) {
l += countBytesRead;
}
System.out.println("Content-length: " + l);
However, I suspect that this is not really what you need to do. The above code will simply return the size of all content in the response, including the HTTP headers and the content. Perhaps what you are looking for is the length of the document (or the file to be downloaded). You can use the Content-length HTTP header for that purpose (see other SO questions for how to get HTTP headers).
Also, note that the content may or may not be gzip-compressed. It depends on what the HTTP request says it accepts.
Please try this one hope so it will be helpful for you.
Using a HEAD request, i got my webserver to reply with the correct content-length field which otherwise was wrong. I don't know if this works in general but in my case it does:
private int tryGetFileSize(URL url) {
HttpURLConnection conn = null;
try {
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("HEAD");
conn.getInputStream();
return conn.getContentLength();
} catch (IOException e) {
return -1;
} finally {
conn.disconnect();
}
}
I am using HttpUrlConnection to get JSON strings from web, but the response is good for smaller strings but not for larger ones: this is what I am seeing in my app, and this is the data from server to a web page I did not include any HTML from server side.
Here is my code:
URL adr = new URL("http://placeform.tk/forapp.php");
HttpURLConnection connection =(HttpURLConnection) adr.openConnection();
connection.connect();
int rcode = connection.getResponseCode();
Log.d("rcode",Integer.toString(rcode));
if (rcode == HttpURLConnection.HTTP_OK) {
InputStream inputStreamReader = connection.getInputStream();
String c = connection.getHeaderField("content-length");
Reader rd = new InputStreamReader(inputStreamReader);
Log.d("contentsize", Integer.toString(connection.getContentLength()) + c);
chars = new char[connection.getContentLength()];
Log.d("contentsize", Integer.toString((int)connection.getContentLength()) + c);
rd.read(chars);
String output = new String(chars);
use this link and add that class in your project and call webservice from that class. cause that class will build up your response string with use of string builder. have a look at that class. and comment if you have any problem.
I'm trying to make a simple chat server. It works fine on the iphone, but not all the chats are going through on the android.
My idear was to use the http protocol so I can use the standard sfkd on the iphone and android to talk to my server
I'm using the URLConnection connection class on the android.
When I was tacing the code on my server, I noticed that I was getting the length of the post data sent in the header. I'm not setting any length value on the URLConnection class. I figured since I was not setting the size in the header that is why its not working. I could not find any info about this.
code that reads in haeder on my server
I have my server code and android code below,
public int ReadHeader()
{
// read in one line
int PageId=0;
// try{
request = new StringBuffer(1000);
System.out.println("get connection reading in header \r");
//InputStream in = new BufferedInputStream( connection.getInputStream() );
StopFlag=false;
String out;
// the first line shouls have the page
out=ReadLine();
int p;
p=out.lastIndexOf("stop");
if (p!=-1)
PageId=1;
p=out.lastIndexOf("enter");
if (p!=-1)
PageId=2;
p=out.lastIndexOf("add");
if (p!=-1)
PageId=3;
p=out.lastIndexOf("exit");
if (p!=-1)
PageId=4;
p=out.lastIndexOf("update");
if (p!=-1)
PageId=5;
int MessageSize=0;
do{
out=ReadLine();
// check for content size
// GET SIZR OF DATA SENT
if (out.contains("Length"))
{
System.out.println("found length \r");
int pes=out.indexOf(' ');
String Length=out.substring(pes+1);
System.out.println("found size");
System.out.println(Length);
//MessageSize=(int)Length;
MessageSize= Integer.parseInt( Length) ;
//MessageSize=36;
}
} while(out!="finish header" && out!="");
System.out.println("finsih header \r");
ClientMessage=ReadSize(MessageSize);
/*
} catch(IOException ec)
{
System.out.println(ec.getMessage());
}
*/
return PageId;
}
// CODE ON ANDROID
String data = URLEncoder.encode("username", "UTF-8") + "=" + URLEncoder.encode( cGlobals.UserName, "UTF-8"); data += "&" + URLEncoder.encode("comment", "UTF-8") + "=" + URLEncoder.encode( s, "UTF-8");
cGlobals.Message=""; // THIS IS NOT TKING IN ANY INFO ABOUT THE LENGTH OF data URLConnection conn = url.openConnection();
// set timeouts to 5 seconds
// conn.setConnectTimeout(1000*5);
// conn.setReadTimeout(5*1000);
conn.setDoOutput(true);
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
wr.close();
I have a wamp server. I have written my android client. If I run that app, the response is fine on an emulator...but the same code does not work on the real device, I mean I dont get a response.....
Here s the code...
public static final String SERVER_URL = "http://192.168.1.3/AndroidListServer/server.php?command=getAnimalList";
private static String executeHttpRequest(String data) {
String result = "";
try {
URL url = new URL(SERVER_URL);
URLConnection connection = url.openConnection();
/*
* We need to make sure we specify that we want to provide input and
* get output from this connection. We also want to disable caching,
* so that we get the most up-to-date result. And, we need to
* specify the correct content type for our data.
*/
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setUseCaches(false);
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
// Send the POST data
DataOutputStream dataOut = new DataOutputStream(connection.getOutputStream());
dataOut.writeBytes(data);
dataOut.flush();
dataOut.close();
// get the response from the server and store it in result
DataInputStream dataIn = new DataInputStream(connection.getInputStream());
String inputLine;
while ((inputLine = dataIn.readLine()) != null) {
result += inputLine;
}
dataIn.close();
} catch (IOException e) {
/*
* In case of an error, we're going to return a null String. This
* can be changed to a specific error message format if the client
* wants to do some error handling. For our simple app, we're just
* going to use the null to communicate a general error in
* retrieving the data.
*/
e.printStackTrace();
result = null;
}
return result;
}
Solved it guys....it was the problem with the firewall as Rajesh mentioned...I should do a thorough testing of all the possible parameters....but hey I'm learning :)