I used BufferedReader to proccess the output of a webpage. When the output of webpage is empty (I used Response.Clear in web side), the last line Log.e("status","finish") does nothing. Is reader.readLine() being stucked in empty output? If yes, how should I check if response is empty before using a reader?
URLConnection connection = new URL(url).openConnection();
connection.setDoOutput(true);
connection.setRequestProperty("Accept-Charset", "utf-8");
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=" + "utf-8");
connection.connect(); // The code works same without this. Do I need this?
try (OutputStream output = connection.getOutputStream()) {
output.write(query.getBytes("utf-8"));
Log.e("status", "post Done"); // This works
}
InputStream response = connection.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(response));
String line="";
while ((line = reader.readLine()) != null) {
urlData += line;
}
reader.close();
Log.e("status","finish");
Yes, it is "stucked", although the correct wording is that it is "blocked". It blocks until it receives a line of text. When the socket is closed at the other party, the TCP connection will indicate termination and the input stream gets closed. At that point you would retrieve null as specified by the API. However, before that happens the high level readLine routine will happily wait until the end of time, or until a time-out is generated by a lower layer.
As such, it might not be a good idea to use readLine or even stream if you don't trust your server connection to return any data. You can however set the socket to time-out and generate an exception instead using Socket.html#setSoTimeout(int) - if you think that the server not responding is an exceptional problem.
Related
I have used map in my android application. I passed origin and destination latlon and get data from map url then parse the response.
But while auditing below code as marked for DOS attack stating that "This code might allow an attacker to crash the program or otherwise make it unavailable to legitimate users."
Concern : What if attacker push too large file then it will go on line by line and loop will be run for too long.
Proposed solution : Do not allow to read more than specific file size, so that it won't read file beyond some limit
Here is my code :
String url = "https://maps.googleapis.com/maps/api/directions/json"+ "?" + str_origin + "&" + str_dest + "&" + "sensor=false";
private String downloadDataFromUrl(String strUrl) throws IOException {
String data = "";
InputStream iStream = null;
HttpsURLConnection urlConnection = null;
try {
URL url = new URL(strUrl);
urlConnection = (HttpsURLConnection) url.openConnection();
urlConnection.connect();
iStream = urlConnection.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(iStream),1024);
StringBuffer sb = new StringBuffer();
String line = "";
while ((line = br.readLine()) != null) {
sb.append(line);
}
data = sb.toString();
br.close();
} catch (Exception e) {
Log.d("Exception", e.toString());
} finally {
iStream.close();
urlConnection.disconnect();
}
return data;
}
Please provide solution. Thanks in advance.
Edit 1:by calling append() it appends Untrusted data to a StringBuilder instance initialized with the default backing-array size (16). This can cause the JVM to over-consume heap memory space.
If you download from an unknown URL, the data can indeed be arbitrary and BufferedReader.readLine() can encounter a line so long the program cannot handle it. This question indicates that limiting BufferedReader line length may not be trivial.
Number of lines can be too big as well, in which case line count check instead of simple null check in the while loop seems to be enough.
Question is why would you allow the user to input an arbitrary URL and download it without checking. The URL can easily be a several GB binary file. Your first line indicates that you intend to use the Google Maps API, which AFAIK does not return excessively large lines, rendering the DOS concern moot (except in some ultrasecure applications, which I do not think Android is suitable to use for).
I'm trying to establish a persistent HTTP connection to an API endpoint that publishes chunked JSON responses as new events occur. I would like to provide a callback that is called each time the server sends a new chunk of data, and keep the connection open indefinitely. As far as I can tell, neither HttpClient nor HttpUrlConnection provide this functionality.
Is there a way to accomplish this without using a TCP socket?
One solution would be to use a delimeter such as \n\n to separate each json event. You could remove blank lines from original json before sending. Calling setChunkedStreamingMode(0) allows you to read content as it comes in (rather than after the entire request has been buffered). Then you can simply go through each line, storing them, until a blank line is reached, then parse the stored lines as JSON.
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setChunkedStreamingMode(0);
conn.connect();
InputStream is = conn.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
StringBuffer sBuffer = new StringBuffer();
String line;
while ((line = reader.readLine()) != null) {
if (line.length() == 0) {
processJsonEvent(sBuffer.toString());
sBuffer.delete(0, sBuffer.length());
} else {
sBuffer.append(line);
sBuffer.append("\n");
}
}
As far as I can tell, Android's HttpURLConnection doesn't support receiving chunks of data across a persistent HTTP connection; it instead waits for the response to fully complete.
Using HttpClient, however, works:
HttpClient httpClient = new DefaultHttpClient();
try {
HttpUriRequest request = new HttpGet(new URI("https://www.yourStreamingUrlHere.com"));
} catch (URISyntaxException e) {
e.printStackTrace();
}
try {
HttpResponse response = httpClient.execute(request);
InputStream responseStream = response.getEntity().getContent();
BufferedReader rd = new BufferedReader(new InputStreamReader(responseStream));
String line;
do {
line = rd.readLine();
// handle new line of data here
} while (!line.isEmpty());
// reaching here means the server closed the connection
} catch (Exception e) {
// connection attempt failed or connection timed out
}
I have a REST service I can't alter, with methods for uploading an image, encoded as a Base64 string.
The problem is that the images can go up to sizes of 5-10MB, perhaps more. When I try to construct a Base64 representation of an image of this size on the device, I get an OutOfMemory exception.
I can however encode chunks of bytes at a time (3000 let's say), but this is useless as I would need the whole string to create a HttpGet/HttpPost object:
DefaultHttpClient client = new DefaultHttpClient();
HttpGet httpGet = new HttpGet("www.server.com/longString");
HttpResponse response = client.execute(httpGet);
Is there a way of going around this?
Edit: trying to use Heiko Rupp's suggestions + the android doc, I get an exception ("java.io.FileNotFoundException: http://www.google.com") at the following line: InputStream in = urlConnection.getInputStream();
try {
URL url = new URL("http://www.google.com");
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setDoOutput(true);
urlConnection.setChunkedStreamingMode(0);
OutputStream out = new BufferedOutputStream(urlConnection.getOutputStream());
out.write("/translate".getBytes());
InputStream in = urlConnection.getInputStream();
BufferedReader r = new BufferedReader(new InputStreamReader(in));
StringBuilder total = new StringBuilder();
String line;
while ((line = r.readLine()) != null) {
total.append(line);
}
System.out.println("response:" + total);
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
Am I missing something? The GET request that I need to execute looks like this:
"http://myRESTService.com/myMethod?params=LOOONG-String", so the idea was to connect to http://myRESTService.com/myMethod and then output a few characters of the long string at a time. Is this correct?
You should try to use the URLConnection instead of the apache http client, as this does not require you to hold the object to send in memory, but instead you can do something like:
pseudocode!
HttpUrlConnection con = restUrl.getConnection();
while (!done) {
byte[] part = base64encode(partOfImage);
con.write (part);
partOfImage = nextPartOfImage();
}
con.flush();
con.close();
Also in Android after 2.2 Google recommends the URLConnection over the http client. See the description of DefaultHttpClient.
The other thing you may want to look into is the amount of data to be sent. 10 MB + base64 will take quite a while to transfer (even with gzip compression, which the URLConnection transparently enables if the server side accepts it) over a mobile network.
You must read docs for this REST service, no such service will require you to send such long data in GET. Images are always sent as POST. POST data is always at the end of request and allows to be added iteratively.
The first time I made a method to read data from my chat server, it frooz. I found out I had the wrong port number and it was freezing at
new BufferedReader(new InputStreamReader(conn.getInputStream()));
Is there a way to have a time out so my program does not freez on a network error? I'm assuming there must be,
the complete methed
void SendMessage()
{
try {
URL url = new URL("http://50.63.66.138:1044/update");
System.out.println("make connection");
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
// String line;
String f=new String("");
String line=new String();
while ((line= rd.readLine() ) != null) {
f=f+line;
f+="\n";
}
mUsers.setText(f);
} catch (Exception e) {
System.out.println("exception");
System.out.println(e.getMessage());
}
}
}
First of all, I hope you execute this code in a separate thread in order to make your UI thread responsive on touches even while connection is being established.
Second, there are two timeout methods available for URLConnection class:
http://developer.android.com/reference/java/net/URLConnection.html#setReadTimeout(int)
http://developer.android.com/reference/java/net/URLConnection.html#setConnectTimeout(int)
Try to play with these guys, maybe it will help.
If not, you can always do your own way:
start the thread with a runnable which tries to establish a connection and InputReader. Then wait for some time-out and try to interrupt the thread if it is still running.
I'm developing an android application which is to collect data and then send it to a web directory.
So lets say a want to collect an array of data on the phone, and then after clicking a button send it all to the online directory as a file or stream. It does not even need to get a response - although in the future a confirmation would be handy.
Here is a guess at the sort of order of things...
dir = "someurl.com/data/files_received";
Array data;
sendDataSomehow(dir, data); //obv the difficult bit!
I am in very early stages of developing for Android although I have a lot of experience coding web so that bit will be fine.
I have found suggestions for things such as JSON, Google GSON, HTTP POST and GET - do these sound like the right track?
I hope I have been clear enough.
Yep, JSON would be a good solution for this.
Encode your array as JSON and then send it to your web server as the body of an HTTP POST request. If you have an hour to kill, here's a really good video from Google IO last year explaining how to implement a REST client on Android (what you're doing isn't strictly REST-ful, but the calls you make to the server are very similer): http://www.google.com/events/io/2010/sessions/developing-RESTful-android-apps.html
Right, just wanted to do a quick thank for putting me on the right track. Just had one of those THE CODE WORKS EUREKA moments, very happy. I haven't used JSON but I have managed to pass a variable from Android to SQL through a HTTP-POST and little bit of PHP.
I'm sure this is not the recommended ideology for many reasons although for prototype and presentation it will do just fine!
Here is the code for android:
try {
URL url = new URL("http://www.yourwebsite.com/php_script.php");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setDoOutput(true);
conn.setDoInput(true);
conn.setUseCaches(false);
conn.setAllowUserInteraction(false);
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
OutputStream out = conn.getOutputStream();
Writer writer = new OutputStreamWriter(out, "UTF-8");
writer.write("stringToPass=I'd like to pass this");
writer.close();
out.close();
if(conn.getResponseCode() != 200)
{
throw new IOException(conn.getResponseMessage());
}
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
StringBuilder sb = new StringBuilder();
String line;
while ((line = rd.readLine()) != null)
{
sb.append(line);
}
rd.close();
conn.disconnect();
} catch (MalformedURLException e1) {
textBox.setText(e1.toString());
} catch (IOException e2) {
textBox.setText(e2.toString());
}
And here is the code for the PHP:
$conn = mysql_connect("localhost","web108-table","********") or die (mysql_error());
mysql_select_db("web108-table",$conn) or die (mysql_error());
$str = $_POST['stringToPass'];
mysql_query("INSERT INTO table(field) VALUES ($str)");
This code works, very simple. Next tests will be to find out if it is suitable for a large number of strings.
I hope this is helpful to somebody else.