I have used map in my android application. I passed origin and destination latlon and get data from map url then parse the response.
But while auditing below code as marked for DOS attack stating that "This code might allow an attacker to crash the program or otherwise make it unavailable to legitimate users."
Concern : What if attacker push too large file then it will go on line by line and loop will be run for too long.
Proposed solution : Do not allow to read more than specific file size, so that it won't read file beyond some limit
Here is my code :
String url = "https://maps.googleapis.com/maps/api/directions/json"+ "?" + str_origin + "&" + str_dest + "&" + "sensor=false";
private String downloadDataFromUrl(String strUrl) throws IOException {
String data = "";
InputStream iStream = null;
HttpsURLConnection urlConnection = null;
try {
URL url = new URL(strUrl);
urlConnection = (HttpsURLConnection) url.openConnection();
urlConnection.connect();
iStream = urlConnection.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(iStream),1024);
StringBuffer sb = new StringBuffer();
String line = "";
while ((line = br.readLine()) != null) {
sb.append(line);
}
data = sb.toString();
br.close();
} catch (Exception e) {
Log.d("Exception", e.toString());
} finally {
iStream.close();
urlConnection.disconnect();
}
return data;
}
Please provide solution. Thanks in advance.
Edit 1:by calling append() it appends Untrusted data to a StringBuilder instance initialized with the default backing-array size (16). This can cause the JVM to over-consume heap memory space.
If you download from an unknown URL, the data can indeed be arbitrary and BufferedReader.readLine() can encounter a line so long the program cannot handle it. This question indicates that limiting BufferedReader line length may not be trivial.
Number of lines can be too big as well, in which case line count check instead of simple null check in the while loop seems to be enough.
Question is why would you allow the user to input an arbitrary URL and download it without checking. The URL can easily be a several GB binary file. Your first line indicates that you intend to use the Google Maps API, which AFAIK does not return excessively large lines, rendering the DOS concern moot (except in some ultrasecure applications, which I do not think Android is suitable to use for).
Related
I used BufferedReader to proccess the output of a webpage. When the output of webpage is empty (I used Response.Clear in web side), the last line Log.e("status","finish") does nothing. Is reader.readLine() being stucked in empty output? If yes, how should I check if response is empty before using a reader?
URLConnection connection = new URL(url).openConnection();
connection.setDoOutput(true);
connection.setRequestProperty("Accept-Charset", "utf-8");
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=" + "utf-8");
connection.connect(); // The code works same without this. Do I need this?
try (OutputStream output = connection.getOutputStream()) {
output.write(query.getBytes("utf-8"));
Log.e("status", "post Done"); // This works
}
InputStream response = connection.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(response));
String line="";
while ((line = reader.readLine()) != null) {
urlData += line;
}
reader.close();
Log.e("status","finish");
Yes, it is "stucked", although the correct wording is that it is "blocked". It blocks until it receives a line of text. When the socket is closed at the other party, the TCP connection will indicate termination and the input stream gets closed. At that point you would retrieve null as specified by the API. However, before that happens the high level readLine routine will happily wait until the end of time, or until a time-out is generated by a lower layer.
As such, it might not be a good idea to use readLine or even stream if you don't trust your server connection to return any data. You can however set the socket to time-out and generate an exception instead using Socket.html#setSoTimeout(int) - if you think that the server not responding is an exceptional problem.
I am reading html source code of a public website using the following code:
Code:
#Override
protected Void doInBackground(Void... params)
{
try
{
URL url = new URL(""+URL);
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
String inputLine;
PageCode = "";
OriginalPageCode = "";
while ((inputLine = in.readLine()) != null)
{
PageCode += inputLine;
}
OriginalPageCode = PageCode;
try
{
extract_website_and_save(); // extracting data from PageCode
}
catch (Exception e1)
{
}
in.close();
}
Background:
The above code sometimes can fetch the most updated website properly. But occasionally it linked to an outdated version of the website and hence unable to obtain the most updated information for the website.
I am curious why the above will occur, does it related to extracting from cache instead of the real updated website??
I therefore used Chrome to browse the same link, and discovered that Chrome also fetched the outdated website.
I have tried restarting the device, but the problem continues.
After 30 minutes to an hour, I requested the app to fetch again and it then can extract the most updated information. I at the same time browse the website using Chrome, Chrome can now obtain the most updated website.
Question:
The above BufferedReader should have no relationship with Chrome? But they follow the same logic and hence extracting from cache instead of from the most updated website?
I strongly suspect the end point is being cached by URL
Try something like this
urlSrt = urlSrt + "?x=" + new Random().nextInt(100000);
// If your URL already is passing parameters i.e. example.com?x=1&p=pass - then modify
// the urlSrt line to to use an "&" and not "?"
// i.e. urlSrt = urlSrt + "&x=" + new Random().nextInt(100000);
URL url = new URL(urlSrt);
URLConnection con = url.openConnection();
con.setUseCaches(false); //This will stop caching!
So if you modify your code to something like this.
URLConnection con = url.openConnection();
con.setUseCaches(false);
BufferedReader in = new BufferedReader(new InputStreamReader(
con.getInputStream()));
Confused too much! What is the sole purpose of using BufferReader, InputStream and StringBuffer. Why they should be used and in what kind of a sequence / pattern we should code them. I recently came across a chunk of code while understanding how to send and receive HTTP requests using HttpUrlConnection in android. I tried to search for all these terms and I did not get what I need. In this case, how to use each of them in a sequence or pattern? Any simple example for using all these three in combination would be great. And also kindly what should be all these 3 in layman terms? Thanks
InputStream is used for reading byte based data from the web server (or url) one byte at a time.
BufferReader it is used for reading data from an input stream all by once
StringBuffer A modifiable sequence of characters for use in creating strings, where all accesses are synchronized. This class has mostly been replaced by StringBuilder because this synchronization is rarely useful. This class is mainly used to interact with legacy APIs that expose it. [Did not understand what it means as my official language is not English]
//These two need to be declared outside the try/catch
//so that they can be closed in the finally block.
HttpURLConnection urlConnection = null;
BufferedReader reader = null;
//Will contain the raw JSON response as a string.
String forecastJsonStr = null;
try {
//Construct the URL for the OpenWeatherMap query
//Possible parameters are avaiable at OWM's forecast API page, at
//http://openweathermap.org/API#forecast
URL url = new URL("http://api.openweathermap.org/data/2.5/forecast/daily?q=94043&mode=json&units=metric&cnt=7");
//Create the request to OpenWeatherMap, and open the connection
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setRequestMethod("GET");
urlConnection.connect();
//Read the input stream into a String
InputStream inputStream = urlConnection.getInputStream();
StringBuffer buffer = new StringBuffer();
if (inputStream == null) {
//Nothing to do.
forecastJsonStr = null;
}
reader = new BufferedReader(new InputStreamReader(inputStream));
String line;
while ((line = reader.readLine()) != null) {
//Since it's JSON, adding a newline isn't necessary (it won't affect parsing)
//But it does make debugging a *lot* easier if you print out the completed
//buffer for debugging.
buffer.append(line + "\n");
}
if (buffer.length() == 0) {
//Stream was empty. No point in parsing.
forecastJsonStr = null;
}
forecastJsonStr = buffer.toString();
} catch (IOException e) {
Log.e("PlaceholderFragment", "Error ", e);
//If the code didn't successfully get the weather data, there's no point in attemping
//to parse it.
forecastJsonStr = null;
} finally{
if (urlConnection != null) {
urlConnection.disconnect();
}
if (reader != null) {
try {
reader.close();
} catch (final IOException e) {
Log.e("PlaceholderFragment", "Error closing stream", e);
}
}
}
I am making an android application which requires to send a mathematical question like 1+1 to google's calculator and I need to get that result which is displayed on the web. How can I achieve this on android?
One possibility is to create a URL for the equation you are trying to calculate and then use a URLConnection to open the URL and read the webpage source code to find the answer to the equation.
For example if you have the equation:
2+2
Then the URL to calculate the result with the Google Chrome calculator would be:
https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=2%2B2
You will have to construct the proper query in the URL for the equation you are solving. In this URL the query at the end has the equation 2+2:
q=2%2B2 (where the %2B represents the + sign)
After constructing the URL open it with a URLConnection and read the source. The answer to the equation will be in this element:
<span class="cwcot" id="cwos">4</span>
So you can parse the source in order to find that particular span element and retrieve the result of your equation.
This is probably more work than you expected but it is the only solution I can think of to accomplish what you asked. Also, this approach may be error prone and may break easily. I would consider using a different approach altogether such as launching an intent to use the calculator app on the mobile device (even though this approach has issues as well).
EDIT:
This worked for me (it will output: 2 + 2 = 4):
public static void test() {
try {
String source = getUrlSource();
String span = "<span class=\"nobr\"><h2 class=\"r\" style=\"display:inline;font-size:138%\">";
int length = span.length();
int index = source.indexOf(span) + length;
String equation = source.substring(index, source.indexOf("<", index));
System.out.println( "equation: " + equation);
} catch (IOException e) {
e.printStackTrace();
}
}
private static String getUrlSource() throws IOException {
String url = "https://www.google.com/search";
String charset = "UTF-8";
String param1 = "2+2";
String query = String.format("?q=%s", URLEncoder.encode(param1, charset));
HttpsURLConnection urlConn = (HttpsURLConnection)new URL(url + query).openConnection();
urlConn.setRequestProperty("User-Agent", "Mozilla/5.0");
urlConn.setRequestProperty("Accept-Charset", charset);
BufferedReader in = new BufferedReader(new InputStreamReader(urlConn.getInputStream()));
String inputLine;
StringBuilder a = new StringBuilder();
while ((inputLine = in.readLine()) != null)
a.append(inputLine);
in.close();
return a.toString();
}
I have a REST service I can't alter, with methods for uploading an image, encoded as a Base64 string.
The problem is that the images can go up to sizes of 5-10MB, perhaps more. When I try to construct a Base64 representation of an image of this size on the device, I get an OutOfMemory exception.
I can however encode chunks of bytes at a time (3000 let's say), but this is useless as I would need the whole string to create a HttpGet/HttpPost object:
DefaultHttpClient client = new DefaultHttpClient();
HttpGet httpGet = new HttpGet("www.server.com/longString");
HttpResponse response = client.execute(httpGet);
Is there a way of going around this?
Edit: trying to use Heiko Rupp's suggestions + the android doc, I get an exception ("java.io.FileNotFoundException: http://www.google.com") at the following line: InputStream in = urlConnection.getInputStream();
try {
URL url = new URL("http://www.google.com");
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setDoOutput(true);
urlConnection.setChunkedStreamingMode(0);
OutputStream out = new BufferedOutputStream(urlConnection.getOutputStream());
out.write("/translate".getBytes());
InputStream in = urlConnection.getInputStream();
BufferedReader r = new BufferedReader(new InputStreamReader(in));
StringBuilder total = new StringBuilder();
String line;
while ((line = r.readLine()) != null) {
total.append(line);
}
System.out.println("response:" + total);
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
Am I missing something? The GET request that I need to execute looks like this:
"http://myRESTService.com/myMethod?params=LOOONG-String", so the idea was to connect to http://myRESTService.com/myMethod and then output a few characters of the long string at a time. Is this correct?
You should try to use the URLConnection instead of the apache http client, as this does not require you to hold the object to send in memory, but instead you can do something like:
pseudocode!
HttpUrlConnection con = restUrl.getConnection();
while (!done) {
byte[] part = base64encode(partOfImage);
con.write (part);
partOfImage = nextPartOfImage();
}
con.flush();
con.close();
Also in Android after 2.2 Google recommends the URLConnection over the http client. See the description of DefaultHttpClient.
The other thing you may want to look into is the amount of data to be sent. 10 MB + base64 will take quite a while to transfer (even with gzip compression, which the URLConnection transparently enables if the server side accepts it) over a mobile network.
You must read docs for this REST service, no such service will require you to send such long data in GET. Images are always sent as POST. POST data is always at the end of request and allows to be added iteratively.