Bizarre URL issue When connecting to Guardian API - android

I am connecting to the Guardian's News feed to download JSON. Every now and then I get get back JavaScript and it causes an error. The URL is stored in a static final string so is always the same. The url that I see in the debugger is correct for JSON. In fact, if I copy the url out of the debugger expressions window and paste it into a browser, I get back JSON. I don't think the problem is with the Guardian. I think something in my connection code might be corrupting the url somehow, and the Guardian is sending me JavaScript as a default. But I really don't know.
Here is the URL that is used in all cases. Notice format-json:
static final String GuardianUrl = "http://content.guardianapis.com/search?format=json&show-fields=headline%2Cbody%2Cthumbnail%2CtrailText%2ClastModified&date-id=date%2Ftoday&api-key=mykey";
Here is the top of their JSON file when it comes back correctly:
{
"response":{
"status":"ok",
"userTier":"approved",
"total":203,
"startIndex":1,
"pageSize":10,
"currentPage":1,
"pages":21,
"orderBy":"newest",
"results":[{
etc.
And this is what I get back when it comes as JavaScript:
09-21 15:00:18.853: E/JSON Parser(22101): Error converting string to json <html><head> <script language='javascript' type='text/javascript'>function init(_frm) { if (_frm.sent.value == 0) { _frm.sent.value=1; _frm.submit(); } }</script></head><body onload=init(auth)><form name=auth action='http://192.168.3.1:10080/ui/dynamic/guest-login.html' METHOD=GET><input type=hidden name='mac_addr' value='e0:75:7d:d3:15:0a'><input type=hidden name='url' value='http://content.guardianapis.com/search?format=json&#38show-fields=headline%2Cbody%2Cthumbnail%2CtrailText%2ClastModified&#38date-id=date%2Ftoday&#38api-key=pdva9u6ac2rqsx9a7hexzrv3'><input type=hidden name='ip_addr' value='192.168.3.142'><input type=hidden id=sent value='0'><noscript><input type=submit value='continue'></noscript></form></body></html> ... etc
And this is the code that connects and downloads the file:
public String getJSONFromUrl(String _url) throws IOException {
URL url = new URL(_url);
HttpURLConnection connection = (HttpURLConnection)url.openConnection();
// Making HTTP request
try {
//ByteArrayOutputStream out = new ByteArrayOutputStream();
//InputStream in = connection.getInputStream();
if (connection.getResponseCode() != HttpURLConnection.HTTP_OK) {
return null;
}
String line;
StringBuilder builder = new StringBuilder();
BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
while((line = reader.readLine()) != null) {
builder.append(line);
}
if (builder.toString().length()>0)
json = builder.toString();
return json;
} finally {
connection.disconnect();
}
}

It looks like a problem with your router, not your code. When it doesn't work, the response you are seeing is coming from your router.
Are you using the guest network feature on a Linksys E4200 by any chance? The response you are getting is the guest login page.
This may be useful: http://homecommunity.cisco.com/t5/Social-Support/Amazon-com-EA6500-Guest-network-became-inaccessible-after-a/td-p/583866

Related

Get Request HttpUrlConnection - Incorrect response

I am working on an Android App, App is using GET Request to connect with server.
The code I have written to connect with server is working perfectly on many devices.
But its not giving good response on LENOVO YOGA TAB3, It returns Html tags instead of JSON text, Firstly I was confused that there may be some issue in the API URL but I checked URL using browser, Its returning good response so I am sure URL is correct.
Here are API URL and its response :
API URL
[http://www.xyz.in/xyzapi/?building=on&address=Assotech Sandal Suites, Sector 135, Noida, Uttar Pradesh 201304, India&bill_amount=12500&type=R&fullstatename=Uttar Pradesh&lat=28.496171099999998&lng=77.4027049&state=UP&country=IN&district=Gautam Buddha Nagar&sublocality=Sector 135&calc-sess=59d4beba305f3&netmetering=1][1]
Response on Many Android Phones:
{"lifetimesaving":"25.0 Lacs","proposed_pv_capacity":10,"billWithSolar":3872,"bill_amount":"12500","sanction":10,"project_cost":"6.0 Lacs","return_oninevstment":"20.8","roi_image":"solar_score6_6.png","treeadded":"346 ","treeofftheroad":"9 "}
Response on Lenovo Yoga Tab3:
<head/><style>.personal-details h2{font-size:34px}.netmetering-tgle{position:absolute;left:0;right:0;bottom:5px;width:211px}.netmetering-tgle .wnm{float:left;position:relative;padding:0 10px 0 0;min-width:130px;text-align:center}.netmetering-tgle .wnm a{color:#c97511;background:none}#radioBtn .btn{border:1px solid transparent;border-radius:0!important;font-family:"Din-Bold"}#radioBtn .notActive{color:#c97511;background-color:#e0e1e2;padding:2px 0;width:38px;background-size:100% 100%;border-color:transparent;font-family:"Din-Bold";border-radius:5px}#radioBtn .active{color:#fff;background-color:#addc6f;border-color:transparent;padding:2px 5px;width:38px;cursor:auto;pointer-events:none;border-radius:5px;box-shadow:inset 0 0 10px rgba(0,0,0,.8)}#radioBtn .notActive[data-
So response is incorrect on Yoga Tab3.
Here is the code I am using to connect with Server :
public static String connectToServerUsingGETMethod(String API_COMPLETE_URL){
URL url;
HttpURLConnection urlConnection = null;
try {
url = new URL(API_COMPLETE_URL);
urlConnection = (HttpURLConnection) url.openConnection();
InputStream in = urlConnection.getInputStream();
InputStreamReader isr = new InputStreamReader(in);
String line = "";
BufferedReader reader = new BufferedReader(isr);
StringBuilder sb = new StringBuilder();
while ((line = reader.readLine()) != null) {
sb.append(line);
}
//get the string version of the response data
return sb.toString();
} catch (Exception e) {
e.printStackTrace();
} finally {
if (urlConnection != null) {
urlConnection.disconnect();
}
}
return "";
}
Suggest me If I can add something in this code so that it can work on Yoga Tab 3 too.
I think you should set static content type when making request to make sure the response sent back is JSON. Similarly, it looks like below:
HttpURLConnection urlConnection = null;
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setRequestProperty("Content-Type", "application/json; charset=UTF-8");
Hope it works for you.

Android: HttpUrlConnection object returns error 301 after connection

I'm trying to connect to a web API using a URL. However, I get a 301 error from the server (Moved Permanently), although the provided URL works very well with no errors when I try it in my browser.
Here is the code that builds the URL:
public Loader<List<Earthquake>> onCreateLoader(int i, Bundle bundle) {
SharedPreferences sharedPrefs = PreferenceManager.getDefaultSharedPreferences(this);
String minMagnitude = sharedPrefs.getString(
getString(R.string.settings_min_magnitude_key),
getString(R.string.settings_min_magnitude_default));
String orderBy = sharedPrefs.getString(
getString(R.string.settings_order_by_key),
getString(R.string.settings_order_by_default)
);
Uri baseUri = Uri.parse(USGS_REQUEST_URL);
Uri.Builder uriBuilder = baseUri.buildUpon();
uriBuilder.appendQueryParameter("format", "geojson");
uriBuilder.appendQueryParameter("limit", "10");
uriBuilder.appendQueryParameter("minmag", minMagnitude);
uriBuilder.appendQueryParameter("orderby", orderBy);
Log.i ("the uri is ", uriBuilder.toString());
return new EarthquakeLoader(this, uriBuilder.toString());
}
Here is the code that tries to connect to the resource represented by the URL:
private static String makeHttpRequest(URL url) throws IOException {
String jsonResponse = "";
// If the URL is null, then return early.
if (url == null) {
return jsonResponse;
}
Log.i("The received url is " , url +"");
HttpURLConnection urlConnection = null;
InputStream inputStream = null;
try {
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setReadTimeout(10000 /* milliseconds */);
urlConnection.setConnectTimeout(15000 /* milliseconds */);
urlConnection.setRequestMethod("GET");
urlConnection.connect();
// If the request was successful (response code 200),
// then read the input stream and parse the response.
if (urlConnection.getResponseCode() == 200) {
inputStream = urlConnection.getInputStream();
jsonResponse = readFromStream(inputStream);
} else {
Log.e(LOG_TAG, "Error response code: " + urlConnection.getResponseCode()); //this log returns 301
}
} catch (IOException e) {
Log.e(LOG_TAG, "Problem retrieving the earthquake JSON results.", e);
} finally {
if (urlConnection != null) {
urlConnection.disconnect();
}
if (inputStream != null) {
// Closing the input stream could throw an IOException, which is why
// the makeHttpRequest(URL url) method signature specifies than an IOException
// could be thrown.
inputStream.close();
}
}
return jsonResponse;
}
I could know that the connection returns status code of 301 from the log provided in the case when the status code is not 200. I have also logged the generated URL, I copied it from the logcat and tried it in my browser and it worked well. Here is the built URL: http://earthquake.usgs.gov/fdsnws/event/1/query?format=geojson&limit=10&minmag=6&orderby=magnitude
I checked this question: Android HttpURLConnection receives HTTP 301 response code but it wasn't clear to me what is the solution for the problem.
Can you please help me identify and solve the problem?
UPDATE: As greenapps indicated in his comment, the connection is done through https. That comment identified the problem and helped me fix the code.
In my code, the string I used to build the basic URL, had the protocol value as http not https, it was:
private static final String USGS_REQUEST_URL =
"http://earthquake.usgs.gov/fdsnws/event/1/query";
After reading greenapps comment, I just changed the protocol part in the string to https, so it became:
private static final String USGS_REQUEST_URL =
"https://earthquake.usgs.gov/fdsnws/event/1/query";
That solved the problem.
Thanks.
If you click your http link here you will see that the browser shows a https page. You better use that url directly as there is redirection now.
This is because the address http to https transferred.
To avoid this, you need to convert the request address to https.

post android simple app

I need a simple androd application, which POST a word to the server and recive the answer from server. I creat a form, which have two line and one button. In first line user write the word and click the button. In this time the app send this word to the server. When server send back response we show this respons in other line.
I right some code but this code don`t work on android 23. I also try to do this with retrofit, but I have varios problem with understending of this fiture.
I know that just need use POST, but don`t know how.
And can I send the ip adress of my android to the server?
Try this code:
It will also return the response from that page as a String
public static String postRequest(String post_url, String data) {
try {
URL url = new URL(post_url);
HttpURLConnection connection = (HttpURLConnection)url.openConnection();
BufferedReader br;
StringBuilder sb = new StringBuilder();
String line;
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setReadTimeout(10000);
Writer writer = new OutputStreamWriter(connection.getOutputStream());
writer.write(data);
writer.flush();
writer.close();
if (200 <= connection.getResponseCode() && connection.getResponseCode() <= 299) {
br = new BufferedReader(new InputStreamReader((connection.getInputStream())));
} else {
br = new BufferedReader(new InputStreamReader((connection.getErrorStream())));
}
while ((line = br.readLine()) != null) {
sb.append(line);
}
return sb.toString();
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
And call it by:
postRequest("http://your-url", "name=value&anothername=somevalue");
I think volley will fit your needs, it's a very simple library that can handle asynchronous requests :
https://developer.android.com/training/volley/index.html
"And can I send the ip adress of my android to the server?"
Sure, to get your android ip just use a code like this on server side, assuming you're using PHP :
<?
$ip = $_SERVER["REMOTE_ADDR"];
echo "<br />Your IP is : $ip";
?>
EDIT : NodsJS example
app.post('/getip', function (req, res) {
var ip = req.headers['x-forwarded-for'] ||
req.connection.remoteAddress ||
req.socket.remoteAddress ||
req.connection.socket.remoteAddress;
})

Android: extracting website using BufferedReader: Information extracted are outdated

I am reading html source code of a public website using the following code:
Code:
#Override
protected Void doInBackground(Void... params)
{
try
{
URL url = new URL(""+URL);
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
String inputLine;
PageCode = "";
OriginalPageCode = "";
while ((inputLine = in.readLine()) != null)
{
PageCode += inputLine;
}
OriginalPageCode = PageCode;
try
{
extract_website_and_save(); // extracting data from PageCode
}
catch (Exception e1)
{
}
in.close();
}
Background:
The above code sometimes can fetch the most updated website properly. But occasionally it linked to an outdated version of the website and hence unable to obtain the most updated information for the website.
I am curious why the above will occur, does it related to extracting from cache instead of the real updated website??
I therefore used Chrome to browse the same link, and discovered that Chrome also fetched the outdated website.
I have tried restarting the device, but the problem continues.
After 30 minutes to an hour, I requested the app to fetch again and it then can extract the most updated information. I at the same time browse the website using Chrome, Chrome can now obtain the most updated website.
Question:
The above BufferedReader should have no relationship with Chrome? But they follow the same logic and hence extracting from cache instead of from the most updated website?
I strongly suspect the end point is being cached by URL
Try something like this
urlSrt = urlSrt + "?x=" + new Random().nextInt(100000);
// If your URL already is passing parameters i.e. example.com?x=1&p=pass - then modify
// the urlSrt line to to use an "&" and not "?"
// i.e. urlSrt = urlSrt + "&x=" + new Random().nextInt(100000);
URL url = new URL(urlSrt);
URLConnection con = url.openConnection();
con.setUseCaches(false); //This will stop caching!
So if you modify your code to something like this.
URLConnection con = url.openConnection();
con.setUseCaches(false);
BufferedReader in = new BufferedReader(new InputStreamReader(
con.getInputStream()));

how to load a big webpage into a string

I'm a novice with Java and Android, but not to programming and HTTP. This HTTP GET method, mostly copied from other examples using the Apache HTTP classes, only retrieves the first few K of a large webpage. I checked that the webpage does not have lines longer than 8192 bytes (is that possible?), but out of webpages around 40K I get back maybe 6K, maybe 20K. The number of bytes read does not seem to have a simple realtionship with the total webpage size, or the webpage modulus 8192, or with the webpage content.
Any ideas folks?
Thanks!
public static String myHttpGet(String url) throws Exception {
BufferedReader in = null;
try {
HttpClient client = getHttpClient();
HttpGet request = new HttpGet();
request.setURI(new URI(url));
HttpResponse response = client.execute(request);
in = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer sbuffer = new StringBuffer("");
String line = "";
while ((line = in.readLine()) != null) {
sbuffer.append(line + "\n");
}
in.close();
String result = sbuffer.toString();
return result;
} finally {
if (in != null) {
try {
in.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
No need to write you own HttpEntity-to-String code, try EntityUtils instead:
// this uses the charset the server encoded the entity in
String result = EntityUtils.toString(entity);
It looks as if the problem is with pages from a certain website starting Goo... I'm not having this problem with large pages from other sites. So the code is probably OK.

Categories

Resources