Parsing FlightRadar24 and JSOUP - android

Hi guys i want parse a flight on flightradar24.com
I have tried with JSOUP and Android but results is null.
http://postimg.org/image/6hdmp4hgv/
I have read... JSOUP doesn't support dinamyc webpages.
There is a solution for this?
I want get Latitude, longitude, and more
Thank you in advance!

In this site the flight details are polled via JavaScript ajax calls. So after page load they invoke an ajax call to http://db8.flightradar24.com/zones/full_all.js?callback=pd_callback&_=1401126256649 to get the flight details. If we zoom into a particular part it uses a separate JavaScript file, say for Europe they use europe_all.js. This essentially returns a json containing all flights details including the speed the altitude etc. this is maintained as a key-value pair and key being the flight id and value an array of details.
First we need to get this json and then parse it to get flight id which is the key and then again invoke http://bma.fr24.com/_external/planedata_json.1.4.php?f=36c0ad6&callback=flight_data_service_cb&_=1401126256666 to get the details flight trails, the name start time, end time, status etc.. The trails is given as an array of latitude and longitude and the first two elements points the current position.
For both url the ending digit is the System.currentTimeMillis();. For the second url the argument "f" is actually the flight ID which is the key of first json. So the below program will parse these two json and give you the data.
I used full_all.js which gives all the flight information which is really huge. To limit the network call i put a break in the for loop. So this program only prints the details of first flight. If you remove the break you'll get all details of all flights but mid you that's like a 10000 calls.
The first json itself gives you enough information like the one given below. Its just one entry from first json and it says flight with id "36c0ae5", the registered key "0D05AD", current lat (25.54), lon (-99.24), speed 287, altitude 16650 ft, etc etc
"36c0ae5": [
"0D05AD",
25.54,
-99.24,
287,
16650,
354,
"0610",
"F-KBRO1",
"A320",
"XA-BIC",
1401129559,
"CUN",
"MTY",
"4O321",
0,
-1920,
"AIJ321",
0
]
Program
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.Map.Entry;
import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import com.google.gson.JsonParser;
public class FlightDetails {
public static void main(String[] args) throws Exception {
String allFlightsURL = "http://db8.flightradar24.com/zones/full_all.js?callback=pd_callback&_=" + System.currentTimeMillis();
String allFlightsJsonString = getJsonString(allFlightsURL);
JsonParser parser = new JsonParser();
JsonObject allFlightsJsonData = (JsonObject)parser.parse(allFlightsJsonString);
String singleFlightUrl = "http://bma.fr24.com/_external/planedata_json.1.4.php?f=###&callback=flight_data_service_cb&_=";
for(Entry<String, JsonElement> allFlightEntry : allFlightsJsonData.entrySet()){
StringBuilder urlBuilder = new StringBuilder(singleFlightUrl.replaceAll("###", allFlightEntry.getKey())).append(System.currentTimeMillis());
System.out.println(allFlightEntry.getKey() + " = " + allFlightEntry.getValue());
String singleFlightJsonString = getJsonString(urlBuilder.toString());
JsonObject singleFlightJsonData = (JsonObject)parser.parse(singleFlightJsonString);
for(Entry<String, JsonElement> singleFlightEntry : singleFlightJsonData.entrySet()){
System.out.println(singleFlightEntry.getKey() + " = " + singleFlightEntry.getValue());
}
break; // Breaking to avoid huge network calls.
}
System.out.println("Done");
}
private static String getJsonString(String allFlightsURL) throws IOException {
HttpURLConnection connection = (HttpURLConnection) ((new URL(allFlightsURL).openConnection()));
connection.setDoOutput(true);
connection.setRequestProperty("Content-Type", "application/json");
connection.setRequestProperty("Accept", "application/json");
connection.setRequestMethod("GET");
connection.connect();
BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
StringBuilder buffer = new StringBuilder();
String line;
while ((line = reader.readLine()) != null) {
buffer.append(line);
}
return buffer.substring(buffer.indexOf("(") + 1, buffer.lastIndexOf(")"));
}
}

you might want to read in the forum:
http://forum.flightradar24.com/threads/24-API-access

Related

How to deal with a large JSON object on Android

I'm making an Android app that runs a ASP.NET WebService. Webservice sends a JSON object and app parses the object and displays on the screen. In one case, JSON object is too big and I get Failed Binder Transaction error. My solution is to get that JSON object and embed it in the app code, so that it wouldn't need to get that JSON object from the server. Can you tell any other things that I can do for this problem?
Or can you tell me how to get that JSON object from Webservice? Thanks.
Sending the large size data from server to mobile. JSON is light weight.
If you want to pass the data using more efficient way then passes it in pagination.
If you want to use more lighter protocol than JSON then implement the below google protocol which are really useful, which are supporting major languages.
Below are smaller Serialised data structure. Google's data interchange protocol.
1.Google Protocol
2.Flat Buffers
3.Nano-proto buffers
Hope this will be useful you.
If data is large then try to save it in the database, then deal with it using SQLite. (but not recommended if its dynamic)
To parse json object use gson or jackson. This will help reduce the memory consumption significantly as the json data being parsed partially.
get Gson, jackson here
https://sites.google.com/site/gson/gson-user-guide
http://jackson.codehaus.org/
A jackson example
http://www.mkyong.com/java/jackson-streaming-api-to-read-and-write-json/
First thing: If there is a crash or exception in your code, you'll probably want to post that. "Failed Binder Exception" is a bit too vague to understand what you're doing.
If you really want to ship your Android app with JSON embeddd inside it (to avoid having to fetch it from a server, consider storing it as an asset and access it using AssetManager. You basically drop the file with the json in your app's assets folder and read them out with AssetManager.
If you still want to download it from the server and act on it, consider using streaming APIs to download and parse the JSON. Android's JSONObject does not do this and it insists on having the entire JSON string in memory before it can be parsed.
If you want to stream directly from a URL download into a streaming parser (such as GSON), try something along these lines. First get an InputStream from the URL you're trying to fetch:
URL u = new URL(url);
URLConnection conn = u.openConnection();
InputStream is = new BufferedInputStream(conn.getInputStream());
Then feed that InputStream directly to your streaming parser. This should prevent the need to pull the entire response into memory before parsing, but you'll still need enough memory to contain all the objects that the parser creates:
GsonBuilder gb = new GsonBuilder(); // configure this as necessary
Gson gson = gb.create();
final Result response = gson.fromJson(
new InputStreamReader(is, Charset.forName("UTF-8")),
Result.class
);
"Result" here is a class that will contain the data from the JSON response. You'll have to make sure all the mappings work for your data, so read up on GSON and do whatever works for your case.
You can also use GSON to parse the JSON data if you store it in an asset. Just hand it the InputStream of the asset data and it works the same way.
The following class ApiUrlClass.java has all methods you require. Please read the comments of the class which I wrote. That will help you to do what you require. This also utilises transparent.
import android.graphics.Bitmap;
import android.net.Uri;
import android.os.Build;
import android.util.Log;
import org.apache.http.entity.mime.HttpMultipartMode;
import org.apache.http.entity.mime.content.ByteArrayBody;
import org.apache.http.entity.mime.content.ContentBody;
import org.apache.http.entity.mime.content.StringBody;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.ByteArrayOutputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.net.HttpURLConnection;
import java.net.SocketTimeoutException;
import java.net.URL;
import java.net.UnknownHostException;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLPeerUnverifiedException;
/*
Usage of the class
Create all the necessary API Call methods you need.
And either use a Thread or AsyncTask to call the following.
JSONObject response = ApiUrlCalls.login("username", "passowrd");
After the response is obtained, check for status code like
if(response.getInt("status_code") == 200){
//TODO: code something
} else {
//TODO: code something
}
*/
public class ApiUrlCalls {
private String HOST = "https://domain/path/"; //This will be concated with the function needed. Ref:1
/*
Now utilizing the method is so simple. Lets consider a login function, which sends username and password.
See below for example.
*/
public static JSONObject login(String username, String password){
String functionCall = "login";
Uri.Builder builder = new Uri.Builder()
.appendQueryParameter("username", username)
.appendQueryParameter("password", password);
/*
The return calls the apiPost method for processing.
Make sure this should't happen in the UI thread, orelse, NetworkOnMainThread exception will be thrown.
*/
return apiPost(builder, functionCall);
}
/*
This method is the one which performs POST operation. If you need GET, just change it
in like Connection.setRequestMethod("GET")
*/
private static JSONObject apiPost(Uri.Builder builder, String function){
try {
int TIMEOUT = 15000;
JSONObject jsonObject = new JSONObject();
try {
URL url = null;
String response = "";
/*
Ref:1
As mentioned, here below, in case the function is "login",
url looks like https://domain/path/login
This is generally a rewrited form by .htaccess in server.
If you need knowledge on RESTful API in PHP, refer
http://stackoverflow.com/questions/34997738/creating-restful-api-what-kind-of-headers-should-be-put-out-before-the-response/35000332#35000332
I have answered how to create a RESTful API. It matches the above URL format, it also includes the .htaccess
*/
url = new URL(HOST + function);
HttpsURLConnection conn = null;
conn = (HttpsURLConnection) url.openConnection();
assert conn != null;
conn.setReadTimeout(TIMEOUT);
conn.setConnectTimeout(TIMEOUT);
conn.setRequestMethod("POST");
conn.setDoInput(true);
conn.setDoOutput(true);
String query = builder.build().getEncodedQuery();
OutputStream os = conn.getOutputStream();
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(os, "UTF-8"));
writer.write(query);
writer.flush();
writer.close();
os.close();
conn.connect();
int responseCode = conn.getResponseCode();
String responseMessage = conn.getResponseMessage();
jsonObject.put("status_code", responseCode);
jsonObject.put("status_message", responseMessage);
/*The if condition below will check if status code is greater than 400 and sets error status
even before trying to read content, because HttpUrlConnection classes will throw exceptions
for status codes 4xx and 5xx. You cannot read content for status codes 4xx and 5xx in HttpUrlConnection
classes.
*/
if (jsonObject.getInt("status_code") >= 400) {
jsonObject.put("status", "Error");
jsonObject.put("msg", "Something is not good. Try again later.");
return jsonObject;
}
String line;
BufferedReader br = new BufferedReader(new InputStreamReader(conn.getInputStream()));
while ((line = br.readLine()) != null) {
response += line;
}
//Log.d("RESP", response);
/*
After the actual payload is read as a string, it is time to change it into JSON.
Simply when it starts with "[" it should be a JSON array and when it starts with "{"
it is a JSONObject. That is what hapenning below.
*/
if(response.startsWith("[")) {
jsonObject.put("content", new JSONArray(response));
}
if(response.startsWith("{")){
jsonObject.put("content", new JSONObject(response));
}
} catch(UnknownHostException e) {
//No explanation needed :)
jsonObject.put("status", "UnknownHostException");
jsonObject.put("msg", "Check your internet connection");
} catch (SocketTimeoutException){
//This is when the connection timeouts. Timeouts can be modified by TIMEOUT variable above.
jsonObject.put("status", "Timeout");
jsonObject.put("msg", "Check your internet connection");
} catch (SSLPeerUnverifiedException se) {
//When an untrusted SSL Certificate is received, this happens. (Only for https.)
jsonObject.put("status", "SSLException");
jsonObject.put("msg", "Unable to establish secure connection.");
se.printStackTrace();
} catch (IOException e) {
//This generally happens when there is a trouble in connection
jsonObject.put("status", "IOException");
jsonObject.put("msg", "Check your internet connection");
e.printStackTrace();
} catch(FileNotFoundException e){
//There is no chance that this catch block will execute as we already checked for 4xx errors
jsonObject.put("status", "FileNotFoundException");
jsonObject.put("msg", "Some 4xx Error");
e.printStackTrace();
} catch (JSONException e){
//This happens when there is a troble reading the content, or some notice or warnings in content,
//which generally happens while we modify the server side files. Read the "msg", and it is clear now :)
jsonObject.put("status", "JSONException");
jsonObject.put("msg", "We are experiencing a glitch, try back in sometime.");
e.printStackTrace();
} return jsonObject;
} catch (JSONException e) {
e.printStackTrace();
}
return null;
}
}
You could embed your JSON in your app's code as you suggested, but this will be a bad approach if the JSON is dynamic. Then you would need to push an update for your app whenever the JSON changes.
A better solution would be to paginate the JSON that you generate from your WebService, i.e., break the JSON into smaller parts that you can fetch sequentially in separate API calls.
Preferably try to break the Json Object to smaller object and get from webService ,
or get data in parts and if u cant do that
You have to use a streaming JSON parser.
For Android u can use these 2:
GSON
Jackson
GSON Streaming is explained at: https://sites.google.com/site/gson/streaming
I personally like Gson .

How to get restaurant menus from Locu API

I have looked at several APIs for acquiring restaurant menu for a particular location and determined that Locu API works best for me.
I was trying the basic example listed on locu website:
curl -X POST https://api.locu.com/v2/venue/search/ -d '{"fields":["name","menu_items","location","categories","description"],"menu_item_queries":[{"price":{"$lt":6},"name":"burrito"}],"venue_queries":[{"location":{"locality":"San Francisco"}}],"api_key":"MY_API_KEY"}'
Where, MY_API_KEY is the API_KEY that I received when I signed up.
As long as I include "menu_items" parameter, I keep getting the response:
{"status": "error", "http_status": 400, "error": "The requested \"menu_items\" field is either invalid or your account does not have permissions to access it."}
I did not come across any documentation regarding what I need to do in order to get the permissions for querying "menu_items". If anyone could point me in the right direction, I will really appreciate that.
I have already gone through some relatively old questions on here and they did not address this particular issue.
Also, there doesn't seem to be a tag for Locu api here. I am going to try and tag the question with some generic tags. Please excuse me for that.
I know this is an old question, but I've found that the solution is to request the "menus" field in a venue search. The API will return the "menu_items" as sub-objects of the menus.
`
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
public class NetClientGet {
public static void main(String[] args) {
try{
String result;
URL url = new URL("https://api.locu.com/v1_0/venue/search/?api_key=" +apiKey);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setRequestProperty("Accept", "application/json");
if (conn.getResponseCode() != 200) {
throw new RuntimeException("Failed : HTTP error code : " + conn.getResponseCode());
}
BufferedReader br = new BufferedReader(new InputStreamReader(
(conn.getInputStream())));
String output;
System.out.println("Output from Server .... \n");
while ((output = br.readLine()) != null) {
System.out.println(output);}
}
conn.disconnect();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}`

google calculator results for android

I am making an android application which requires to send a mathematical question like 1+1 to google's calculator and I need to get that result which is displayed on the web. How can I achieve this on android?
One possibility is to create a URL for the equation you are trying to calculate and then use a URLConnection to open the URL and read the webpage source code to find the answer to the equation.
For example if you have the equation:
2+2
Then the URL to calculate the result with the Google Chrome calculator would be:
https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=2%2B2
You will have to construct the proper query in the URL for the equation you are solving. In this URL the query at the end has the equation 2+2:
q=2%2B2 (where the %2B represents the + sign)
After constructing the URL open it with a URLConnection and read the source. The answer to the equation will be in this element:
<span class="cwcot" id="cwos">4</span>
So you can parse the source in order to find that particular span element and retrieve the result of your equation.
This is probably more work than you expected but it is the only solution I can think of to accomplish what you asked. Also, this approach may be error prone and may break easily. I would consider using a different approach altogether such as launching an intent to use the calculator app on the mobile device (even though this approach has issues as well).
EDIT:
This worked for me (it will output: 2 + 2 = 4):
public static void test() {
try {
String source = getUrlSource();
String span = "<span class=\"nobr\"><h2 class=\"r\" style=\"display:inline;font-size:138%\">";
int length = span.length();
int index = source.indexOf(span) + length;
String equation = source.substring(index, source.indexOf("<", index));
System.out.println( "equation: " + equation);
} catch (IOException e) {
e.printStackTrace();
}
}
private static String getUrlSource() throws IOException {
String url = "https://www.google.com/search";
String charset = "UTF-8";
String param1 = "2+2";
String query = String.format("?q=%s", URLEncoder.encode(param1, charset));
HttpsURLConnection urlConn = (HttpsURLConnection)new URL(url + query).openConnection();
urlConn.setRequestProperty("User-Agent", "Mozilla/5.0");
urlConn.setRequestProperty("Accept-Charset", charset);
BufferedReader in = new BufferedReader(new InputStreamReader(urlConn.getInputStream()));
String inputLine;
StringBuilder a = new StringBuilder();
while ((inputLine = in.readLine()) != null)
a.append(inputLine);
in.close();
return a.toString();
}

How to fetching content from an URL and parse json

I have 2 TextViews in my layout with id's (matricula, nome) and i need get this values from this json request.
I have difficults in both make json request as in get values, here is an example how would i do in php and jquery:
PHP
$alunos = json_decode("let's pretend json data is here");
echo "Matricula: " . $alunos['Aluno']['matricula'];
echo "Nome: " . $alunos['Aluno']['nome'];
Jquery
var alunos = $.parseJSON("let's pretend json data is here");
console.log("Matricula: " + alunos.aluno.matricula);
console.log("Nome: " + alunos.aluno.nome);
To help:
Aluno = Student
Matricula = Student id
Nome = name
I read some answers here about parsing json but i admit it, its hard to understand.
It is also easy in Java (I left out all error handling to focus on the main flow, please add that yourself):
import org.json.JSONObject;
import java.net.URL;
import java.net.HttpURLConnection;
import java.io.InputStream;
import java.io.InputStreamReader;
...
private String readString(Reader r) throws IOException {
char[] buffer = new char[4096];
StringBuilder sb = new StringBuilder(1024);
int len;
while ((len = r.read(buffer)) > 0) {
sb.append(buffer, 0, len);
}
return sb.toString();
}
...
// fetch the content from the URL
URL url = new URL("http://..."); // add URL here
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
InputStreamReader in = new InputStreamReader(conn.getInputStream(), "UTF-8");
String jsonString = readString(in);
in.close();
conn.disconnect();
// parse it and extract values
JSONObject student = new JSONObject(jsonString);
String id = student.getJSONObject("Aluno").getString("matricula");
String name = student.getJSONObject("Aluno").getString("nome");
For details please see the documentation.

Json call making in android

I am new bee in Android , so the knowledge regarding android is not so vast.
I am trying to implement Json call in android and i am using the foolowing code to get the list of all the contacts in the database.
package com.example.library;
import java.io.BufferedReader;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.UnsupportedEncodingException;
public class SecondActivity extends Activity {
Button show_data;
JSONObject my_json_obj;
String path,firstname,lastname;
{
path = "http://192.168.71.129:3000/contacts";
HttpClient client = new DefaultHttpClient();
HttpConnectionParams.setConnectionTimeout(client.getParams(), 10000);
HttpEntity entity;
HttpResponse response = null;
HttpURLConnection urlconn;
my_json_obj = new JSONObject();
}
}
I dont know if this is the right method but this code was already existing in another project and i have just made some change.
Please guide me through this one as i have gone through many stackoverflow and google answers,but it is very confusing as i am just a beginner and dont have knowledge of json calls in android.
I could give you a chunk of code and say "Hey try this", but like you stated that you are very new to Android so I simply wont.
I think its of more value that you can learn something beter by trying then simply copy pasting code(most of the time)
There are a couple of things you need to consider when you do network request and parsing data.
Network request you must always do this in a seperate thread then the UI thread, because if you dont youll get a NetworkOnMainUiThreadException if I am correct out the top of my head.
The same applies for parsing the data you have retrieved from your request.
I dont see any parsing of data in your current code but I just wanted to give you a headsup because you will prob do this at some point in your application.
Here you can find a tutorial how to do threading with the AsyncTask. this is "the way" how it should be done in Android, they realy made it easy for you.
When reading that tutorial you will get the basic knowlage to do stuff in this class.
When you get the concept of threading and how to work with this newly added skill I would suggest reading and following up on this json tutorial here.
I hope this helps
try this, result variable has your responce
try {
HttpClient client = new DefaultHttpClient();
HttpGet request = new HttpGet("paset_your_url_here");
HttpResponse response = client.execute(request);
BufferedReader in = new BufferedReader
(new InputStreamReader(response.getEntity().getContent()));
StringBuffer sb = new StringBuffer("");
String line = "";
String NL = System.getProperty("line.separator");
while ((line = in.readLine()) != null) {
sb.append(line + NL);
}
in.close();
result = sb.toString();
Log.i("", "-----------------------"+result);
} catch(Exception e) {
e.printStackTrace();
}finally {
if (in != null) {
try {
in.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
if you want to prase json then first do googling and if you get your answer by this then vote up :)

Categories

Resources