I'm working on an Android admissions app and I'm trying to do a Khan connection into my app through the API with a basic search functionality where a user can enter a keyword and the app shows a list of topics / videos / exercises that match search criteria but I'm finding it almost impossible as API doesn't provide an official "Search by keyword" method, and as any who works with Khan may know its impossible to download the full topic tree as it ends in a 70mb file.
It's been many days of hard research and struggling my head but still wasn't able to find any documentation (if any) on how to deal with this so I thougth on asking for help. Does anyone have any idea on how to deal with this or anyone who successfully done it who can give me some hints on which direction to take?
I know you can do http://www.khanacademy.org/api/v1/topictree?kind=topic (for example) but even with this filter the json results are huge (more than 10mb stream of data).
This is the way I stream the json:
private static String connect2Url(String urlString){
URL url;
HttpURLConnection urlConnection = null;
InputStream inStream = null;
StringBuilder response = new StringBuilder();
try {
url = new URL(urlString);
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setRequestMethod("GET");
urlConnection.setDoInput(true);
urlConnection.connect();
inStream = urlConnection.getInputStream();
BufferedReader bReader = new BufferedReader(new InputStreamReader(inStream));
String temp = "";
while ((temp = bReader.readLine()) != null) {
response.append(temp);
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (inStream != null) {
try {
inStream.close();
} catch (IOException ignored) {
}
}
if (urlConnection != null) {
urlConnection.disconnect();
}
}
return response.toString();
}
And then I create the JSONObject with this result.
For the moment I'm passing "http://www.khanacademy.org/api/v1/topic/%s" as url but with this you can only locate topic by id (topic slug) but cannot make a free search.
Ok, I hope anyone can help, and thanks in advance.
Edit:
As per AlphaQ suggestion I'm already investigating about GSON and Jackson and of course agree with him my parsing function is not efficient for big json streaming but anyway I'm running into two problems, the first is the json structure returned by Khan Academy is quite complex to create a class of it (I think) and the second is even if I can make a more efficient parsing the more I'm being able to filter Khan Academy json response from server is resulting in 14mb of data (the full tree is 70mb) and I guess this is still too much for a mobile app regardless on how optimized my function is, because it will have to download 14mb anyway and Khan Academy API doesn't provide a "search by keyword" method.
Give me your opinion, don't you think 14mb is still too much json data to stream for a mobile app?
This is a very little extract from the json response as an example:
{
"icon_src": "",
"twitter_url": "",
"domain_slug": null,
"relative_url": "/",
"creation_date": "2016-12-14T00:46:38Z",
"web_url": "",
"ka_url": "https://www.khanacademy.org/",
"translated_standalone_title": "Khan Academy",
"has_user_authored_content_types": true,
"translated_title": "Khan Academy",
"gplus_url": "",
"children": [{
"icon_src": "",
"twitter_url": "",
"domain_slug": "new-and-noteworthy",
"relative_url": "/new-and-noteworthy",
"creation_date": "2016-09-13T20:35:19Z",
"web_url": "",
"ka_url": "https://www.khanacademy.org/new-and-noteworthy",
"translated_standalone_title": "New and noteworthy",
"has_user_authored_content_types": false,
"translated_title": "New and noteworthy",
"gplus_url": "",
"children": [],
"hide": false,
"node_slug": "new-and-noteworthy",
"title": "New and noteworthy",
"child_data": [{
"kind": "Video",
"id": "1101921746"
}, {
"kind": "Video",
"id": "796509722"
}, {
"kind": "Video",
"id": "796388970"
}, {
"kind": "Video",
"id": "753767269"
}, {
"kind": "Video",
"id": "225139731"
}, {
"kind": "Video",
"id": "225139729"
}, {
"kind": "Video",
"id": "988721865"
}, {
"kind": "Video",
"id": "884313452"
}, {
"kind": "Video",
"id": "671536496"
}, {
"kind": "Video",
"id": "887553535"
}, {
"kind": "Video",
"id": "1049279550"
}, {
"kind": "Video",
"id": "796535894"
}, {
"kind": "Video",
"id": "796307891"
}, {
"kind": "Video",
"id": "1094139389"
}, {
"kind": "Video",
"id": "796502981"
}, {
"kind": "Video",
"id": "796434657"
}, {
"kind": "Video",
"id": "871510490"
}, {
"kind": "Video",
"id": "884392300"
}, {
"kind": "Video",
"id": "x77e83a17"
}, {
"kind": "Video",
"id": "884395195"
}, {
"kind": "Video",
"id": "1077533640"
}, {
"kind": "Video",
"id": "1094227108"
}, {
"kind": "Video",
"id": "1094119956"
}, {
"kind": "Video",
"id": "1112847811"
}, {
"kind": "Video",
"id": "1114711148"
}, {
"kind": "Video",
"id": "1081957510"
}, {
"kind": "Video",
"id": "1095856196"
}, {
"kind": "Video",
"id": "1090948774"
}, {
"kind": "Video",
"id": "1013985489"
}, {
"kind": "Video",
"id": "xca214ba1"
}
],
"user_authored_content_types_info": [],
"id": "x29232c6b",
"user_authored_content_types": [],
"translated_description": "",
"alternate_slugs": [],
"standalone_title": "New and noteworthy",
"logo_image_url": "",
"in_knowledge_map": false,
"description": "",
"tags": [],
"deleted": false,
"listed_locales": [],
"facebook_url": "",
"render_type": "UncuratedTutorial",
"background_image_url": "",
"background_image_caption": "",
"has_peer_reviewed_projects": false,
"topic_page_url": "/new-and-noteworthy",
"extended_slug": "new-and-noteworthy",
"deleted_mod_time": "2013-07-13T00:03:09Z",
"kind": "Topic",
"curation": {},
"slug": "new-and-noteworthy",
"do_not_publish": false,
"sha": "a6c251bd225b9d23e1a98f1a7fce3ccd0c8cb4fa",
"branding_image_url": "",
"current_revision_key": "a6c251bd225b9d23e1a98f1a7fce3ccd0c8cb4fa",
"content_id": "x29232c6b",
"content_kind": "Topic",
"curriculum_key": ""
}, {
"icon_src": "",
"twitter_url": "",
"domain_slug": "math",
"relative_url": "/math",
"creation_date": "2017-05-08T20:15:34Z",
"web_url": "",
"ka_url": "https://www.khanacademy.org/math",
"translated_standalone_title": "Math",
"has_user_authored_content_types": false,
"translated_title": "Math",
"gplus_url": "",
"children": [{
"icon_src": "",
"twitter_url": "",
"domain_slug": "math",
"relative_url": "/math/k-8-grades",
"creation_date": "2017-03-17T23:26:26Z",
"web_url": "",
"ka_url": "https://www.khanacademy.org/math/k-8-grades",
"translated_standalone_title": "K-8th grades",
"has_user_authored_content_types": false,
"translated_title": "K-8th grades",
"gplus_url": "",
"children": [],
"hide": false,
"node_slug": "k-8-grades",
"title": "K-8th grades",
"child_data": [],
"user_authored_content_types_info": [],
"id": "xf3cb93f0",
"user_authored_content_types": [],
"translated_description": "",
"alternate_slugs": [],
"standalone_title": "K-8th grades",
"logo_image_url": "",
"in_knowledge_map": false,
"description": "",
"tags": [],
"deleted": false,
"listed_locales": ["fr", "en", "pt", "de", "tr", "pl", "nb", "es"],
"facebook_url": "",
"render_type": "Subject",
"background_image_url": "",
"background_image_caption": "",
"has_peer_reviewed_projects": false,
"topic_page_url": "/math/k-8-grades",
"extended_slug": "math/k-8-grades",
"deleted_mod_time": null,
"kind": "Topic",
"curation": {
"modules": [{
"content": ["Video:xd2b0fcb4", "Article:x175a9d66", "Video:x102d9798", "Exercise:2015", "Exercise:3022"],
"referrer": "k_8_grades_staff_picks",
"kind": "ContentCarousel",
"title": "Staff picks"
}, {
"topic": "Topic:xd0ae8a03",
"kind": "SubjectIntro"
}
],
"whitelist": ["Topic:xfe881476a971a3cb", "Topic:xb5feb28c", "Topic:xcef32ab6", "Topic:xef8f4cb4", "Topic:x3184e0ec", "Topic:xc7f617f2", "Topic:x5ec3eb59", "Topic:xb830458a", "Topic:x0267d782", "Topic:x6b17ba59", "Topic:x7c7044d7", "Topic:xa617314f", "Topic:x8708676b", "Topic:x7ed4701d", "Topic:x6ee1f3c2", "Topic:x0f2eb71b", "Topic:xa18e5391"],
"hide_community_questions": true
},
Edit 2:
I was reading all GSON documentation AlphaQ suggested me and checking out some examples but my json response is a little bit complex and I can't figure out how to define mapping class(es).
Below is the url from where I am getting the json.
Any help on how to define correctly mapping class(es) will be much appreciated.
http://www.khanacademy.org/api/v1/topictree
(this is without any filtering but I could add ?kind=topic to see a full list of topics, but no more filters are allowed)
The above is a 70mb stream so I don't think it will be possible even with GSON but for me would be anough if I can do this
http://www.khanacademy.org/api/v1/topic/math
to retrieve all math topics from this huge file.
As you see the json is too big and complex so I am not being able to figure out how to create mapping classes for this. I only need url, title and description of topics located under "children" nodes.
I will recommend you use GSON or Jackson like I said in comments.
You can use the hybrid streaming approach in Jackson to efficiently parse only the required data from the stream.
Here's an example of using GSON.
Here's a piece of info of what the guys at Google are saying:
Deserialized strings of over 25MB without any problems
More information here.
You can use http://json2csharp.com/ to create mapping classes for the json. If you have Visual Studio you can also copy the json and "Paste special" to create classes automatically in Visual studio.
I need to get the images array data how to put into the adapter and not required adapter code? Only loop and how to add that's important for me
and then after second I can access option_value array.
First all images can be accessed then after a option_value array.
{
"success": true,
"data": {
"id": "50",
"seo_h1": "",
"name": "Shirt 10001",
"manufacturer": "",
"sku": "",
"model": "10001",
"image": "http://api.yellowskydemo.com/image/cache/data/shirts/7228-500x500.jpg",
"images": [
"http://api.yellowskydemo.com/image/cache/data/shirts/13-500x500.jpg",
"http://api.yellowskydemo.com/image/cache/data/shirts/302-500x500.jpg",
"http://api.yellowskydemo.com/image/cache/data/shirts/5-500x500.jpg",
"http://api.yellowskydemo.com/image/cache/data/shirts/205-500x500.jpg"
],
"price": "$540.00",
"rating": 0,
"description": "<p>Fasten your fashion belts, for this is the ultimate edit of swanky casual wear by Ralph Lauren! For men who are born with a strong sense of style, the American powerhouse packs a colourful punch with this high-fash collection of shirts and sporty, monogrammed polos.</p>\r\n\r\n<p>Fasten your fashion belts, for this is the ultimate edit of swanky casual wear by Ralph Lauren! For men who are born with a strong sense of style, the American powerhouse packs a colourful punch with this high-fash collection of shirts and sporty, monogrammed polos.</p>\r\n",
"attribute_groups": [],
"special": "",
"discounts": [],
"options": [
{
"name": "Size",
"type": "select",
"option_value": [
{
"image": "http://api.yellowskydemo.com/image/cache/no_image-100x100.jpg",
"price": "$50.00",
"price_prefix": "-",
"product_option_value_id": "17",
"option_value_id": "55",
"name": "L-40",
"quantity": "99"
},
{
"image": "http://api.yellowskydemo.com/image/cache/no_image-100x100.jpg",
"price": "$40.00",
"price_prefix": "+",
"product_option_value_id": "18",
"option_value_id": "57",
"name": "XXL-44",
"quantity": "100"
}
],
"required": "1",
"product_option_id": "227",
"option_id": "14"
}
],
"minimum": "1",
"meta_description": "",
"meta_keyword": "",
"tag": "",
"upc": "",
"ean": "",
"jan": "",
"isbn": "",
"mpn": "",
"location": "",
"stock_status": "Out Of Stock",
"manufacturer_id": null,
"tax_class_id": "0",
"date_available": "2014-08-12",
"weight": "0.00000000",
"weight_class_id": "1",
"length": "0.00000000",
"width": "0.00000000",
"height": "0.00000000",
"length_class_id": "1",
"subtract": "0",
"sort_order": "1",
"status": "1",
"date_added": "2014-08-13 12:05:56",
"date_modified": "2015-06-30 11:19:39",
"viewed": "8",
"weight_class": "kg",
"length_class": "cm",
"reward": "0",
"points": "0",
"category": [
{
"name": "Shirts",
"id": "59"
},
{
"name": "Casual Shirts",
"id": "60"
}
],
"quantity": "99",
"reviews": {
"review_total": "0"
}
}
}
You should really look into How to parse JsonObject / JsonArray in android.
Please put your code of also what you have tried because people are here to help solve error/problem not to do coding.
Here is code from which you can Parse your json
JSONObject jsonObject = new JSONObject();
JSONObject dataObject = jsonObject.getJSONObject("data");
JSONArray imagesArray = dataObject.getJSONArray("images");
ArrayList<String> listOfImagesUrl = new ArrayList<>();
for(int i = 0; i < imagesArray.length(); i++)
{
listOfImagesUrl.add(imagesArray.getString(i)); // listOfImages
}
// code for option value
JSONArray optionsArray = dataObject.getJSONArray("options");
for(int i = 0; i < optionsArray.length(); i++)
{
JSONObject option = optionsArray.getJSONObject(i);
JSONArray optionsValueArray = option.getJSONArray("option_value");
for(int j = 0 ; j < optionsValueArray.length(); j++)
{
JSONObject optionValueObject = optionsValueArray.getJSONObject(j);
String image = optionValueObject.getString("image");
String price = optionValueObject.getString("price");
String price_prefix = optionValueObject.getString("price_prefix");
//same like this
}
}
For example : if the given below is json of the person's profile on facebook got thorugh facebook sdk login through android app , How we will get the School Name fron the education Field in th json data in android . Please Help
Data :
{
"id": "1464730016",
"name": "Ravi Tamada",
"first_name": "Ravi",
"last_name": "Tamada",
"link": "https://www.facebook.com/ravi8x",
"username": "ravi8x",
"birthday": "12/22/1988",
"hometown": {
"id": "112158005464147",
"name": "Baruva"
},
"location": {
"id": "102186159822587",
"name": "Chennai, Tamil Nadu"
},
"bio": "Author: www.androidhive.info\r\nCo-author: www.9lessons.info",
"work": [
{
"employer": {
"id": "179366562092719",
"name": "ByteAlly"
},
"location": {
"id": "102186159822587",
"name": "Chennai, Tamil Nadu"
},
"position": {
"id": "124917314217511",
"name": "Product Head"
}
]
}
],
"favorite_athletes": [
{
"id": "18620649907",
"name": "Virat Kohli"
}
],
"education": [
{
"school": {
"id": "131587206873093",
"name": "Raghu Engineering College (REC)"
},
"degree": {
"id": "140065339390579",
"name": "B.Tech"
},
"year": {
"id": "142963519060927",
"name": "2010"
},
"type": "Graduate School",
"classes": [
{
"id": "192259410803415",
"name": "2010",
"with": [
{
"id": "584960408",
"name": "Santosh Patnaik"
}
],
"from": {
"id": "584960408",
"name": "Santosh Patnaik"
}
}
]
}
],
"gender": "male",
"relationship_status": "Single",
"website": "www.androidhive.info\nwww.9lessons.info\nwww.twitter.com/ravitamada\nwww.about.me/rv",
"timezone": 5.5,
"locale": "en_US",
"languages": [
{
"id": "106059522759137",
"name": "English"
},
{
"id": "107617475934611",
"name": "Telugu"
},
{
"id": "112969428713061",
"name": "Hindi"
},
{
"id": "343306413260",
"name": "Tamil"
}
],
"verified": true,
"updated_time": "2012-03-02T17:04:18+0000"
}
JSONObject jsonResult = new JSONObject(jsonUser);
JSONArray data = jsonResult.getJSONArray("education");
if(data != null)
{
for(int i = 0 ; i < data.length() ; i++)
{
JSONObject c = data.getJSONObject(i);
String type = c.getString("type");
if(type.equalsIgnoreCase("college"))
{
JSONObject school = c.getJSONObject("school");
String id2 = school.getString("id");
String name2 = school.getString("name");
JSONObject year = c.getJSONObject("year");
String id_y = school.getString("id");
String name_y = school.getString("name");
}
}
}
Supposing that you've this json into a jsonObject that you retrieve as response, this is the way:
// Get jsonArray 'education' from main jsonObject
JSONArray jsonArrayEducation = jsonObject.getJSONArray("education");
JSONObject jsonSchool = jsonArrayEducation.getJSONObject("school");
Note that if you are interested only at the name, you can group the two lines above into
JSONObject jsonSchool = jsonObject.getJSONArray("education").getJSONObject("school");
// get school name
String schoolName = jsonSchool.getString("name");
How can i look into this Json structure and store each element id name link picutre nb_album nb_fan radio,type into the list. Here is my json file.
{
"data": [
{
"id": "1214294",
"name": "The Pop Rock Boys",
"link": "http://www.deezer.com/artist/1214294",
"picture": "https://api.deezer.com/2.0/artist/1214294/image",
"nb_album": 7,
"nb_fan": 3,
"radio": false,
"type": "artist"
},
{
"id": "813196",
"name": "Ringtone Pop Rock",
"link": "http://www.deezer.com/artist/813196",
"picture": "https://api.deezer.com/2.0/artist/813196/image",
"nb_album": 0,
"nb_fan": 4,
"radio": false,
"type": "artist"
},
{
"id": "4165034",
"name": "Rock of Pop",
"link": "http://www.deezer.com/artist/4165034",
"picture": "https://api.deezer.com/2.0/artist/4165034/image",
"nb_album": 1,
"nb_fan": 0,
"radio": false,
"type": "artist"
},
{
"id": "4022223",
"name": "instrumental / Pop / Rock",
"link": "http://www.deezer.com/artist/4022223",
"picture": "https://api.deezer.com/2.0/artist/4022223/image",
"nb_album": 0,
"nb_fan": 1,
"radio": false,
"type": "artist"
}
],
"total": 4
}
You're parsing the initial string as an array. It isn't an array, its an object that contains a single array named "data". You need to parse it into a JSONObject, then get the array named data from it, then loop like you do.
try {
JSONArray articlesArray = new JSONObject(jString).getJSONArray("data");
JSONObject aObject;
for(int i=0; i<articlesArray.length(); i++){
aObject=articlesArray.getJSONObject(i);
SearchTrack a=new SearchTrack();
a.setId(aObject.getString("id"));
a.setLink(aObject.getString("link"));
a.setName(aObject.getString("name"));
a.setPicture(aObject.getString("picture"));
a.setNbAlbum(aObject.getString("nb_album"));
a.setNbFan(aObject.getString("nb_fan"));
a.setRadio(aObject.getString("radio"));
a.setType(aObject.getString("type"));
articles.add(a);
}
}catch (JSONException e) {
e.printStackTrace();
}