Develop Reference

Search by keyword at Khan Academy - android

I'm working on an Android admissions app and I'm trying to do a Khan connection into my app through the API with a basic search functionality where a user can enter a keyword and the app shows a list of topics / videos / exercises that match search criteria but I'm finding it almost impossible as API doesn't provide an official "Search by keyword" method, and as any who works with Khan may know its impossible to download the full topic tree as it ends in a 70mb file.
It's been many days of hard research and struggling my head but still wasn't able to find any documentation (if any) on how to deal with this so I thougth on asking for help. Does anyone have any idea on how to deal with this or anyone who successfully done it who can give me some hints on which direction to take?
I know you can do http://www.khanacademy.org/api/v1/topictree?kind=topic (for example) but even with this filter the json results are huge (more than 10mb stream of data).
This is the way I stream the json:
private static String connect2Url(String urlString){
URL url;
HttpURLConnection urlConnection = null;
InputStream inStream = null;
StringBuilder response = new StringBuilder();
try {
url = new URL(urlString);
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setRequestMethod("GET");
urlConnection.setDoInput(true);
urlConnection.connect();
inStream = urlConnection.getInputStream();
BufferedReader bReader = new BufferedReader(new InputStreamReader(inStream));
String temp = "";
while ((temp = bReader.readLine()) != null) {
response.append(temp);
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (inStream != null) {
try {
inStream.close();
} catch (IOException ignored) {
}
}
if (urlConnection != null) {
urlConnection.disconnect();
}
}
return response.toString();
}
And then I create the JSONObject with this result.
For the moment I'm passing "http://www.khanacademy.org/api/v1/topic/%s" as url but with this you can only locate topic by id (topic slug) but cannot make a free search.
Ok, I hope anyone can help, and thanks in advance.
Edit:
As per AlphaQ suggestion I'm already investigating about GSON and Jackson and of course agree with him my parsing function is not efficient for big json streaming but anyway I'm running into two problems, the first is the json structure returned by Khan Academy is quite complex to create a class of it (I think) and the second is even if I can make a more efficient parsing the more I'm being able to filter Khan Academy json response from server is resulting in 14mb of data (the full tree is 70mb) and I guess this is still too much for a mobile app regardless on how optimized my function is, because it will have to download 14mb anyway and Khan Academy API doesn't provide a "search by keyword" method.
Give me your opinion, don't you think 14mb is still too much json data to stream for a mobile app?
This is a very little extract from the json response as an example:
{
"icon_src": "",
"twitter_url": "",
"domain_slug": null,
"relative_url": "/",
"creation_date": "2016-12-14T00:46:38Z",
"web_url": "",
"ka_url": "https://www.khanacademy.org/",
"translated_standalone_title": "Khan Academy",
"has_user_authored_content_types": true,
"translated_title": "Khan Academy",
"gplus_url": "",
"children": [{
"icon_src": "",
"twitter_url": "",
"domain_slug": "new-and-noteworthy",
"relative_url": "/new-and-noteworthy",
"creation_date": "2016-09-13T20:35:19Z",
"web_url": "",
"ka_url": "https://www.khanacademy.org/new-and-noteworthy",
"translated_standalone_title": "New and noteworthy",
"has_user_authored_content_types": false,
"translated_title": "New and noteworthy",
"gplus_url": "",
"children": [],
"hide": false,
"node_slug": "new-and-noteworthy",
"title": "New and noteworthy",
"child_data": [{
"kind": "Video",
"id": "1101921746"
}, {
"kind": "Video",
"id": "796509722"
}, {
"kind": "Video",
"id": "796388970"
}, {
"kind": "Video",
"id": "753767269"
}, {
"kind": "Video",
"id": "225139731"
}, {
"kind": "Video",
"id": "225139729"
}, {
"kind": "Video",
"id": "988721865"
}, {
"kind": "Video",
"id": "884313452"
}, {
"kind": "Video",
"id": "671536496"
}, {
"kind": "Video",
"id": "887553535"
}, {
"kind": "Video",
"id": "1049279550"
}, {
"kind": "Video",
"id": "796535894"
}, {
"kind": "Video",
"id": "796307891"
}, {
"kind": "Video",
"id": "1094139389"
}, {
"kind": "Video",
"id": "796502981"
}, {
"kind": "Video",
"id": "796434657"
}, {
"kind": "Video",
"id": "871510490"
}, {
"kind": "Video",
"id": "884392300"
}, {
"kind": "Video",
"id": "x77e83a17"
}, {
"kind": "Video",
"id": "884395195"
}, {
"kind": "Video",
"id": "1077533640"
}, {
"kind": "Video",
"id": "1094227108"
}, {
"kind": "Video",
"id": "1094119956"
}, {
"kind": "Video",
"id": "1112847811"
}, {
"kind": "Video",
"id": "1114711148"
}, {
"kind": "Video",
"id": "1081957510"
}, {
"kind": "Video",
"id": "1095856196"
}, {
"kind": "Video",
"id": "1090948774"
}, {
"kind": "Video",
"id": "1013985489"
}, {
"kind": "Video",
"id": "xca214ba1"
}
],
"user_authored_content_types_info": [],
"id": "x29232c6b",
"user_authored_content_types": [],
"translated_description": "",
"alternate_slugs": [],
"standalone_title": "New and noteworthy",
"logo_image_url": "",
"in_knowledge_map": false,
"description": "",
"tags": [],
"deleted": false,
"listed_locales": [],
"facebook_url": "",
"render_type": "UncuratedTutorial",
"background_image_url": "",
"background_image_caption": "",
"has_peer_reviewed_projects": false,
"topic_page_url": "/new-and-noteworthy",
"extended_slug": "new-and-noteworthy",
"deleted_mod_time": "2013-07-13T00:03:09Z",
"kind": "Topic",
"curation": {},
"slug": "new-and-noteworthy",
"do_not_publish": false,
"sha": "a6c251bd225b9d23e1a98f1a7fce3ccd0c8cb4fa",
"branding_image_url": "",
"current_revision_key": "a6c251bd225b9d23e1a98f1a7fce3ccd0c8cb4fa",
"content_id": "x29232c6b",
"content_kind": "Topic",
"curriculum_key": ""
}, {
"icon_src": "",
"twitter_url": "",
"domain_slug": "math",
"relative_url": "/math",
"creation_date": "2017-05-08T20:15:34Z",
"web_url": "",
"ka_url": "https://www.khanacademy.org/math",
"translated_standalone_title": "Math",
"has_user_authored_content_types": false,
"translated_title": "Math",
"gplus_url": "",
"children": [{
"icon_src": "",
"twitter_url": "",
"domain_slug": "math",
"relative_url": "/math/k-8-grades",
"creation_date": "2017-03-17T23:26:26Z",
"web_url": "",
"ka_url": "https://www.khanacademy.org/math/k-8-grades",
"translated_standalone_title": "K-8th grades",
"has_user_authored_content_types": false,
"translated_title": "K-8th grades",
"gplus_url": "",
"children": [],
"hide": false,
"node_slug": "k-8-grades",
"title": "K-8th grades",
"child_data": [],
"user_authored_content_types_info": [],
"id": "xf3cb93f0",
"user_authored_content_types": [],
"translated_description": "",
"alternate_slugs": [],
"standalone_title": "K-8th grades",
"logo_image_url": "",
"in_knowledge_map": false,
"description": "",
"tags": [],
"deleted": false,
"listed_locales": ["fr", "en", "pt", "de", "tr", "pl", "nb", "es"],
"facebook_url": "",
"render_type": "Subject",
"background_image_url": "",
"background_image_caption": "",
"has_peer_reviewed_projects": false,
"topic_page_url": "/math/k-8-grades",
"extended_slug": "math/k-8-grades",
"deleted_mod_time": null,
"kind": "Topic",
"curation": {
"modules": [{
"content": ["Video:xd2b0fcb4", "Article:x175a9d66", "Video:x102d9798", "Exercise:2015", "Exercise:3022"],
"referrer": "k_8_grades_staff_picks",
"kind": "ContentCarousel",
"title": "Staff picks"
}, {
"topic": "Topic:xd0ae8a03",
"kind": "SubjectIntro"
}
],
"whitelist": ["Topic:xfe881476a971a3cb", "Topic:xb5feb28c", "Topic:xcef32ab6", "Topic:xef8f4cb4", "Topic:x3184e0ec", "Topic:xc7f617f2", "Topic:x5ec3eb59", "Topic:xb830458a", "Topic:x0267d782", "Topic:x6b17ba59", "Topic:x7c7044d7", "Topic:xa617314f", "Topic:x8708676b", "Topic:x7ed4701d", "Topic:x6ee1f3c2", "Topic:x0f2eb71b", "Topic:xa18e5391"],
"hide_community_questions": true
},
Edit 2:
I was reading all GSON documentation AlphaQ suggested me and checking out some examples but my json response is a little bit complex and I can't figure out how to define mapping class(es).
Below is the url from where I am getting the json.
Any help on how to define correctly mapping class(es) will be much appreciated.
http://www.khanacademy.org/api/v1/topictree
(this is without any filtering but I could add ?kind=topic to see a full list of topics, but no more filters are allowed)
The above is a 70mb stream so I don't think it will be possible even with GSON but for me would be anough if I can do this
http://www.khanacademy.org/api/v1/topic/math
to retrieve all math topics from this huge file.
As you see the json is too big and complex so I am not being able to figure out how to create mapping classes for this. I only need url, title and description of topics located under "children" nodes.

I will recommend you use GSON or Jackson like I said in comments.
You can use the hybrid streaming approach in Jackson to efficiently parse only the required data from the stream.
Here's an example of using GSON.
Here's a piece of info of what the guys at Google are saying:
Deserialized strings of over 25MB without any problems
More information here.

You can use http://json2csharp.com/ to create mapping classes for the json. If you have Visual Studio you can also copy the json and "Paste special" to create classes automatically in Visual studio.

Related

I have a complex JSON which I need to parse and store the information that are inside the JSON data into Room database. Is it possible to store the whole JSON in Room? If that is possible I am up for that too. I am using Kotlin. Can anyone guide me to it?
Here's my JSON sample:
{
"chapters": [
{
"play": "manual",
"items": [
{
"image": "/path/to/image",
"background": "#ffffff",
"text": "text to show on image",
"points": 1,
"voice": {
"type": "tts"
},
"words": {
"type": "wav",
"file": "/path to words file",
"parts": [
{
"word": "text",
"from": 122,
"to": 116
}
]
}
},
{
"image": "/path/to/image",
"background": "#ffffff",
"text": "text to show on image",
"points": 1,
"voice": {
"type": "wav",
"file": "/path/to/file"
}
}
]
}
]
}

I am a newbie to android. I am stuck at JSON parsing.
Following is my JSON, in this, the type of menu categories will be dynamic, and I want to arrange this JSON in sectioned recycler view.
Please help me out.
This is my JSON
{
"menu_items": {
"Plat": [{
"menu_name": "jambon à l'os",
"menu_cost": "18.00",
"menu_count": "1",
"category": "Plat",
"menu_tax": "5.50",
"code": "€"
},
{
"menu_name": "tacos",
"menu_cost": "6.00",
"menu_count": "1",
"category": "Plat",
"menu_tax": "5.50",
"code": "€"
}
],
"Entrée": [{
"menu_name": "avocat",
"menu_cost": "7.50",
"menu_count": "1",
"category": "Entrée",
"menu_tax": "5.50",
"code": "€"
}]
},
"order_data": {
"order_id": "278",
"order_number": "REF-5d5e6b86",
"created_at": "1566468998",
"instructions": "",
"fullname": "Rohan Chitnis",
"phonenumber": "7610771871",
"useraddress": "Brest, France",
"image_path": "https://restau-at-home.bzh/uploads/attachments/Jellyfish_2019_08_21_10_49_21.jpg",
"wp": "pune",
"code": "€",
"promocode": null,
"promo_value": null,
"promo_type": null
},
"order_details": {
"total_items": "3",
"order_total": "31.50"
},
"promo_applied": "0",
"error": {
"code": "13",
"status": "200",
"message": "Ok"
}
}

Check this out. Create a generic response class. Extend your classes from that class. To do this properly you can look at Java Generics and Inheritance topics if you are using Java. And then search about Sectioned RecyclerView in Android. Here is a library for this.

My JSON contains only name of the food categories, Looks like below
[
{
"cu_id": "2",
"cuisine_type": "American"
},
{
"cu_id": "24",
"cuisine_type": "Sandwich"
},
{
"cu_id": "17",
"cuisine_type": "Seafood"
},
{
"cu_id": "29",
"cuisine_type": "Sports Bar"
},
{
"cu_id": "5",
"cuisine_type": "Steak"
},
{
"cu_id": "20",
"cuisine_type": "Sushi"
}
]
I want to get default image automatically from Internet by searching 'cuisine_type', How to do that?

You can use pixabay api it's a free RESTful api for searching and retrieving free images and videos released on Pixabay under Creative Commons CC0.
Sending a GET request like this :
https://pixabay.com/api/?key={ KEY }&q=query&image_type=photo
where :
Key : the api key you get after creating an account
q : the queried object you need to look for
image_type : what kind of data you need (picture, video)
returned data will be something like this :
{
"total": 4692,
"totalHits": 500,
"hits": [
{
"id": 195893,
"pageURL": "https://pixabay.com/en/blossom-bloom-flower-yellow-close-195893/",
"type": "photo",
"tags": "blossom, bloom, flower",
"previewURL": "https://static.pixabay.com/photo/2013/10/15/09/12/flower-195893_150.jpg"
"previewWidth": 150,
"previewHeight": 84,
"webformatURL": "https://pixabay.com/get/35bbf209db8dc9f2fa36746403097ae226b796b9e13e39d2_640.jpg",
"webformatWidth": 640,
"webformatHeight": 360,
"imageWidth": 4000,
"imageHeight": 2250,
"imageSize": 4731420,
"views": 7671,
"downloads": 6439,
"favorites": 1,
"likes": 5,
"comments": 2,
"user_id": 48777,
"user": "Josch13",
"userImageURL": "https://static.pixabay.com/user/2013/11/05/02-10-23-764_250x250.jpg",
},
{
"id": 14724,
...
},
...
]
}
for more detail visit this link

I decided I wanted to try using the Google shopping API out last week, but I had no idea how to parse JCON objects.
After much searching here I was able to get the product information for an item! However, I cannot narrow down to just a string in the product. So for example I want to just get the title of a product.
I have the following:
jsonString:
{
"kind": "shopping#products",
"etag": "\"GKsxsRlaBDslDpMe-MT1O7wqUDE/dMvQ5Pu2C806fWZJbNJ0GjdesJs\"",
"id": "tag:google.com,2010:shopping/products",
"selfLink": "https://www.googleapis.com/shopping/search/v1/public/products?country=US&restrictBy=gtin:051500240908&startIndex=1&maxResults=25",
"totalItems": 3,
"startIndex": 1,
"itemsPerPage": 25,
"currentItemCount": 3,
"items": [
{
"kind": "shopping#product",
"id": "tag:google.com,2010:shopping/products/7585088/9884865157760252836",
"selfLink": "https://www.googleapis.com/shopping/search/v1/public/products/7585088/gid/9884865157760252836",
"product": {
"googleId": "9884865157760252836",
"author": {
"name": "Southeastern Delivery",
"accountId": "7585088"
},
"creationTime": "2011-07-25T00:15:58.000Z",
"modificationTime": "2012-02-11T09:29:00.000Z",
"country": "US",
"language": "en",
"title": "Jif Peanut Butter, Creamy",
"description": "Jif Creamy Peanut Butter. Fresh Roasted Peanut Taste. Look for the flavor seal. Contains no preservatives. No refrigeration required.",
"link": "http://www.southeasterndelivery.com/Jif_Peanut_Butter_00051500240908/",
"brand": "Jif Peanut Butter",
"condition": "new",
"gtin": "00051500240908",
"gtins": [
"00051500240908"
],
"inventories": [
{
"channel": "online",
"availability": "inStock",
"price": 15.64,
"shipping": 1.56,
"currency": "USD"
}
],
"images": [
{
"link": "http://www.southeasterndelivery.com/images/ProductImages/00051500240908.jpg"
}
]
}
},
],
"requestId": "0CLGzkcKVo64CFRDd5wod4mMAAA"
}
I have the following code in my android app to parse it:
try {
JSONObject jsonObject = new JSONObject(jsonString);
JSONArray itemsArray = jsonObject.getJSONArray("items");
JSONObject productObject = itemsArray.getJSONObject(0);
//String productTitle = productObject.getString("title");
//tv.setText(productTitle);
tv.setText(productObject.toString());
} catch (JSONException e) {
// TODO Auto-generated catch block
tv.setText("JSONOBJECT Error: " + e);
e.printStackTrace();
}
setContentView(tv);
The TextView on my Android app will now display (obviously not indented I did that for easy reading):
"product": {
"googleId": "9884865157760252836",
"author": {
"name": "Southeastern Delivery",
"accountId": "7585088"
},
"creationTime": "2011-07-25T00:15:58.000Z",
"modificationTime": "2012-02-11T09:29:00.000Z",
"country": "US",
"language": "en",
"title": "Jif Peanut Butter, Creamy",
"description": "Jif Creamy Peanut Butter. Fresh Roasted Peanut Taste. Look for the flavor seal. Contains no preservatives. No refrigeration required.",
"link": "http://www.southeasterndelivery.com/Jif_Peanut_Butter_00051500240908/",
"brand": "Jif Peanut Butter",
"condition": "new",
"gtin": "00051500240908",
"gtins": [
"00051500240908"
],
"inventories": [
{
"channel": "online",
"availability": "inStock",
"price": 15.64,
"shipping": 1.56,
"currency": "USD"
}
],
"images": [
{
"link": "http://www.southeasterndelivery.com/images/ProductImages/00051500240908.jpg"
}
]
}
Now if you notice in my Java code I have two lines commented out. If I uncomment those lines and then comment the line:
tv.setText(productObject.toString());
tv gets set to this error: "JSONOBJECT Error: org.json.JSONEception: No value for title". I am not sure why this is true because clearly their is a title in the productObject.
Any help would be great!

I'm using json-smart in my project. It is very small and very fast. To convert your JSON from Sting to the actual object use JSONObjet json = (JSONObject)JSONValue.parse(rawString);
When you have JSONObject you treat it just like a Map. So String title = (String) json.get("title")

You are missing a level of information in your code. The JSON array contains items, which contains products, which have a title.
Your code should look like:
JSONObject jsonObject = new JSONObject(jsonString);
JSONArray itemsArray = jsonObject.getJSONArray("items");
JSONObject itemObject = itemsArray.getJSONObject(0);
JSONObject productObject = itemObject.getJSONObject("product");
String productTitle = productObject.getString("title");

I am getting the response
{
"returnCode": "0",
"message": "Sucessfully get credit card for value(1) ",
"token": "",
"CreditCard": {
"class": "CreditCard",
"id": 1,
"bankName": "NA",
"cardNumber": "1233435467789",
"ccvNumber": "3455",
"dateCreated": "2012-02-10T10:20:06Z",
"expiryDate": "2012-02-29T18:30:00Z",
"expiryDateStr": null,
"lastUpdated": "2012-02-10T10:20:06Z",
"securityCode": null,
"type": {
"enumType": "CreditCardType",
"name": "Visa"
},
"user": {
"class": "User",
"id": 4
}
}
}
I can't change server code, so how do i parse it. any helped..

your enum:
enum CreditCardType{
Visa, MasterCard, Diners
}
and while parsing, when you reach till type
//param is JSONObject
CreditCardType card = CreditCardType.valueOf(param.getString("name"));