This question is not for android programmers only , but also for whom interested in web pages design .
I would like to make an android app that renders some parts of specific web pages only (not all part of them) .
I am heard about jsoun library as a tool that does this task
My main problem is:-
How I can choose the correct link from web page's source that render some part of a web page ?.
For example let us take the famous website FORBES
How can I render the list of richest men by their name and Rank,Name net Worth,Change,Age,Source,Country of Citizenship as they appearthere excluding other parts of web page.
Here is a good example of an application that accomplishes like this task
You may have a good suggestion.
You need to screen-scrape the HTML. I'm not sure about any Android libraries for doing this, but I would build a RESTful service to return the data I needed. The service would than do the heavy lifting of scraping the webpage and converting the data to JSON to be sent back to device.
On the server side I would use a library like Beautiful Soup to do the scraping. It is easy enough to use once have it installed. You create a beautifulSoup object from the HTML and make calls like myObject.getTitle() to return the title of the HTML. You can use the tags in the HTML to drill down to the elements you want and build up a JSON object from there. Here is an image of the elements you are interested in for that list. Note the #ids on the right for that list item.
http://i.imgur.com/TMjhYvY.jpg
Related
My goal is to interact with a website (not mine), getting and posting data from it to my Android app coded using Kotlin. The interaction part is to be done in the background, as the result is to be shown in a RecyclerView in my app.
The website in question uses Knockout.js - the responsiveness and dynamically changing data makes it impossible to use libraries such as Jsoup for my goal at hand.
I am an aspiring App developer (n00b), and the question I have for the more senior devs here:
Is my project impossible? I have read it is "complex" to interact with a website that is dynamic, and I have also heard it is impossible. Is it? If not, could you guide me to the libraries I should be using? It is ok if these are in Java, I could probably look at adapting these to Kotlin.
If the site you need to extract data from produces a predictable result when you make a request to a URL then it would be easy to extract the data you need from it using a library like Jsoup which you've mentioned. Looking at the Jsoup docs that would be something like:
Document doc = Jsoup.connect("https://en.wikipedia.org/").get();
log(doc.title());
Elements newsHeadlines = doc.select("#mp-itn b a");
for (Element headline : newsHeadlines) {
log("%s\n\t%s",
headline.attr("title"), headline.absUrl("href"));
}
Where doc.select references an id in a given div (or other element) whose contents you're looking to extract.
Whether the site uses knockout or other JS library to help it render content shouldn't matter at all since all you're doing is parsing the string contents of the request--basically what you see when you view source in your browser. Knockout or any other script will have already run, doing its work in the rendering of the final HTML which you're going to parse with Jsoup.
But doing all of this is rather irregular as #Gushan indicates since normally unless you're doing some sort of scraping type of activity which would be weird for an android app, a site that wants to give you data and which you want to get data from will provide an API (usually some sort of REST API) that will simplify (document) how to go about getting that data. But I guess things aren't always like that. :)
I am looking to create an app that has about 50 pages of static content. I can give an example of what it would look like, so that it will be easy to understand the questions.
Imagine a Jokes app, with tens or hundreds of pages
The user can see a full list of jokes, which shows the headings in a list view
Selecting a joke subject will take them to the joke page
From there they can go 'Next' or 'Previous'
They should also be able to favorite a joke
Going to the Favorites pages, will list the favorites for them
The joke pages are static. I could add more jokes with an app update but there is no dynamic content. So I am planning to have any server side code that the app can call.
Now the questions:
In Android, can I achieve this with a single activity (for the joke display) and switch the content based on selection?
There are several to store the jokes - sqlite, separate html pages or just strings.xml. Which is better for these use cases?
If there are multiple headings within a single joke (i.e. formatting as bold for them to stand out), I need to store the formatting along with the content. So HTML looks like the option?
This may be out of scope, but I want to capture the content in a standard way so that if I build an iOS app for this, I can just worry about the UI part and use the same content. Again HTML is the option?
Thanks for looking.
Yes you can achieve this with a single activity.
This is really up to you but Android provides support for SQL Databases. You may also consider looking into content providers.
Note: I would not use strings.xml because you can't load new jokes into strings.xml. If you are getting your jokes dynamically from a website, then you really should either load your content into a database and have the app display from the database, or else just load each html page individually. The html page will be easier as you will basically just be making a browser app, but the database will certainly be faster and cleaner.
HTML is certainly an option, though this question seems a little bit vague. It really depends on how you want to get your jokes. If you want to grab them as HTML pages and just display them, then the work is done for you. If you want to parse through them and display them as an android specific app, then it will be more work but you have more control on the app side.
Yes if you want your app to work cross platform you can use HTML to standardize your view across multiple devices.
Is there a faster way to display Jsoup elements onto android apps?
The app I am working on download and parses a page using jsoup, gets a section of the page I want to display, edit some parts of it, then converts that section's element to HTML and displays it onto a webview.
Is there a faster way to do this? I noticed that getting the HTML from the elements takes a long time. Is there a way to directly add the element onto webview or something else without having to acquire its HTML?
Unfortunately, what you're experiencing is a side effect of trying to do (relatively) complex processing on a mobile device. Keep in mind that your app may be installed on low-resource devices and stuff like this screams low ratings.
Having said that, you usually want to do these sort of complex transformations on the server side, and then send the (formatted) data to the device - which you can then render in your WebView, for example. There is no inherent way to make jsoup faster - you're only limited by the resources of your device (as you already found out).
I have read the example for Rss Parsing from the ibm site.(http://www.ibm.com/developerworks/opensource/library/x-android/).
In this example,the rss are shown in a listview and then,if you press one announcement you can see it in the web browser of the device.How could i see them in the app,with no use of the device browser?
Thanks a lot
Create a layout with a WebView then load the URL from each "announcement" using WebView.loadUrl.
I'm a little confused but you seem to have answered your own question.
You say you don't want to use the web browser on the device but the example in your question doesn't use the browser. It does exactly what you're asking for.
The idea is that you download the html from the website and then use the parser to break it up into separate "announcements" and store them in list view items in your program.
I have done a bit of this type of thing myself in android. I used jsoup java library, which makes breaking the html into the bits you want to display really easy.
If you want some more help I can give you an example of an app I made that pulls movie times from google.com/movies as an example. here are links to the classes where I did the html download and parse:
ScreenScraper.java
HtmlParser.java
I am looking into developing an App that will convert a website into more readable data for an android app. I am at university and have an online notice board which can be viewed on the web but if possible I would like to transfer this into an app on android to make it more easy to read on mobile devices.
What I thinking is that the app would go to the website where the notice board is held and read in the html code to display each notice in a list adapter view. Each notice is within its own div so I assume I could use that to split each notice up into its own button on the list adapter view. Is this possible and if so how I can go about doing this. I have tried google for an answer but I have not yet found a solution to this problem.
Thanks for your help
It seems overly complicated to me. I wouldn't handle all that using Android. I'd crawl the data on a machine (server) and then I'd convert all needed data to JSON and have the Android (client) fetch the data using a simple JSON parser.
In my opinion that would be the easiest solution if you don't have access to the server the website is hosted on to get it generate a JSON feed for you directly.
EDIT: In answer to your comment Boardy.
Here is the official website of the JSON project in order to get an understanding of what it is. Then if you have access to the webserver providing that page (I assume it is a PHP based site) and want to modify or add the functionality of providing a JSON feed then you should also take a look at the PHP JSON documentation.
To parse JSON on Android check out this SO question and also don't forget to take a look at the official Android documentation on their JSON implementation.