Develop Reference

Android: Extract article main content

Currently I am creating an Android application which allows to extract main content and picture from a website. Now I am using Jsoup API to extract all p tags from the HTML. However, it is not a good solution. Any suggestion or better solution enable me to extract main content and picture from a website in Android?

I didn't find anything that works for me, so I published Goose for Android, here: https://github.com/milosmns/goose
Some description follows...
Document cleaning
When you pass a URL to Goose, the first thing it starts to do is clean
up the document to make it easier to parse. It will go through the
whole document and remove comments, common social network sharing
elements, convert em and other tags to plain text nodes, try to
convert divs used as text nodes to paragraphs, as well as do a general
document cleanup (spaces, new lines, quotes, encoding, etc).
Content / Images Extraction
When dealing with random article links you're bound to come across the
craziest of HTML files. Some sites even like to include 2 or more HTML
files per site. Goose uses a scoring system based on clustering of
English stop words and other factors that you can find in the code.
Goose also does descending scoring so as the nodes move down - the
lower their scores become. The goal is to find the strongest grouping
of text nodes inside a parent container and assume that's the relevant
group of content as long as it's high enough (up) on the page.
Image extraction is the one that takes the longest. Trying to find the
most important image on a page proved to be challenging and required
to download all the images to manually inspect them using external
tools (not all images are considered, Goose checks mime types,
dimensions, byte sizes, compression quality, etc). Java's Image
functions were just too unreliable and inaccurate. On Android, Goose
uses the BitmapFactory class, it is well documented, tested, and is
fast and accurate. Images are analyzed from the top node that Goose
finds the content in, then comes a recursive run outwards trying to
find good images - Goose also checks if those images are ads, banners
or author logos, and ignores them if so.
Output Formatting
Once Goose has the top node where we think the content is, Goose will
try to format the content of that node for the output. For example,
for NLP-type applications, Goose's output formatter will just suck all
the text and ignore everything else, and other (custom) extractors can
be built to offer a more Flipboardy-type experience.

Why do you think it's not a good solution to use Jsoup?
I've written many web scrapers for different webpages, and in my experience Jsoup is the way to go for that task. You should study the Jsoup Syntax it is very powerful and with the right selectors you could extract most information from HTML documents very easy. Generally it becomes harder to extract information when the document has no id, class attributes or other unique features.
Other HTML parsers that might be interesting for you are JTidy and TagSoup

You could try the textracto api it automatically identifies the main content of HTML documents. There is also the opportunity to parse OpenGraph meta data, therefore you were also able to extract a picture (og:image).

Cross Device Rich Text Formatting

We're trying to work on a feature where users can enter rich text (Bold, numbered/bulleted lists, underlines, links) for content, but this content needs to be displayed in multiple places:
On browser based websites/apps.
In native iOS/Android apps.
What's the best format to use for storing this data? We are considering storing the text as html, but is this adequate, or is there a better solution/option to storing this data and rendering it?
Can native iOS/Android apps display html content well in native apps amidst other native controls, or will this text have to be converted to a different format to render well?
Thanks!

Check for input in webview

I have an app that has a web-view which has a basic web-form that has a few fields and a submit button. I would like to figure out in my app if the form has any input in any of the fields. I cannot change the form from the server side, and I can't be certain much about the fields (ids / names in the html).
In iOS we accomplish this with an interesting process of pulling all the html out when loading the form, and comparing it to the html at any given point, if they don't match, the user must have entered something into a field. I believe we were able to get the html by injecting and running some javascript into the web-view. I'm not sure exactly how to approach the problem on android, or if android has any better tools to get whether a form has been edited.
Anybody have any ideas / pseudo-code how I can tell if a form has had input in any of the fields in a webview in android?

Unfortunately, there are no special form-related tools in Android WebView either. You can use the same approach as you have described for iOS.
A couple of links to get you started:
Read HTML content of webview widgets
Android Web-View : Inject local Javascript file to Remote Webpage

Best way to display large amount of text in an Android App

I'm relatively new to mobile app development - I'm kinda learning as I go. I'm creating an app that will serve multiple purposes - notifications, audio/video, etc. One of the features of the app will be to display the contents of an unpublished book (no plans to publish it either via the traditional methods available today). Essentially, I want the part of the app to do teh following:
1) Have a menu which will server as a table of contents.
2) Display the text, which will be in English and Arabic.
3) Have the english text searchable.
4) Have the ability to favorite certain sections of the text.
Just wondering what's the best way to build this? Should I convert sections of my file to html and use webview? Or should I use textview?
I'm looking for the option that gives me the most robustness in terms of functionality, and flexibility when it comes to design (i.e. background images, custom fonts, formatting).
Thanks in advance.

WebView or HTML page is not a very good approach.
You can try an approach in which your data is stored in json format in your resources-->raw folder and then parse each element of the JsonObject to populate views dynamically.(If you have server then you fetch data via HttpConnection). For start you can see here

convert your file to html file and display it using webview container...If you have your book in word format then convert it to html file in any website and then display it using webview.

How can I search content in HTML and not the tags

I have a database of content of which the majority are HTML pages which are then used for display purposes in an app.
We are looking to build out a search feature but I have some concerns over false positives appearing due to the results including HTML code.
E.g searching for "title" will return any content pages which have a title html tag
We are currently using NSPredicates to perform the query on a Core Data database.
Are there any easy/efficient ways to prevent these results being returned?
I have the same problem on Windows and Android as well!

One idea for iOS is to actually store a separate a text version apart from the HTML version. You could then use very simple (even if not very efficient) predicates lie
[NSPredicate predicateWithFormat:#"text CONTAINS[cd] %#", searchText];
A more performant way would be to strip out the words and store them in lowercase in an indexed attribute of another entity.
In both cases, the parsing should be done beforehand via one of the available libraries (see e.g. link in the comment).