After much frustration I need some help getting a HTML parser for Android.
I basically need a parser which will log into a website and retrieve the main page, which heavily uses javascript and ajax to render data. From there I need to be able to access other pages which are linked with anchor tags. Again, emphasis on the Ajax and javascript on the page, all the data is created through these means.
EDIT: After some research it looks like I need a headless browser that is compatible with Android.
At first I tried with JSoup and it only retrieves the page without the javascript and ajax data being written in. Unless I'm understanding something wrong here, JSoup is not fitting in this situation.
I have also tried HtmlUnit, which works perfectly, but I am unable to get it up and running in android because of the known jar conflicts and the "not being able to convert to dalivik" error.
If anyone knows another suitable HTML parser, or can help me with either getting JSoup and ajax pages to play nice, or knows how to get HtmlUnit running on android, I would be forever in your debt.
I have already spent the majority of a week with JSoup and HtmlUnit, and basically have gone back to square one. Frustration has set in and I'm completely lost on what to do.
Again, I need a HTML parser, that will allow Ajax and Javascript to load before I can parse, and it needs to be android friendly!
Thank you!
You can try to intercept the individual Ajax communications. They are either get or post communication. And you can call them with JSoup one by one. It is feasible for a handful of Ajax call. In fact there are less data and can be more quicker to parse on Android this way.
Related
I'm new to programming and I would like to display data from this site into my activity. Simple textviews with name and numbers nothing more. The problem is I don't know how to get those info from this site. When I open it with Firefox and try to inspect code, I can't find any variables from which it could be read, if I'm saying this correct!? :)
Can someone point me to right direction???
This is the web page:
http://parking-servis.co.rs/eng/where-to-park/
Like the other user mentioned this is too broad, but I can give you a few pointers and things you can start reading up on.
First: You want to get info from a website.
This can be easily done via web scraping. What this means is you read through the HTML code of a webpage and extract information from it. Libraries such as Jsoup will help you a lot here. They have great documentation so that should get you going. You can just hover over any HTML element to get its related information when the element is highlighted.
Second:You need to get this information to your App.
Well you are in luck because Jsoup has its own android port and you can simply import the library to your project and go ahead from there. You can write a simple flow that scrapes the website every day or e erytime the app is opened. You can pass net requests via something called AsysncTask.
So the library to start looking into is Jsoup and you should also read up on AsyncTask. Besides these, there are other links you might be interested in :
Web Parsing with JSoup
JSoup on Android
AsyncTask Basic Tutorial
This is a fairly complicated task, if the website doesn't provide an API you will have to web scrape to get the information off of that page. as for getting the data into android studio you will have to research how to use GET / POST calls to (hopefully) an API with that information.
Not much else I can do as this question is very broad, but hope it has helped.
I've looked at this doc: http://developer.android.com/guide/webapps/webview.html
It seems it might be attractive strategy to use HTML/Javascript to get:
result faster (I can use my browser to create a prototype very quickly, just updating my page in the browser, and not waiting for emulator to be update my changes)
avoid to to learn Android API too deep (but use what I know HTML/CSS/JS instead)
reuse my code, putting it to iOS, BlackBerry (not only Android)
The only question I have.. Is whether someone already did it for real? Because tutorials that I've found - just show how to print 'Hello world!' message in simple HTML.
But what would be more interesting to get is 'how it works with embedded database' in android,
OR better say, as an example what I mean:
How to handle JS' "onClick" to get some data stored to db?
UPDATE:
Yeah.. there are a lot of PRODUCTS. But what if I need just a storage support (need to put and read data from db). no need anything sophisticated like camera support. What I really want to avoid is:
waiting for Emulator updated
write simple and easy components and layout in HTML to get instant result (it seems more natural for me, even though I'm Java Dev.. but it's lame to wait for result building and waiting when working with UI)
I'm looking for someone's existing experience to avoid addition complexity and time gaps between builds/tries while working with UI.
Look into Phonegap to convert HTML, CSS, and JS code into a native phone app for android or iOS. Very useful as it puts a web wrapper around the web code to run it natively on your phone. This should solve storage issues as well, lots of documentation around for phonegap, be sure to check it out!
I just released http://www.protocoder.org under MIT license, give it a try! Its meant to basically do what you mention in the first point (prototype fast), and on the second avoid to learn the Android API to deeply.
Its still a young project but I hope it will grow really fast :)
I am making a Book like app in Phonegap with Quiz... I want to keep 20 chapters for learning C. I am confused on how to use the single page architecture for such an app. I have also made a similar app for Android using JQuery Mobile and Android WebView before I was familiar with Phonegap but the application contained a number of (around 15 ) pages..and the Look & Feel was no more much native.
Any Suggestions on how to follow a single page architecture to create a book accessing its page contents from WebSQL for e.g. Chapter_title, page_heading, topic_contents ... and images if any.
Couple of months back I was in your situation. I experimented with jquery mobile, native application and phonegap. Building a web application at first seemed very attractive at first as the coding is a lot easier compared to native applications.Even though my answer is not very specific to your question it might be helpful to others. Here is my opinion based on my research:
Using phonegap slows your application responses, it loads slowly and together with jquery mobile the response and load time increases. So I didnot use it.
Using jquery alone was no different in Gingerbread mobile and on emulator you can clearly see the lag in switching between one div to another div. I didnot use it either.
What I did was android Webview. I wrote my own javascript and css and placed them on the section of my page. Used .... for changing pages on button click. This approach worked wonders for me and gave nearly native performance.
To follow the single page architecture in html use div element as I have mentioned in step 3 above. You can write a script to convert your WebSql data to json format. Then in your page where you need to call the data using javascript parse the data. Using json in really simple.
I want to provide a better interface to a web browser game - dedicated to Android. The game uses javascript and a bit of flash (so simple that it appears in webview correctly). It has lots of images that make the page load for ages on my device.
My first idea was to make a webview GONE, but it doesn't solve my problem that page loads really long.
My second idea is to use something like HttpClient (any advices?) to get HTML code of site and then parse it (all data that I need is there). BUT here comes two troubles:
1) How can I handle cookies? Is it hard?
2) How can I execute javascript without a WebView?
It's easy to create HTTP(s) request. Take a look at HttpURLConnection class.
There is an exemple how to handle cookies.
Here you can find a quick introduction to the HttpURLConnection.
You won't be able to execute javascript. But if you are creating an app for you webapp you will need to recreate the behaviour. The javascript is useless with a native app.
I want to make an app for android that updates via the internet so the information in the app is always current. What is the best way to implement this?
I understand I will need the information stored somewhere?
And I will need to be able to access it and read it into the app?
I am not looking for specific code so much as a general way this could be implemented.
As an example think of a weather app that must pull weather data from an online source.
Thanks
In general you could achieve what you want in one of the following ways (both are popular methods and used in many apps):
Implement a web service layer to pull the data from the server back to your application. In Android, this is normally done via JSON based REST API. JSON is used nowadays as the data is regarded more compact that XML and it's easier to deal and manipulate in Android.
Note that it's common to implement caching in phone local storage if the data is not updated frequently.
Most of the time, the authoritative data is stored in the server
With this method, you could arrange the layout freely on your app. However, you need to write Java code to arrange the layout
You could also add better interactivity especially if you are thinking that your UI will have a lot of touch interfaces or animation
Other way you could do this is by developing a mobile HTML webpage that you could embed in Android via Webview. A simple tutorial is available on Android's website
This method will retrieve HTML from the server, the same way your desktop browser retrieves HTML. The only difference is that the HTML is mobile optimized. Good example of mobile optimized HTML is gMail from Google
Using this method, unless you have full control of the server side, you don't necessary can control the layout as the server is the one that generates the HTML
A lot of times that interactivity is reduced with this approach though Javascript framework such as Sencha and others are quickly reducing the interactivity gap between native code (#1) and using HTML