I'm new to programming and I would like to display data from this site into my activity. Simple textviews with name and numbers nothing more. The problem is I don't know how to get those info from this site. When I open it with Firefox and try to inspect code, I can't find any variables from which it could be read, if I'm saying this correct!? :)
Can someone point me to right direction???
This is the web page:
http://parking-servis.co.rs/eng/where-to-park/
Like the other user mentioned this is too broad, but I can give you a few pointers and things you can start reading up on.
First: You want to get info from a website.
This can be easily done via web scraping. What this means is you read through the HTML code of a webpage and extract information from it. Libraries such as Jsoup will help you a lot here. They have great documentation so that should get you going. You can just hover over any HTML element to get its related information when the element is highlighted.
Second:You need to get this information to your App.
Well you are in luck because Jsoup has its own android port and you can simply import the library to your project and go ahead from there. You can write a simple flow that scrapes the website every day or e erytime the app is opened. You can pass net requests via something called AsysncTask.
So the library to start looking into is Jsoup and you should also read up on AsyncTask. Besides these, there are other links you might be interested in :
Web Parsing with JSoup
JSoup on Android
AsyncTask Basic Tutorial
This is a fairly complicated task, if the website doesn't provide an API you will have to web scrape to get the information off of that page. as for getting the data into android studio you will have to research how to use GET / POST calls to (hopefully) an API with that information.
Not much else I can do as this question is very broad, but hope it has helped.
Related
I just finished a tutorial on how to develop an android application that retrieves updated posts from a Blog using JSON data.
The link to the JSON data used to retrieve the posts, was the blog name ending with "/api/get_recent_summary"
How can I find the link of the JSON data for different websites?
For example the website for The Time Magazine
http://time.com
The quickest and easiest way is to use google developer tools in Google Chrome.
1st go to google developer tools. F12 or Ctrl+Shift+I or top right Ellipsis->More Tools->Developer Tools
2nd Click on the "Network" tab.
3rd click on the "XHR" sub-tab. XHR(XMLHttpRequest)
if a site uses json it will be listed under the XHR sub-tab. You can search through the different return objects, select one and use the "preview" sub-sub-tab to view it.
View JSON
View JSON URL
Although the above way is the easiest it is not the most stable way of getting the info you need. Many sites make changes to the return data without any notice. This will probably break your app...
I think what you are looking for is an API(Application programming interface). Most web APIs return json or xml. You should start by searching for the api documentation for the specific site that you want to get json data from. Example documentation for sites that have public api feeds are github api or youtub api. Many of these will require authentication in order to get the desired json response but the documentation should show you how to do it.
Using a legitimate web api is the most stable way to go. Meaning your app has less chance of randomly breaking all of the time due to feed changes, url changes... I hope this helps!
I know this is an older question, but I felt compelled to chime in. If you goal is to simply determine if a site uses JSON for data exchange, then the solution proposed by Tim is a very good solution. However, if you are looking to scrape data from an arbitrary site, there is no guarantee that the site uses JSON as data exchange, as stated by #KenanZahirovic. There are numerous sites that do not do this. Instead they embed the data into the HTML, or use XML, or some other method for getting the content to the client. There is no standard way of doing it, which is why data scraping is so difficult. It requires figuring out how data is transferred and building a client for that.
For scenarios where you need to gather data from multiple sources, you may end up with multiple clients due to the nuances between the sources. This site explains some best practices for data scraping. However, this would likely require a server-side application. Having a server-side application that gathers data and stores it in a database would make the most sense for this scenario. This way you can have a consistent API that the client hits to access the data.
An algorithm has been proposed that can scrape many sites fairly confidently. If you only want to have a client, this may or may not be the best way to go. It all depends on how much processing you want the client to do. If at all possible, try and off load processing to a server.
This answer might be way more than is required, but ,again, I felt compelled to chime in. I am sure the previous answer was sufficient. I do recommend though that you mark an answer as the accepted answer.
Best of luck!
After much frustration I need some help getting a HTML parser for Android.
I basically need a parser which will log into a website and retrieve the main page, which heavily uses javascript and ajax to render data. From there I need to be able to access other pages which are linked with anchor tags. Again, emphasis on the Ajax and javascript on the page, all the data is created through these means.
EDIT: After some research it looks like I need a headless browser that is compatible with Android.
At first I tried with JSoup and it only retrieves the page without the javascript and ajax data being written in. Unless I'm understanding something wrong here, JSoup is not fitting in this situation.
I have also tried HtmlUnit, which works perfectly, but I am unable to get it up and running in android because of the known jar conflicts and the "not being able to convert to dalivik" error.
If anyone knows another suitable HTML parser, or can help me with either getting JSoup and ajax pages to play nice, or knows how to get HtmlUnit running on android, I would be forever in your debt.
I have already spent the majority of a week with JSoup and HtmlUnit, and basically have gone back to square one. Frustration has set in and I'm completely lost on what to do.
Again, I need a HTML parser, that will allow Ajax and Javascript to load before I can parse, and it needs to be android friendly!
Thank you!
You can try to intercept the individual Ajax communications. They are either get or post communication. And you can call them with JSoup one by one. It is feasible for a handful of Ajax call. In fact there are less data and can be more quicker to parse on Android this way.
It's the very first time for me asking question on this forum, so hope that everyone won't be too strict.
This semester I've been doing Project and I decided to make android application that will be able to parse articles from websites(not one, but many) and save it in text file or SQLite Database. I've been doing this for already 5 months and I really don't know how can I get particular plain text from webpage without getting unrelated to particular article stuff, such as adds etc.
Through this forum I found out how I can parse data from one website using Jsoup(I'm using one in my Project). And I thought it might work with multiple websites as well, but since many websites using different html tags to represent content of certain article, I couldn't find any common parameters that can be used.
I'm not sure, but I think I saw somebody on this forum said this kind of thing is impossible, but then how we can explain existence of such kind of apps like Pocket(Former Read it Later), Instapaper etc. These apps are doing what ultimately I want my app to do.
So can anybody give any suggestions about what can I do about it? Thanks.
I'm not sure whether it's going to show whole story, but here is the code:
doc = Jsoup.connect("http://"+URLField.getText().toString()+"").get();
// get page title
title = doc.title();
doc = new Cleaner(Whitelist.none()).clean(doc);
Here I'm simply using Jsoup library and its Cleaner class, but I get a lot of text that I don't want to appear. I wanted to post image to make it clear(what I want), but this forum doesn't let me do that.
I was trying to write an android app to help people do easier searches on multiple websites like craiglist, ebay ..etc
I am not sure where really to start from, or the name of the technology I would be using. To make it more accurate question, these websites are all forms based that you either post stuffs or retreive stuffs through searching. How can I make my android app actually in which it will ask the user for a item name (that part I know) and then send item name to say craiglist so it gets the results back?
I was thinking of copying the url that is displayed on my broswer when I click search on any of the website and using the same link with string manipulation to include whats needed.
Is this correct? Any other ideas are I know? I am kinda all over the place, but I need a starting point.
Some sites might expose a public API for you to connect with, but that's site specific.
Sites that do not expose such a system can be "scraped", which means you're using string manipulation (as you mentioned) to parse our the data you're after.
Both ways require basic knowledge on how to perform HTTP operations, there are several ways to do this but the easiest way it to use the HttpURLConnection class as described in various tutorials on the web.
Googling for "android http GET POST tutorial" should point you in the right way to get started with your adventure :)
I'm trying to write my first app where in I login to website and setup some preferences.
My problem is this website does not have api's or documentation on how to do it. This is my first time doing any kind of web programming. Please pardon me if my questions are naive.
While reading android documentation and searching through Stack overflow i understand HTTPPOST is the preferable way to do such a thing.
If the website does not provide URI and documentation, how to figure out these things? I tried looking at the HTML source code. I could figure out login form. I'm not sure if that always works.
How to figure out which encoding is to be used while creating HTTPPOST requests?
Is there any other way to achieve this?
Thanks
i think u must parse html . try see this mb it's help u Android : Parse HTML block of code