How to extract data from website? - android

I want to create an Android App in which student can see their attendance. Actually attendance is updated daily in website as a database. In the website homepage student must enter their log in details such as Username, password and captcha. Then their profile is displayed in which their attendance can be seen. I want to extract the data alone (that is attendance percentage). I think the website uses .asp database. Anybody just give me an idea!
Also sorry if my question doesn't sounds good...

When you say, you are saving in DB, you mean you are saving in SQL Server DB or any other DB and your web App reads DB displays data. So definitely you can develop an android App to interact with that DB and show values in that App.

You could download data from particular user in JSON format.

It depends! Try to analyse the website, How credentials are verifying and interacting with the database. Firebug is a good browser plugin to do so. For achieving the purpose you need to make a script which will take the username, password and capcha to the database and after return you the attendance page.The most hard part is to reading capcha. here is good discussion over it.
Once you get the page you can scrap the page through beautiful Library called JSOUP in java. It is quite easy to learn. After getting the desired value after parsing, you can show the data as you want to show in your app.
Second option can be you anylyse the website extract that div which has the Credential thing and show only that div in webview, WebView has options to change UI element such as background and fonts. After successful Login and getting the result page, parse the information of your use and show the result as you want.
Lastly -The lazy way to do thing by using WebView for whole app. so every thing in webview after changing the UI. It will be same but with different look.

Related

Replicating phone gap to a native ios and android app issue

We have a phone gap cross-platform built app and we are building the same app in native.
The app has pages/views with text with embedded hyperlinks in them where it sends the user to other pages in the same app. Of course because this is HTML based so this is possible using phone gap apps. But what if we changed that to native how can we have links inside a text where it redirects users to other views or grabs other data from the database and pushes them to the UIViewController?
How will the backend send us JSON strings with hyperlinks in them that will be able to replicate what the phone gap built app does. Is this possible?
Ok maybe we can send embedded HTML in the JSON data. But how will these <a> links open or grab other data from the server?
I find it very complicated or maybe not doable at all. Anybody has ideas on how this could be solved?
A screen shot of what i mean. This is a phone gap app.
All right first and foremost you need structured data.
Lets say I opend a main post with id '14'. Api will fetch data from server for id 14.
JSON structure can be like this:
{"title":"What are origional sin?","content":"Although Adam & Eve had been given. God has given man free will.","inlineLinks":[{"startIndex":"8","endIndex":"18","contentId":"141"},{"startIndex":"55","endIndex":"64","contentId":"142"}]}
Now using spannable string in android and attributed string in ios you now have id for specific inline link content and start and end index which you can use to add clicks to specific inline links. Once user clicks these link you have id of that specific content which you can open in your app.
Since you know the spannable string I hope you dont need the detail code.
Wish you have started your question with Spannable String problem.
Please ignore the indexes incase they are wrong ;)

In android studio with java How to load web text as string from an activity for Android 6+ devices

From an Activity, How can I retrieve code or text from a website and display to the user ? I'm having difficulties programming back to Android, now developing as for Android 6.0 and newer. Getting deprecated about http client etc.. so I need a little of help.
Can you paste me some code example including "imports" in order to when the app load, from an Activity class, and also when a user click a "button1", to retrieve the text displayed from a web say "yahoo.es", and show a message to the user with the text from the web (loaded and parsed to a string) using Toast.
Help much appreciated in advance
Firstly you are way too ambiguous about your question. Correct me if i am wrong, what you basically want is to get data from internet and display them to user. Right.
If Yes, then the best way to do this is to get JSON response and parse it to display the relevant information onto the screen.
Take for example, you want an App displaying current Weather status of a city.Then to get the Weather data from Internet you need to connect to those websites which provide you that data.
Now one such website is OpenWeatherMap.org. You can use its APIs to retrieve data in JSON format and then display. You need to go through its API documentation.
This is link which will explain you the full process of connecting to an website's API, its retrieval of Data in Json format and ways to render and display them on screen effectively.
https://www.udacity.com/course/android-basics-networking--ud843
Hope it helps.

Android application getting information from a website

I want to write an app get the information from posts (mostly about new classes) on my university website then make some annoucement once there is new class.
I've google and see RSS mostly, but the page I want to get data does not use RSS. I read something about website using service so we can read XML in the app, but I don't know if the page using service or not. Last resort is using host (somee for example) and js to get text from the page, then the app could read data from the host.
That's all I have right now, but it's still unclear for me. Any suggestion about what to read, what should I use?? Much appreciation
I think the first thing you do should be to ask your uni if they provide any feed or webservice for these data that you could use.
If they don't, or don't answer at all, your last resort would be scraping. To do this, you can use an HTML parser, like jsoup, and then go through the HTML data, getting information as you need.
To see if there are been changes, you would just have to cache your current processed information, probably in a database, and compare the new one to the one currently saved.

Is there a way to pass a variable to an aspx file via the url?

I am trying to write a little Android app for my daughter. The goal is to scan a book's bar code and pass the ISBN number to this website: http://www.arbookfind.com/default.aspx . The result will show if the book is part of the Accelerated Reader program and how many points the book is worth. I am trying to automate the part where the ISBN would need to be entered into the search field.
For simplicity's sake, and because I'm not a programmer, I am using MIT's App Inventor 2. I can now scan and get the ISBN but I will need to know how to format a URL to the website that will allow me to pass the ISBN to it's search page.
Is it possible to send a variable via the URL similar to index.php?myvar=testing&someothervar=somethingelse ? I've tried but perhaps I am not using the correct variable name or format for aspx. Is there an easy way to see what the variable name is in the aspx displayed page in my browser?
EDIT To clarify, I am not trying to scrape data and avoid showing ads from the site I am using to generate the results. I am wanting to pass the ISBN number to the page and have it search and display the resulting page in the phone's browser. I am also fine with a method that would populate the search field and the user would have to hit the search button if that can be accomplished easier.
I would recommend abandoning this route, as it is highly unlikely that the owners of this website will want you passing a query string to their site anyway, but rather they will most likely point you to an Application Programming Interface (API) that they provide, so that your program can connect to this service (free or paid, depending upon the company) and then you can request the book's details by providing the ISBN in the request.
There is no discovery mechanism for an .aspx page like there is for a web service to find out the names of things to pass. Even if you figure out what the name of the query string is that you could pass in for ISBN, you run the risk of the implementation being changed and your "application breaking". While this is also true of web service APIs, since APIs are the route the website providers want you to use, as opposed to screenscraping, then they generally inform their users of breaking changes or newer versions of the API via documentation.
From what I can see that page does not accept URL-variables for their search field the way google.com and other does. The page is generated through some sort of content management system (CMS) and it relies heavily on javascript to make things work. I tried doing a normal search there, and you have two issues you need to wriggle around.
First, the page redirects you to a page where you select if your a student, parent etc. It seems that it relies on some session cookie to remember the setting, but it times out pretty fast.
Second, the form uses javascript to trigger the search, and it appears to be done using AJAX, a method of using javascript to trigger actions on the server and displaying the results, without actually loading the page again. You might be able to get a hold of the javascript code used and re-engineer it for your purposes, and call that using HTTP POST and/or GET from your app, but it is a tricky path, and quite possibly not allowed by the company since you will be loading data from their site, without presenting their advertisements and thus be costing them money.

Allow Android SDK to search through website?

Let's say there's website X. Website X has a search function that'll return articles/documents/whatever. Whatever it returns will include a link so the user could click on it to go to said article. All of this would be without an API. Would it be possible for an android application to query a search on a random website, get the results and display them in the app, without the user actually seeing the website?
Yes, of course. All you want to do is write the app such that it performs the query to the website on behalf of the user; that way it gets the HTML response. When it gets the HTML page as a response, you parse that and reformat it into something more appropriate for the app.
Look at this SO question if you want more information on HTML parsers in Java: Parse HTML in Android

Categories

Resources