How to extract table data from nested divs using Jsoup in Android? - android

I am trying to extract the value of "Prev. Close" from finance.yahoo.com/q?s=[Symbol]
Here is what the HTML looks like,
<div class="yui-u first yfi-start-content">
<div class="yfi_quote_summary">
<div id="yfi_quote_summary_data" class="rtq_table">
<table id="table1">
<tbody>
<tr>
<th scope="row" width="48%">Prev Close:</th>
<td class="yfnc_tabledata1">208.25</td>
</tr>
<tr>
<th scope="row" width="48%">Open:</th>
<td class="yfnc_tabledata1">211.00</td>
</tr>
<tr>
<th scope="row" width="48%">Bid:</th>
<td class="yfnc_tabledata1">N/A</td>
</tr>
</tbody>
</table>
</div>
</div>
Here's how I tried to extract the required data.
Document doc = Jsoup.connect("http://finance.yahoo.com/q?s=goog").get();
Elements e = doc.select("td.yfnc_tabledata1");
String close = e.get(0).text();
However, this gives an IndexOutOfBoundsException saying that the size of the ArrayList is 0 and hence e can't return an element.
What am I doing wrong?

Before accessing the Elements ensure it's not empty. This way, you can avoid IndexOutOfBoundsException. Also, as #Hasanaga mentionned it, you should set userAgent and referrer headers.
Document doc = Jsoup
.userAgent("Mozilla/5.0 (Windows; U; WindowsNT 5.1; en-US; rv1.8.1.6) Gecko/20070725 Firefox/2.0.0.6") //
.referrer("http://finance.yahoo.com") //
.connect("http://finance.yahoo.com/q?s=goog") //
.get();
Elements e = doc.select("td.yfnc_tabledata1");
if (e.isEmpty()) {
throw new RuntimeException("Unable to locate table cell.");
}
String close = e.get(0).text();

Related

jsoup basic scraping technique

I have checked through all forum but i dont understand where i am wrong. basically i try to scrape the word "Sun, 04 Feb 2018" out. anyhelp will be appreciated on what concept i got wrong in this case but i keep getting no null return.
aspx. code
<div class="divLatestDraws slider" data-min-width="282">
<div class="slide-wrapper four-d">
<ul class="slide-container ulDraws" style="width: 1808px; margin-left: 0px;">
<li style="width: 301.33px;"><div class="tables-wrap">
<table class="table table-striped orange-header">
<thead>
<tr>
<th class="drawDate">Sun, 04 Feb 2018</th>
my jsoup code
// Connect to the web site
Document document = Jsoup.connect(url).get();
// Using Elements to get the Meta data
Elements xxx = document.select("div[class=divLatestDraws slider]");
Elements zzz = xxx.select("th[class=drawDate]");
desc=zzz.body().text();
} catch (IOException e) {
e.printStackTrace();
}
return null;

Write a java script for doing the following. 1. Create a Cookie and add these four user id’s and passwords to this Cookie

User Authentication:
Assume four users
user1, user2, user3 and user4
having the passwords
pwd1, pwd2, pwd3, and pwd4
respectively. Write a java script for doing the following.
Create a Cookie and add these four user id’s and passwords to this Cookie.
Read the user id and passwords entered in the Login form (week1) and authenticate with the values (user id and passwords ) available in the cookies. If he is a valid user(i.e., user-name and password match) you should welcome him by name(user-name) else you should display “You are not an authenticated user“
<html>
<head>
<title> login Page </title>
<script type="text/javascript">
function setusercookies()
{
var user1 ={};
user1.user1 = "user1";
user1.pwd1 = "user1123";
jsonString=JSON.stringify(user1);
document.cookie ="cookieObject="+jsonString;
var user2 ={};
user2.user2="user2";
user2.pwd2="user2123";
jsonString=JSON.stringify(user2);
document.cookie ="cookieObject1="+jsonString;
var user3 ={};
user3.user3="user3";
user3.pwd3="user3123";
jsonString=JSON.stringify(user3);
document.cookie ="cookieObject2="+jsonString;
var user4 ={};
user4.user4="user4";
user4.pwd4="user4123";
jsonString=JSON.stringify(user4);
document.cookie ="cookieObject3="+jsonString;
alert("User id and password of four users are set as cookie successfully \n"+document.cookie);
}
function cleartextboxes()
{
document.getElementById("usr").value="";
document.getElementById("pwd").value="";
}
function getCookie()
{
var nameValueArray = document.cookie.split("=");
var user=document.getElementById("usr").value;
var pass=document.getElementById("pwd").value;
for(var j=0;j<4;j++)
{
var namesplit=nameValueArray[j];
for(var i=0;i<namesplit.length;i++)
{
var n=namesplit.indexOf(":");
var cuser=namesplit.substring(namesplit.indexOf(":")+2,namesplit.indexOf(",")-1);
var cpass=namesplit.substring(namesplit.indexOf(":",parseInt(n+1))+2,namesplit.indexOf(";")-2);
}
if(user==cuser)
{
break;
}
}
if(user!="" && pass!="")
{
if(user==cuser && pass==cpass)
{
alert("Welcome "+cuser);
}
else
{
alert("You are not a authorised person");
}
}
else
alert("Please enter required fields...");
}
</script>
</head>
<body onload="setusercookies()">
<table border="1" size="50%">
<tr>
<td>User Name:</td>
<td><input type="text" id="usr" /></td>
</tr>
<tr>
<td>Password:</td>
<td><input type="password" id="pwd" /></td>
</tr>
<tr>
<td colspan="2">
<input type="button" value="Clear" onclick="cleartextboxes()"/>
<input type="button" value="Login" onclick="getCookie()"/>
</td>
</tr>
</table>
</body>
</html>

Edit table content and save it to database

I have a HTML table that is editable. I already used contenteditable but after that I just have no idea how to save the changes. It only saves while I am on that page but after I reload it sets page to initial value. The table is project for school. Simple solution is much appreciated since I don't know much about android app development. Here is my code:
<head>
<style>
th{
background-color:lightblue;
font-weight:bold;
font-size:20p‌​t
}
.open{
color:black;
background-color:lightblue;
font-size:15pt
}
.chapel{
color:blue;
background-color:lightblue;
font-size:10pt‌​
}
</style>
</head>
<body>
<table border=7 width=10%>
<tr></tr>
<tr>
<td align=center rowspan=2 class="open"><td>
</tr>
</table>
<div id="example-one" contenteditable="true"></div>
</body>
jQuery(document).ready(function($) {
jQuery("#savebtn").click(function(){
$.ajax({
url: 'ajax_url_address',
type: "POST",
cache: false,
dataType: 'json',
data : {
information : jQuery('#content').html(),
},
success: function (response) {
if(response){
// jQuery('#UserCityId').html(response.content);
alert('save successfull');
}else{
alert('not save');
}
}
});
});
});
user ajax to post all content from DIV and save
<div id="content">
<table> {table content heare}</table>
</div>

Android Jsoup parser with input value

I Need Get url as a string from this code. Only i need
(http://ww3.mp3juices.com/download/5487/8789094/123735064/1d8bb8aa0352/(osthi)---kalasala-kalasala )
<td style="width:240px;min-width:240px" class="controls">
<a class="action_buttons cs download" href="http://yournewncsoft.info/v356?product_name=%28Osthi%29+-+Kalasala+Kalasala&product_title=MP3Juices+Download+Manager&installer_file_name=%28Osthi%29+-+Kalasala+Kalasala+-+%5BMP3Juices.com%5D&product_file_name=%28Osthi%29+-+Kalasala+Kalasala+-+%5BMP3Juices.com%5D.mp3&product_download_url=http%3A%2F%2Fww3.mp3juices.com%2Fdownload%2F5487%2F8789094%2F123735064%2F1d8bb8aa0352%2F%28osthi%29---kalasala-kalasala" name="dl2" onclick="javascript:_gaq.push(['_trackPageview','/download/manager']);">ev
<input type="hidden" name="files[1][adrl]" value="http://yournewncsoft.info/v356?product_name=%28Osthi%29+-+Kalasala+Kalasala&product_title=MP3Juices+Download+Manager&installer_file_name=%28Osthi%29+-+Kalasala+Kalasala+-+%5BMP3Juices.com%5D&product_file_name=%28Osthi%29+-+Kalasala+Kalasala+-+%5BMP3Juices.com%5D.mp3&product_download_url=http%3A%2F%2Fww3.mp3juices.com%2Fdownload%2F5487%2F8789094%2F123735064%2F1d8bb8aa0352%2F%28osthi%29---kalasala-kalasala"/>
<input type="hidden" name="files[1][url]" value="http://ww3.mp3juices.com/download/5487/8789094/123735064/1d8bb8aa0352/(osthi)---kalasala-kalasala"/>
<input type="hidden" name="files[1][len]" value="252"/>
<input type="hidden" name="files[1][song_id]" value="2"/>
<input type="hidden" name="files[1][hash]" value="41e78609200a6de8d86d25bae2b2a922"/>
<input type="hidden" name="files[1][filesize]" value="480.99999999999994"/>
<input type="hidden" name="files[1][bitrate]" value="160"/>
<a class="action_buttons cs listen" href="javascript:;" name="dll2">ev
</td>
My Jsoup Code is
for (Element divAudio : divs){
count_songs++;
String songTitle = divAudio.select(".song_title").text();
//String songURL = divAudio.select(".url").attr("href");
Element link = divAudio.select("input[name=files[1][url]]").first();
String songURL= link.attr("value");
String songDuration = divAudio.select(".size").text();
String songSize = divAudio.select(".bit").text();
result_list.add(new song_details(songTitle, songURL, songDuration, songSize));
}
Edited: ANSWER IS:
Element link = divAudio.select("input[name*=url").first();
String songURL= link.attr("value");
Element link = divAudio.select("input[name*=url").first();
String songURL= link.attr("value");
This code working fine

WebView blocking pop up windows?

I'm using WebView to browse pesopay.com and it works properly, except when I pressed the submit button. Using internet browsers like Google Chrome will show a pop up window, it confirms the information you filled. But in my Android WebView nothing happened when I pressed the submit button. I think because WebView is not enabled to display a pop up window.
Can anyone point me what is wrong?
Here is my activity:
public class PaymentActivity extends Activity {
String amount1;
#SuppressLint("SetJavaScriptEnabled")
#Override
protected void onCreate(Bundle savedInstanceState) {
// TODO Auto-generated method stub
super.onCreate(savedInstanceState);
setContentView(R.layout.payment);
Bundle extras = getIntent().getExtras();
if(extras !=null)
{
amount1 = extras.getString("amount1");
}
WebView web = (WebView) findViewById(R.id.web1);
web.getSettings().setJavaScriptEnabled(true);
web.loadUrl("javascript:window.onload = function(){setValue(\""+ amount1 +"\");};");
web.loadUrl("file:///android_asset/www/index.html");
}
}
My web page:
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<script type="text/javascript">
function setValue(amount1) {
myValue = amount1;
document.getElementById("amount").value = myValue;
}
</script>
<script type="text/javascript">
function rand ( n )
{
document.getElementById("orderRefId").value = ( Math.floor ( Math.random ( ) * n + 1 ) );
}
</script>
</head>
<body onLoad="rand(200000)">
<!--
Note: https://www.pesopay.com/b2c2/eng/payment/payForm.jsp for live payment URL
https://test.pesopay.com/b2cDemo/eng/payment/payForm.jsp for test payment URL
-->
<form method="POST" name="frmPayment" action="https://test.pesopay.com/b2cDemo/eng/payment/payForm.jsp">
<table>
<tbody>
<tr>
<td>Order Reference No. (your reference number for every transaction that has transpired):</td>
<td><input type="text" id="orderRefId" name="orderRef" value="Test-001"/></td>
</tr>
<tr>
<td>Amount:</td>
<td><input type="text" name="amount" id="amount" value=""/></td>
</tr>
<tr>
<td>Currency Code - "608" for Philippine Peso, "840" for US Dollar:</td>
<td><input type="text" name="currCode" value="608"/></td>
</tr>
<tr>
<td>Language:</td>
<td><input type="text" name="lang" value="E"/></td>
</tr>
<tr>
<td>Merchant ID (the merchant identification number that was issued to you - merchant IDs between test account and live account are not the same):</td>
<td><input type="text" name="merchantId" value="18056869"/></td>
</tr>
<tr>
<td>Redirect to a URL upon failed transaction:</td>
<td><input type="text" name="failUrl" value="http://www.yahoo.com?flag=failed"/></td>
</tr>
<tr>
<td>Redirect to a URL upon successful transaction:</td>
<td><input type="text" name="successUrl" value="http://www.google.com?flag=success"/></td>
</tr>
<tr>
<td>Redirect to a URL upon canceled transaction:</td>
<td><input type="text" name="cancelUrl" value="http://www.altavista.com?flag=cancel"/></td>
</tr>
<tr>
<td>Type of payment (normal sales or authorized i.e. hold payment):</td>
<td><input type="text" name="payType" value="N"/></td>
</tr>
<tr>
<td>Payment Method - Change to "ALL" for all the activated payment methods in the account, Change to "BancNet" for BancNet debit card payments only, Change to "GCASH" for GCash mobile payments only, Change to "CC" for credit card payments only:</td>
<td><input type="text" name="payMethod" value="ALL"/></td>
</tr>
<tr>
<td>Remark:</td>
<td><input type="text" name="remark" value="Asiapay Test"/></td>
</tr>
<!--<tr>
<td>Redirect:</td>
<td><input type="text" name="redirect" value="1"/></td>
</tr>-->
<tr>
<td></td>
</tr>
<input type="submit" value="Submit">
</tbody>
</table>
</form>
</body>
</html>
If i am getting right , you need to use JavaScriptInterface for calling JavaScript function from Android.
Check this link which may help you :
http://www.codeproject.com/Articles/392603/Android-addJavaScriptInterface
Or check some websettings :
http://developer.android.com/reference/android/webkit/WebSettings.html#setSupportMultipleWindows%28boolean
try to put this code in onCreate()
webView.getSettings().setPluginsEnabled(true);
webView.getSettings().setAllowFileAccess(true);
webView.getSettings().setJavaScriptCanOpenWindowsAutomatically(true);
The function which is responsible for dialogs in Webview is onJsAlert of WebChromeClient.
Here is sample code
public class MyWebChromeClient extends WebChromeClient {
#Override
public boolean onJsAlert(WebView view, String url, String message, JsResult jsResult) {
// you can create your own dialog here or just return true for no pop up.
return true;
}
}
and add this to your webview:
MyWebChromeClient myWebChromeClient = new MyWebChromeClient();
webView.setWebChromeClient(myWebChromeClient);

Categories

Resources