how to get dynamic data using JSoup? - android

I'm a beginner at android and I'm trying to make an app that get values from a html file hosted in a webserver that gets data from sensors of an arduino. But JSoup only gets the first value, i don't know how to get the values that changed.
Here is the MainAcivity.java code:
Public class MainActivity extends AppCompatActivity {
final String TAG = this.getClass().getSimpleName();
private AnimationDrawable creepyAnimation;
private ImageView creepyImage;
private TextView result;
RequestQueue requestQueue;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
creepyImage = (ImageView) findViewById(R.id.creepy);
creepyImage.setBackgroundResource(R.drawable.animation);
creepyAnimation = (AnimationDrawable) creepyImage.getBackground();
requestQueue = Volley.newRequestQueue(this);
result = (TextView) findViewById(R.id.textView);
getWebSite();
}
private void getWebSite(){
new Thread(new Runnable() {
#Override
public void run() {
final StringBuilder builder = new StringBuilder();
try {
Document doc = (Document) Jsoup.connect("http://munhoz-unifei.000webhostapp.com/").get();
Element valor1 = doc.getElementById("sala");
Element valor2 = doc.getElementById("quarto");
builder.append(valor1.attr("sala")).append("xxx: ").append(valor1.text()).append("\n");
builder.append(valor2.attr("quarto")).append("lalala: ").append(valor2.text()).append("\n");
/*
for (Element element : valor){
builder.append(element.attr("sala"))
.append("\n").append("text: ").append(element.text());
}
*/
}catch (IOException e){
builder.append("ERROR: ").append(e.getMessage()).append("\n");
}
runOnUiThread(new Runnable() {
#Override
public void run() {
result.setText(builder.toString());
}
});
}
}).start();
}
}
i just want to get the values to change on the app as it changes on the website. Could someone give me a hint or an example of how to do this work?

Jsoup can only parse or process the html that is sent by the server.
Anything part of the DOM that is generated by the javascript on the client's end can't be parsed through Jsoup.
What you can try is selenium web automation to obtain the html rendered by the browser.

Related

Andriod Not Showing Process Dialague Box While Call Goes To Server

I'm new to asking questions here so any advice about that will be appreciated...but to my problem:enter code here
I am Trying To show Process Dialague While Waiting for Data From Server..I am Using Retrofit Call For Get Data From Server And Using MVP Pattern In Our Project..
But Showing Black Screen While Call Goes To Server.. And Load Content Directly After Gettting Response
I Am Stuck This Point From Few Days..
public class ReceiptRegisterActivity extends AppCompatActivity implements ReceiptRegisterPresenter.ReceiptRegisterPresenterListener {
#Override
protected void onCreate(Bundle savedInstanceState) {
setContentView(R.layout.activity_receipt_register);
progressBar = new ProgressDialog(ReceiptRegisterActivity.this);
progressBar.setCancelable(true);
progressBar.setMessage("Fetching Data...");
progressBar.setProgress(20);
progressBar.setProgressStyle(ProgressDialog.STYLE_SPINNER);
progressBar.show();
progressBarStatus = 0;
spinner = (SearchableSpinner)findViewById(R.id.spinner);
editsearch = (android.widget.SearchView) findViewById(R.id.search);
editsearch.setOnQueryTextListener(this);
expandButton =(Button)findViewById(R.id.expandButton);
byDate =(Button)findViewById(R.id.byDate);
byCustomer=(Button)findViewById(R.id.byCustomer);
byDate.setFocusable(false);
allEventLIstener();
dbHelperObj=new DBHelper(this);
try{
dbHelperObj.createDB();
}catch (Exception e){
throw new Error(e);
}
try{
dbHelperObj.openDataBase();
}catch (SQLiteException e){
e.printStackTrace();
}
//srActivity = this;
// progressBar = new ProgressBar(this);
// progressBar.setVisibility(View.VISIBLE);
receptRegisterPresenterObj = new ReceiptRegisterPresenter(this,this);
receptRegisterPresenterObj.getReceiptRegisterData();
}
public void receiptRegisterDataReady(Object obj) {
/// Getting Response In This Block Doing Some Manupilation And Dismiss progressBar...
progressBar.dismiss();
}`
Thanks for any help
You are doing too much work in the OnCreate main Thread. This is why it freezes your Android Application.
Do your Progress UI in the Main thread like so:
#Override
protected void onCreate(Bundle savedInstanceState) {
setContentView(R.layout.activity_receipt_register);
progressBar = new ProgressDialog(this, "title", "loading");
heavywork();
}
public void heavywork()
{
//start a new thread to process job
new Thread(new Runnable() {
#Override
public void run() {
// Do all the Data From Server code here !
}
}).start();
}
This might help you:
ProgressDialog in a separate thread

JSOUP html parse giving different results on android device vs windows machine

The following code ran on a java file on my computer gives the correct result of "/pws/client/pdf/offers-in-store-10-11-16.pdf"
String pdfLink= null;
try {
Document doc = Jsoup.connect("http://www.dunnesstores.com/offer20/food-wine/fcp-category/home").get();
Element links = doc.select("a[title=\"Download offers in store\"]").first();
System.out.println(links.attr("href"));
} catch (IOException e) {
//e.printStackTrace();
}
However when I run the code on my android device in an app where it is activated by the button press,I get a null pointer exception on "pdfLink=links.attr("href");" so it isnt finding the link "/pws/client/pdf/offers-in-store-10-11-16.pdf" for whatever reason.In my android app the code is on a onclick listener of a button and I have stepped through the code,it does activate the on click and runs the right lines of code but for whatever reason JSOUP is not finding the link.Below is the android code
public class MainActivity extends AppCompatActivity {
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
Button btnFetchData = (Button) findViewById(R.id.buttonTest);
btnFetchData.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View v) {
new FetchWebsiteData().execute();
}
});
}
private class FetchWebsiteData extends AsyncTask<Void, Void, Void> {
private String pdfLink = "didnt work";
#Override
protected void onPreExecute() {
super.onPreExecute();
}
#Override
protected Void doInBackground(Void... params) {
try {
Document doc = Jsoup.connect("http://www.dunnesstores.com/offer20/food-wine/fcp-category/home").get();
//Elements links = doc.select("a[title=\"Download offers in store\"]");
Element links = doc.select("a[title=\"Download offers in store\"]").first();
pdfLink=links.attr("href");
} catch (IOException e) {
//e.printStackTrace();
}
return null;
}
#Override
protected void onPostExecute(Void result) {
TextView txttitle = (TextView) findViewById(R.id.resultTextView);
txttitle.setText(pdfLink);
}
}
}
the mobile browser user-agent differs from the desktop browser; therefore, the HTML responses differ. In order to get the same result you have to set a desktop user-agent. Change this line :
Document doc = Jsoup.connect("http://www.dunnesstores.com/offer20/food-wine/fcp-category/home").get();
to:
Document doc = Jsoup.connect("http://www.dunnesstores.com/offer20/food-wine/fcp-category/home").userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0").get();

Android: Refresh ListView every minute

I've been reading all day threads regarding this issue I came up with a strategy but can't make it work
I have a listview fetching json data from a sql server
this listview already has a on swipe refresh function
I need this listview to refresh automatically only when new row was inserted in the data base.
So I wrote a php file fetching number of rows and echoing it witha 3 second refresh (on the php itself) so every time I enter the php file I get the realtime row numbers of my table.
I'm trying to build a function inside my MainActivity:
int OldNumberOfRows = data from the php file
while(true){
int newNumberOfRows = fetch data again using that php
if(both arent equal) execute refresh command.
}
Note: I got no idea how to extract the string from my asynctask to start manipulating my code with it.
That's it in general, Iv'e added the main activity , the "outer class" (FetchNumRowAsync) calling that php the swipe class and the php itself
MainActivity:
public class MainActivity extends AppCompatActivity implements SwipeRefreshLayout.OnRefreshListener {
private String TAG = MainActivity.class.getSimpleName();
private String URL = "http://troyka.esy.es/troyka/orders.php";
private SwipeRefreshLayout swipeRefreshLayout;
private ListView listView;
private SwipeListAdapter adapter;
private List<Order> orderList;
// initially offset will be 0, later will be updated while parsing the json
private int offSet = 0;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
new FetchRowNumAsync(this).execute("http://troyka.esy.es/numberofrows.php");
listView = (ListView) findViewById(R.id.listView);
//RelativeLayout.LayoutParams layout_description = new RelativeLayout.LayoutParams(50,10);
//Rl.setLayoutParams(layout_description);
swipeRefreshLayout = (SwipeRefreshLayout) findViewById(R.id.swipe_refresh_layout);
orderList = new ArrayList<>();
adapter = new SwipeListAdapter(this, orderList);
listView.setAdapter(adapter);
swipeRefreshLayout.setOnRefreshListener(this);
/**
* Showing Swipe Refresh animation on activity create
* As animation won't start on onCreate, post runnable is used
*/
swipeRefreshLayout.post(new Runnable() {
#Override
public void run() {
swipeRefreshLayout.setRefreshing(true);
fetchOrders();
}
}
);
}
/**
* This method is called when swipe refresh is pulled down
*/
#Override
public void onRefresh() {
fetchOrders();
}
/**
* Fetching movies json by making http call
*/
private void fetchOrders() {
// showing refresh animation before making http call
swipeRefreshLayout.setRefreshing(true);
// appending offset to url
String url = URL + offSet;
// Volley's json array request object
JsonArrayRequest req = new JsonArrayRequest(url,
new Response.Listener<JSONArray>() {
#Override
public void onResponse(JSONArray response) {
Log.d(TAG, response.toString());
if (response.length() > 0) {
// looping through json and adding to order list
for (int i = 0; i < response.length(); i++) {
try {
JSONObject orderObj = response.getJSONObject(i);
int rank = orderObj.getInt("rank");
String title = orderObj.getString("title");
Order m = new Order(rank, title);
orderList.add(0, m);
// updating offset value to highest value
if (rank >= offSet)
offSet = rank;
} catch (JSONException e) {
Log.e(TAG, "JSON Parsing error: " + e.getMessage());
}
}
adapter.notifyDataSetChanged();
}
// stopping swipe refresh
swipeRefreshLayout.setRefreshing(false);
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
Log.e(TAG, "Server Error: " + error.getMessage());
Toast.makeText(getApplicationContext(), error.getMessage(), Toast.LENGTH_LONG).show();
// stopping swipe refresh
swipeRefreshLayout.setRefreshing(false);
}
});
// Adding request to request queue
MyApplication.getInstance().addToRequestQueue(req);
}
}
FetchRowNumAsync:
public class FetchRowNumAsync extends AsyncTask<String, Void, String> {
private Context mContext;
public FetchRowNumAsync(Context ctx){
this.mContext = ctx;
}
protected String doInBackground(String... urls)
{
String fullString = "";
try{
URL url = new URL(urls[0]);
BufferedReader reader = new BufferedReader(new InputStreamReader(url.openStream()));
String line;
while ((line = reader.readLine()) != null) {
fullString += line;
}
reader.close();
}catch(Exception e ){
e.getMessage();
}
return fullString;
}
#Override
protected void onPostExecute(String value){
try{
((OnValueFetchedListener) mContext).onValueFetched(value);
}catch(ClassCastException e){}
}
public interface OnValueFetchedListener{
void onValueFetched(String columns);
}
}
SwipeListAdapter:
public class SwipeListAdapter extends BaseAdapter {
private Activity activity;
private LayoutInflater inflater;
private List<Order> orderList;
private String[] bgColors;
public SwipeListAdapter(Activity activity, List<Order> orderList) {
this.activity = activity;
this.orderList = orderList;
bgColors = activity.getApplicationContext().getResources().getStringArray(R.array.movie_serial_bg);
}
#Override
public int getCount() {
return orderList.size();
}
#Override
public Object getItem(int location) {
return orderList.get(location);
}
#Override
public long getItemId(int position) {
return position;
}
#Override
public View getView(int position, View convertView, ViewGroup parent) {
if (inflater == null)
inflater = (LayoutInflater) activity.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
if (convertView == null)
convertView = inflater.inflate(R.layout.list_row, null);
TextView serial = (TextView) convertView.findViewById(R.id.serial);
TextView title = (TextView) convertView.findViewById(R.id.title);
serial.setText(String.valueOf(orderList.get(position).id));
title.setText(orderList.get(position).title);
String color = bgColors[position % bgColors.length];
serial.setBackgroundColor(Color.parseColor(color));
return convertView;
}
}
PHP
<?php
header("refresh: 3;");
$mysqli = new mysqli("irrelevant","irrelevant","irrelevant","irrelevant");
/* check connection */
if (mysqli_connect_errno()) {
printf("Connect failed: %s\n", mysqli_connect_error());
exit();
}
$query = "SELECT COUNT(*) FROM orders";
$result = mysqli_query($mysqli,$query);
$rows = mysqli_fetch_row($result);
echo ($rows[0]);
$result->close();
$mysqli->close();
?>
Try this approach:
Create an endpoint in your server like the following:
//http://somesite.com/api/data/pull/check
Then, you can easily check this endpoint that returns some value like true or false depending on whether there is new data inserted into the db.
From the result you receive, you can then decide on whether to refresh your data on the phone by making another HTTP request or not. You always want to avoid making unnecessary requests to the server - remember users spend money every time they use their data plan (service).
I, like in the comments above, recommend having a column with a timestamp that you can check so that you only get the newly added data instead of everything!
I hope this gives you a simple idea on how to approach this issue! Good luck!
android app will not know when you have added/updated data in your table on the server until and unless you call script from app and fetch the data and update in your device.
only if your app has implemented these feature's
push notification- call Script every time you receive notification.
XMPP service- used for chat apps(which is not probably answer for
your question right now)
here is my suggestion for you
From server side:
create timestamp field in your table on server. update it with
current timestamp value every time you do changes(i.e update/add) in
the table.and when when that script is called send it across in json
and make your app save it in sqlite along with data.
server will compare for timestamp posted by app everytime with the
saved timestamp in the server for new data.
from client side:
for fist time timestamp from app will be 0. server will check it and
send the whole data along with the timestamp saved during changes in
table. save the data along with time stamp . second time when the
script is called App will be sending the timestamp that was last
saved.
with all this your app will not come to know still if new data is added until you call script and check. but atleast it will come to know if new data is received or not and whether to refresh ur screen
now comes script calling part from client side that is executing of assynch task, do it using handler to execute assynch class every minute
final Handler timerHandler = new Handler();
Runnable timerRunnable;
timerRunnable = new Runnable() {
#Override
public void run() {
new FetchRowNumAsync(context).execute(url);
timerHandler.postDelayed(timerRunnable, 60000); // run every minute
}
};
and unregister it in onDestroy()
#Override
public void onDestroyView() {
// TODO Auto-generated method stub
super.onDestroyView();
timerHandler.removeCallbacks(timerRunnable);
}

JSoup not working in android

I am creating an app to fetch XKCD comics (just for learning purpose). I am using JSoup to fetch comic title and url of the image and Universal Image Loader to post the image in ImageView. However I'm facing some problems. Here is my main activity.
public class MainActivity extends Activity {
private static final String TAG = "MyApp";
ImageView xkcdImgIV;
TextView titleTV;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
xkcdImgIV=(ImageView) findViewById(R.id.xkcdImgIV);
titleTV=(TextView) findViewById(R.id.titleTV);
try{
new AsyncImg().execute(new String[]{"http://www.xkcd.com/"});
} catch(NullPointerException e){
Log.d(TAG,"Null pointer exception")
}
}
private class AsyncImg extends AsyncTask<String,Void,Void>{
Document doc;
Elements elXkcdTitle;
Elements elXkcdImgUrl;
#Override
protected Void doInBackground(String... arg) {
try {
doc = Jsoup.connect(arg[0]).timeout(0).get();
elXkcdTitle=doc.select("#ctitle");
elXkcdImgUrl=doc.select("#comic img");
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
protected void onPostExecute(String result) {
titleTV.setText(elXkcdTitle.first().text());
ImageLoader imageLoader = ImageLoader.getInstance();
imageLoader.init( ImageLoaderConfiguration.createDefault(getApplicationContext() ));
imageLoader.displayImage(elXkcdImgUrl.first().attr("src"), xkcdImgIV);
}
}
}
There is no network activity. I've given the Internet access permission to the app. Moreover Logcat doesn't show any error. Both Jsoup and Universal Image Loader JARS have been included in build path. Feel free to ask for additional info.

Pass String to Activity

I am new to Android development and Java and was wondering if somebody could help me with the following:
I have created an application that runs a server thread listening on a specified port. I would like to print messages received from a connected client into a TextView in the activity.
The server thread is in a separate class. The run method in this class listens for a client connection and reads any data received into a String.
What would be the best way for me to transfer the contents of this String back to the activity so that it can update the TextView?
From my (limited) understanding, only the ui thread should update a TextView and I can't find a way to get runOnUiThread to update the TextView.
Added code as requested.
Activity code:
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
TextView messages = (TextView) findViewById(R.id.messages);
try {
newThread server = new newThread(this, messages);
} catch(Exception e) {
Toast.makeText(ChatActivity.this, e.toString(), Toast.LENGTH_LONG).show();
}
}
Run method in newThread class:
public void run()
{
serv = new ServerSocket(8000);
while(true)
{
cli = serv.accept();
user = cli.getInetAddress().toString();
BufferedReader cli_in = new BufferedReader(new InputStreamReader(cli.getInputStream()));
OutputStreamWriter cli_out = new OutputStreamWriter(cli.getOutputStream());
while((buf = cli_in.readLine()) != null)
{
// Update the messages TextView with buf
}
}
}
To avoid making things too cluttered I have omitted what irrelevant code I can.
Basically, in the inner while loop in run() I would like to pass the "buf" String to the activity so that the messages textview can be updated with it's content.
Cheers
Maybe a bad idea, but how about using AsyncTask? Didn't try if this would work, but it just might, since onProgressUpdate has access to UI thread.
private TextView messages;
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
messages = (TextView) findViewById(R.id.messages);
ReceiveTask receive = new ReceiveTask();
receive.execute(100)
}
private void updateTextView(String text)
{
messages.setText(text);
}
private class ReceiveTask extends AsyncTask<Integer, String, Long> {
#Override
protected void onPreExecute() {
}
protected Long doInBackground(Integer... urls) {
newThread nt = new newThread();
while(true)
{
publishProgress(run());
}
return (long)0;
}
protected void onProgressUpdate(String... value) {
updateTextView(value[0]); //method in Activity class, to update TextView
}
protected void onPostExecute(Long result) {
}
}
Basically publishProgress will send data to onProgressUpdate, which will then send data to method (updateTextView) in main class and update TextView.
Usually it helps if you tell people you're working on a chat. Also, run() will need to be modified, to return string back, and remove while(true) loop from it. This is NOT the best idea, I suggest you go through a few tutorials on how to make an android chat first.

Categories

Resources