I have been dealing with a twitter application in android. I used Oauth authentication and did some search for a given keyword.
The problem is rate limit. I can't perform to many searches.
Well after an exhaust search I came across with that link:Using the Search API
It says that:
As users, the best thing for you to do is perform your search from twitter.com/search
Then I found an application in Google Play Twitter Search which doesn't even use authentication and performs unlimited searches.
I doubt Twitter Search developer has used that logic.
Would it be possible to make the search directly from twitter.com/search ?
If yes, what libraries/methods/ways should I follow?
Would it be possible to make the search directly from
twitter.com/search ?
Technically yes, but that should be avoided if at all possible. You would have to parse the HTML and that is a both slow and error prone process.
What libraries/methods/ways should I follow?
You should stick to the official API. I realise that this rate limit is kind of in the way, but I am confident you can mitigate the problem by reducing the overall amount of calls to the API and by more efficiently using each call. You have to remember that this is the official Twitter API. Twitter certainly wants for Twitter apps to be good and awesome and while the rate limit of 100 calls per hour seems pretty low it should normally be more than enough. Should the rate limit be reached anyway you just need some proper error handling. I know for a fact that most third party twitter apps just kind of freeze their content when reaching the rate limit. An error is displayed to inform the user that he cannot update anymore for an hour, but he can still continue using the app as before with all the cached data from the previous calls.
Since you haven't posted any code I am not sure what else to tell you just try to be smart about each call. Use cached data as much as possible and only perform API calls when absolutely necessary.
EDIT: I think parsing data from https://mobile.twitter.com/search would be best in your case. Just look for the div with the class timeline. Every <table /> in this div contains one tweet. It shouldn't be that difficult to parse.
I tested it using HtmlCleaner:
HtmlCleaner cleaner = new HtmlCleaner();
TagNode root = cleaner.clean("https://mobile.twitter.com/search?q=asdf");
... // Filter out div with class "timeline"
List<? extends TagNode> tweets = timeLineDiv.getElementListByAttValue("class", "tweet", false, true);
Related
How can I show real time database changes in a flask website?
Like on_update and on_insert, the data will be pushed to the website for the user to see.
I want to get alert from an IOT device and insert it to the database and the users who are subbed to that device should get the real-time alerts.
So I thought
>IOT detects
>HTTP POST to database
>Flask App detects the database change
>push to clients on web app and android
I made a web app that queries the DB with flask-sqlalchemy but thats it, these are supposed to be real-time alerts! I'm so frustrated it's been a week. I am going nowhere and I feel so lost now.
>polling
>web sockets
>SSE
>flask sse
>use AJAX
>use JQUERY
HHHOOOOOWWWWW?????? Most of the examples are for chat apps, and I see NO method where you listen to database changes and send it to clients ;(
A very easy way to implement this is using an event SaaS like pusher.com. This should get you set up in no time. They have examples for many different languages and it should fit your need perfectly.
You mention that you see so many chat-apps as examples. This is pretty normal as they are almost the "hello-world" of event-based systems. And that's exactly what you are looking for. An event happens and you want to trigger something on that event.
For chats, the event is: "The user entered a message" and the triggered action is: "Display it to every connected/subscribed user".
Next to a hosted service like pusher.com, you can roll your own. All the required technology are free and open standards. You could use websockets or WebRTC for this. Websockets is easier but it's trickier if you have many open channels. On the other hand, WebRTC scales but it's a bit more complicated to set up. But you won't need video or audio which makes it easier.
You mention "polling" and I am not sure whether you mean "normal" polling or long-polling as in Comet. That is of course an option. It is the easiest solution, but not the cleanest.
SSE seems like a valid option as well. Although I have no personal experience with it yet. But it seems like it's designed for this use-case.
AJAX and jQuery are less related. AJAX is just an umbrella term for programming using XmlHTTPRequest and is used in all solutions mentioned above. jQuery is just a JS framework and is completely unrelated to the task.
long story short: All your mentioned options allow you to do this. I would suggest looking at pusher.com to get started. And their examples have exactly what you need. Their free plan is already really generous for personal projects. If your application outgrows the free plan you can either pay, or roll your own solution.
The situation: I have many real life locations with specific information associated with them, and updated frequently. I am unsure of how to store this information for use in an android application.
My original thought was storing the data on some server/cloud source/database, reading from the server from each Activity in the app to make sure the info is up to date, and update the server with any changes that may or may not have been made.
For example: there are 200 people inside the library, one person leaves.
So we would read the number of people from the server, display this on the app, person leaves, subtract one, send the new number back to the server.
Would this be an incorrect approach? I'm fairly new to Android in general, and I really have no experience on how to approach this type of situation, what services to use, etc.
I would look into using Parse, its a pretty sweet way to power the backend, and their website is very detailed in explaining how to use it.
I'm working on an Android app that works intensively with the Twitter search api, retrieving twitts that contain certain keywords, and then tokenizing other words in them. My question is the following: form an architectural point of view, is it better to do all the work inside the app or to retrieve the information from a webservice? (I was considering this last option at f
first, but I'm afraid that even caching the results wouldn't be enough to bypass the limit rates of the Twitter api).
i am a newbie android coder.
i am writing a practicing app to search Movie name. I have made XML and java which has a textbox for user to type movies name, but i don't know how to search this over internet!
as i know i have to use IMDb or TMDb API, but i have no idea how to use it! i found this site :
http://www.javacodegeeks.com/2010/10/android-full-app-part-2-using-http-api.html
but there is no explanation for codes. and also i didn't found any other learning.
can somebody please write a full explanation for how to use IMDb or TMDb API for newbie?
it would be a great help to new coders like me! :)
you are most likely going to interact with these api using HttpClient. Go thru those examples first, like pulling in twitter feeds etc. Then you will be ready for the specifics of IMDb. So you are going to have to
1) Determine the base request url. Maybe it is imdb.com/api (it will be in the documentation).
2) you might need to sign up for a key which you will pass over as a parameter. (also in the documentation)
3) read the documentation to determine if you are going to use get/post since it effects how you encode the parameters. One of those parameters might be the key or you might not need a key.
4) In general you should try first in browser client before writing code, just to see what is returned. Then do the same in your code before processing.
5) all http clients are much the same, but determine what you are getting back. Is is JSON, use simple_json to parse. Is it XML, then probably use a SAXParser to handle what is returned. If you have specific questions please post them. The best we can do is give you sort of an algorithm like this as to how you go about it.
Thats really all there is to it. Just make sure you know the right url, if there is a key, if the communication is via get or post, if they are using REST you will encode url without parameters usually. Then its just a matter of parsing what you get back.
The real answer is take it one step at a time. At each step, ask if you have questions. The truth is unless we have used a particular protocol no one knows up front. Trust me, just take it one step at a time, and you will be able to handle any http api.
I already published an android app where you can see a list of specific objects and detailed informations about them. The list changes every day but some of the objects can appear again.
The application is communicating with a PHP server over HTTP and periodically pulls the list of objects.
I now plan to extend the app to make it possible to rate the objects and add a comment similar to how it is done in the android market. I'd like to avoid forcing the user to sign up for an account for being able to comment.
I see two problems:
The comment-system could be abused by spammers
A comment could be added from another system
So my questions are:
How to protect the system from spam?
How to authenticate the application with the server?
How do I limit the number of comments to one per user and object?
What about the androids device id? Is it unique enough to use it as identifier for the user?
Which other problems do you see?
2020 Commenting/Rating/Reviews Options
Since Socialize is out, here are a few options you can explore:
Build your own comment/rating implementation. Personally I love reddit and how it handles nested comments and ratings. Here's a library I found that implements it beautifully. Please note you'll need to tie this with a cloud-database. This is based on groupie. Article & implementation. Many ways to do this - https://stackoverflow.com/a/59472206/668240
Disqus - SDK's coming soon to iOS and Android.
BazaarVoice - commercial
Social Networks SDKs like Facebook, Twitter, etc. Personally I dislike this as we'll need to authenticate users with respective networks to use the APIs. It's like we are shipping off users of our apps to social networks. If you don't have a problem with that - then it might be for you
Legacy Option in 2014:
You can try out Socialize SDK which is open-source and a really good SDK for the rating and commenting you are looking for. It already has a well-functioning Commenting system built-in along with a 'like/love' facility and sharing to FB and Twitter. Each 'entity' (object in your case) can have metadata associated with it. So all you have to do is construct/use a rating widget, then send that rating with the entity attached to your object. To display your rating/comment is as simple as retrieving them from Socialize.
Each object (element from your app) should be associated with an entity which has a unique key in the form of a URL - sort of like a primary key to recognize your items. This entity can have meta-data - any data that you can insert on behalf of your object. Once you do that, you can retrieve that metadata any time you want.
I've been using Socialize for around a year now. They've matured over this period and are always aspiring to be the best at what they do.
Look at the Socialize Bar at the bottom. Its can be customized to your needs.
What's more - Socialize is free.
As for your questions:
There is comment moderation built into the Socialize Web Component
where you can filter out anything you feel is out of place.
Socialize allows you to authenticate through Facebook and Twitter.
Limiting to one comment per user can be achieved by using their User
and Comments API.
Socialize has both Anonymous authentication as well as Social A/c
authentication. I believe you can remove anonymous auth. So that
ensure that every user is authenticated before rating/commenting.
For authentication, you could use OpenID like StackOverflow does or Facebook authentication. Once you have them authentication, it shoud be easy to limit the number of comments to one per user per object. As far as spam, you could follow StackOverflow's model and allow users to vote comments up or down or flag as spam. Perhaps users with comments that have been voted up would have more power and be able to flag comments as spam.
You'll need some sort of rate limiting. I've used this one in this example before.
So you need a table with the user's ID and how many api calls they have left, and then when their last api call was. Then use the algorithm to update the values in the table every time a method is called.
Read through this, I think it should be possible to create an UUID for every case:
http://android-developers.blogspot.de/2011/03/identifying-app-installations.html
And then keep a hidden api key which is hard coded, or at least get's everytime calculated the same or in enigma style influenced by the time it is used. But you will be never be sure, that it won't be find out by crackers/hackers and maybe abused, you will always have this Problem.
Authenticate with the UUID of the user + api-key.