I am developing an android native application for sentimental analysis. The context is I need to get the information from the social media like Twitter and Facebook. I have done it for Twitter by using the API to get the 100 latest tweets about the particular organisation. That is fine.
Now I am trying to do the same for Facebook. Is there any free API available for this sentimental analysis.
I'm assuming to get the Twitter analysis working you used their 'firehose.'
If that is the case, and you're looking for a Facebook equivalent: it doesn't exist. Perhaps it's because the dynamics of these two social networks are not exactly identical and so what each is trying to achieve is different.
While some Facebook users' activities can be completely public (as per their choosing), most of the content is behind walls, so to speak. So unless you scrape the public info and do some analysis on it, as of right now there is no API support for what you're looking for.
If you're doing it for a particular organization they probably have a facebook page. Since almost all pages are public you can use the Facebook API to to query that page and get all the comments from that page. Once you have those comments you can perform sentiment analysis on them. The query might look something like:
$facebook->api("pageName/posts?fields=created_time,id,message,story,type,picture,shares,likes,comments.limit(5000)");
The above example is in php but you can find an equivalent Java example.
Since there is no stream like functionality for Facebook you would have to set, maybe a cron job, to run it every hour to get the latest comments.
Related
I'm currently building an Android app that makes use of the Facebook API. I've gotten to the stage where I want to be able to retrieve a user's Newfeed i.e. the one you see at https://www.facebook.com/home.php. It seems that was possible using the /{user-id}/home endpoint but of course, that's now depreciated and they suggest using /{user-id}/feed but it seems entirely different since the home endpoint states
returns posts a person sees in their News Feed
however the feed endpoint states
links published by this person, or by others on this person's profile
which suggests the user's timeline instead.
Was that the only was of retrieving the newfeed? And is there any new endpoint I can use? I've tried looking already but it seems getting the user's newsfeed is no longer possible, is that correct?
Use feed. It's basically a replacement for home, but yes, it only gets the posts they are tagged in and such that would appear on their timeline.
They removed the functionality you're looking for when they removed home.
Documentation.
I have been dealing with a twitter application in android. I used Oauth authentication and did some search for a given keyword.
The problem is rate limit. I can't perform to many searches.
Well after an exhaust search I came across with that link:Using the Search API
It says that:
As users, the best thing for you to do is perform your search from twitter.com/search
Then I found an application in Google Play Twitter Search which doesn't even use authentication and performs unlimited searches.
I doubt Twitter Search developer has used that logic.
Would it be possible to make the search directly from twitter.com/search ?
If yes, what libraries/methods/ways should I follow?
Would it be possible to make the search directly from
twitter.com/search ?
Technically yes, but that should be avoided if at all possible. You would have to parse the HTML and that is a both slow and error prone process.
What libraries/methods/ways should I follow?
You should stick to the official API. I realise that this rate limit is kind of in the way, but I am confident you can mitigate the problem by reducing the overall amount of calls to the API and by more efficiently using each call. You have to remember that this is the official Twitter API. Twitter certainly wants for Twitter apps to be good and awesome and while the rate limit of 100 calls per hour seems pretty low it should normally be more than enough. Should the rate limit be reached anyway you just need some proper error handling. I know for a fact that most third party twitter apps just kind of freeze their content when reaching the rate limit. An error is displayed to inform the user that he cannot update anymore for an hour, but he can still continue using the app as before with all the cached data from the previous calls.
Since you haven't posted any code I am not sure what else to tell you just try to be smart about each call. Use cached data as much as possible and only perform API calls when absolutely necessary.
EDIT: I think parsing data from https://mobile.twitter.com/search would be best in your case. Just look for the div with the class timeline. Every <table /> in this div contains one tweet. It shouldn't be that difficult to parse.
I tested it using HtmlCleaner:
HtmlCleaner cleaner = new HtmlCleaner();
TagNode root = cleaner.clean("https://mobile.twitter.com/search?q=asdf");
... // Filter out div with class "timeline"
List<? extends TagNode> tweets = timeLineDiv.getElementListByAttValue("class", "tweet", false, true);
I am not sure about how can I tell that a tweet is tweeted from a mobile/smart phone.
I am using Tweepy for the twitter API. Twitter API can only tell us the source/client of a tweet (e.g. Twitter for Android).
That's why the only solution I see, is to compare the name of the client used to tweet to a list of mobile clients. (I should build the list by myself)
The list can be huge, that's why I am searching for another suggestion, hack or magic.
(Alternatively, do you know where can I find a good list of mobile app?)
As you've discovered, you can look at the source parameter.
This will give you two interesting things
The name of the client.
The URL of the client.
I'd suggest a two pronged approach. Look to see if the name says something like "for Android" or "mobile" - and check whether the URL points to iTunes, play.google, or similar.
According to Wikipedia
A new app is registered every 1.5 seconds, according to Twitter.
I would suggest looking at the Top 100 Twitter Applications (or similar) and then build your own list of the most commonly seen apps.
I'm working on an Android app that works intensively with the Twitter search api, retrieving twitts that contain certain keywords, and then tokenizing other words in them. My question is the following: form an architectural point of view, is it better to do all the work inside the app or to retrieve the information from a webservice? (I was considering this last option at f
first, but I'm afraid that even caching the results wouldn't be enough to bypass the limit rates of the Twitter api).
I already published an android app where you can see a list of specific objects and detailed informations about them. The list changes every day but some of the objects can appear again.
The application is communicating with a PHP server over HTTP and periodically pulls the list of objects.
I now plan to extend the app to make it possible to rate the objects and add a comment similar to how it is done in the android market. I'd like to avoid forcing the user to sign up for an account for being able to comment.
I see two problems:
The comment-system could be abused by spammers
A comment could be added from another system
So my questions are:
How to protect the system from spam?
How to authenticate the application with the server?
How do I limit the number of comments to one per user and object?
What about the androids device id? Is it unique enough to use it as identifier for the user?
Which other problems do you see?
2020 Commenting/Rating/Reviews Options
Since Socialize is out, here are a few options you can explore:
Build your own comment/rating implementation. Personally I love reddit and how it handles nested comments and ratings. Here's a library I found that implements it beautifully. Please note you'll need to tie this with a cloud-database. This is based on groupie. Article & implementation. Many ways to do this - https://stackoverflow.com/a/59472206/668240
Disqus - SDK's coming soon to iOS and Android.
BazaarVoice - commercial
Social Networks SDKs like Facebook, Twitter, etc. Personally I dislike this as we'll need to authenticate users with respective networks to use the APIs. It's like we are shipping off users of our apps to social networks. If you don't have a problem with that - then it might be for you
Legacy Option in 2014:
You can try out Socialize SDK which is open-source and a really good SDK for the rating and commenting you are looking for. It already has a well-functioning Commenting system built-in along with a 'like/love' facility and sharing to FB and Twitter. Each 'entity' (object in your case) can have metadata associated with it. So all you have to do is construct/use a rating widget, then send that rating with the entity attached to your object. To display your rating/comment is as simple as retrieving them from Socialize.
Each object (element from your app) should be associated with an entity which has a unique key in the form of a URL - sort of like a primary key to recognize your items. This entity can have meta-data - any data that you can insert on behalf of your object. Once you do that, you can retrieve that metadata any time you want.
I've been using Socialize for around a year now. They've matured over this period and are always aspiring to be the best at what they do.
Look at the Socialize Bar at the bottom. Its can be customized to your needs.
What's more - Socialize is free.
As for your questions:
There is comment moderation built into the Socialize Web Component
where you can filter out anything you feel is out of place.
Socialize allows you to authenticate through Facebook and Twitter.
Limiting to one comment per user can be achieved by using their User
and Comments API.
Socialize has both Anonymous authentication as well as Social A/c
authentication. I believe you can remove anonymous auth. So that
ensure that every user is authenticated before rating/commenting.
For authentication, you could use OpenID like StackOverflow does or Facebook authentication. Once you have them authentication, it shoud be easy to limit the number of comments to one per user per object. As far as spam, you could follow StackOverflow's model and allow users to vote comments up or down or flag as spam. Perhaps users with comments that have been voted up would have more power and be able to flag comments as spam.
You'll need some sort of rate limiting. I've used this one in this example before.
So you need a table with the user's ID and how many api calls they have left, and then when their last api call was. Then use the algorithm to update the values in the table every time a method is called.
Read through this, I think it should be possible to create an UUID for every case:
http://android-developers.blogspot.de/2011/03/identifying-app-installations.html
And then keep a hidden api key which is hard coded, or at least get's everytime calculated the same or in enigma style influenced by the time it is used. But you will be never be sure, that it won't be find out by crackers/hackers and maybe abused, you will always have this Problem.
Authenticate with the UUID of the user + api-key.