I have just become a SHOUTcast partner and received my DevID.
I read pretty much everything here:
SHOUTcast_Radio_Directory_API
and here: SHOUTcast_API_License_Agreement
And I have a tough question: should I cache the results from their API calls?
If decided to cache, I would be going against what is requested in the SHOUTcast Radio Directory API:
Do not cache the XML for more than 1 day, as station ID's can and will change.
If I did not cache anything, I would be going against the SHOUTcast API License Agreement:
Please do not hammer the servers. We request reasonable usage and recommend that you utilize local caching.
Moreover, let's suppose I decide to cache results. Since those results are 100% based on users' queries, wouldn't it be somewhat silly to cache those results, as my app's users would probably enter different queries every time????
In an effort to try to solve my problem, I had the idea of downloading the entire SHOUTcast database, once a day/week, and then, upload it to my own server. Afterwards, my app would send all queries to my server, and wouldn't bother SHOUTcast's servers at all.
The problem with such an approach, though, is that I could not find a way to download the entire SHOUTcast database at once, like we can do with IceCast's dir.xiph.org/yp.xml.
What should I do next???? How should I proceed?
I'm afraid of hurting the SHOUTcast API License Agreement and getting my DevId revoked:
We reserve the right to revoke access for DevIDs which abuse the system.
Has anyone here ever created an application that connects to SHOUTcast's directory? If so, which approach have you used not to be banned?
Just for the sake of completeness, I will answer my own question with the solution I came up with.
After a lot of research I decided to create a 30-minute cache, from which I retrieve the last results. Should the user change the search filter, or should the cache expire, all cached results are cleared, a new request is made to SHOUTcast's servers and a new cache is created.
The code can be found here:
ShoutcastRadioStationList.java
RadioStationCache.java
Related
For better UX mobile apps store data on the client side (on the device) to provide immediate information when loading an app without having to wait for data from the internet and providing data even when the device is offline. Of course data is updated/fetched whenever possible later on.
I am building an app (in flutter) which is a social network/information feed like app: there are users, profiles, feed, posts etc. When the user opens the app I would like to show data that was available the last time the app ran.
My question is what is the right way to implement cache? There are two main ways I can think of, an easier/uncertain way and a more difficult/stable way, and I would like your opinion about them. I have time/resource constraints ofc. Most information is through HTTP requests, so:
The easier way: HTTP Cache interception
I would use an out-of-the-box cache plugin for my HTTP client. I think I can just cache the response for each request I make (for some time) and rely on the cached info. Images are also cached based on url. When I make a request on application load I return the cached result if there's any, and if it was a cache hit I fire the request again, so when you open the app you will see immediate information, but after a sec or so you will get the fresh data too. Usability of this solution ofc depends on how well I design my API.
Harder: Store data in a structured database.
This is the option I try to avoid, because it's more time implementing this. It could be either a SQL or document store, and I would have to implement the cache look up/save/update mechanism. Since I am just building the app, I think this would slow me down because data types/ architecture might still change. But is this the ultimate way to go with mobile side caching?
Thank you
I think the easier way is your best bet the only time i can think of that cache could be a problem is if you need critical data that has to be correct and not a old cache value, but you can avoid this problem by not caching the critical values.
Also if you use firebase it does some automatic caching which might be useful.
I am running into an issue with rate-limiting and think I have found a good solution.
I would like to create a server which periodically requests data from 2 different APIs (Twitter, RSS feed) and caches it on the server. The client would then request the data from the server and display it. I only have limited experience in server-side development. Can anyone provide any tips or point me in the right direction in order to implement this. I was thinking about using Amazon Web Services to host the server? Any pointers are much appreciated! Thanks!
Lots of ways to do this, but if you want to use AWS, you could choose S3 as the place to store a copy of the RSS feed (perhaps in JSON format), and your android ap could just request it from there.
S3 is super cheap, super redundant and super fast - plus you can use http://aws.amazon.com/cloudfront/ and push out copies to geographically closer regions around the world (for pennies), if you think you would benefit from that reduction in latency (and assuming you have geographically diverse users).
The only part you would still need to do would be a small/simple back-end process that polls the two source API's, on a schedule you determine, and writes the data to S3.
For something like this a micro-EC2 instance would probably be plenty of horsepower and only cost a few dollars per month.
I think you might have a bigger privacy concern if you started saving other peoples data yourself (I'm not sure how many people would be ok with you to saving their Twitter data on your server).
I'm sure you can cache Twitter and RSS data locally, and just ask for whatever has changed? which might reduce the number of requests your sending?
Alternatively, you could save the API key your using on a web service and get your apps request that key each time. When you've run into limitations with one API key, just switch to using another? Google actually recommend this method for using their own API's (this way you can revoke an API key easily without having to upload a new app).
Hope that makes sense,
Danielle
This is a theory question more than an implementation question. What, in your opinion, is the best way to create a mobile application that syncs data to a server?
I have been writing an application that has a user sign-in, allows them to create notes and then selectively share them with other users. I have been doing this with a Rails webapp that returns JSON data to my iOS app. It seems like a lot of overhead for something that so many apps are doing. Is there a better way? How would, or do, you do it?
You should optimize the data quantity you exchange between server and device. You need to set up a flag that indicate you if something changed, case you need to sync.
Let say you have an app that allow to a group of users to update/load from the same file. you can save on the server the time of the last changes, and on mobile device you have the last update time you get. When you want to update your app data, on the request you can include the time you got the last updates. if the time you send and the time you have on the server differs, update; else... do nothing.
Because the request/response is minimal (Req = time; resp = empty), you can check for updates as often as you like.
The option I found that proved to be the easiest and allows you to focus on mobile development while virtually ignoring the data storage element is Parse. I wish I had found this months ago.
I really don't think there's necessarily anything wrong with maintaining a simple Rails web service as a REST interface for your database, but I can see how that might seem like unnecessary overhead. You could find a DB which has a REST interface by default. Here are two to start you off:
CouchDB
Amazon SimpleDB
I have a database and I need to present a data via HTTP using JSON web services. Currently, I'm designing JSON datasets that will be provided as web services. The data from the tables will be aggregated to suite an app needs.
If the data size is large and we will try to download that at once then it might take too much time and an app will not be responsive at start. That's bad. It's well known we should minimize the number of http requests the app will make to download the data. However if we would split the data into small chunks then during every app action step an app will performe http requests and that might be unresponsive solution too.
We are talking here about mobile app development, so internet will be exposed over cellular ISP or wifi, so the speed might be quite slow. I understand the split process depends on app workflow process and etc., just curious are there any general guidelines? For example: if JSON data is larger than 1MB then definitely split it into smaller chunks...
Look at how your mail read work. You probably have tens of thousands of emails in your account. The app will show the first ones then provide a button at the bottom of the list to display more items. It's a pretty good solution usually to provide a lot of data.
Also #Selvin ideas are just great. Don't use UI thread to download stuff, use a different thread. Services are pretty good for getting data asynchronously.
One of the way is to create a service which gets started when there is network. All downloaded data can be cached in sqlite and use a content provider to get the data. But it depends on your app.
Sometimes it depends on your UI Screen. For eg. You can create a Listview with load more. On click of it load extra data.
Other way is to create API such as which gives only updates based on timestamp. But its all depends on the app. Just sharing my ideas. Might not be perfect. Others can surely give a better one.
I'd like a bit of advice on how to retrieve, process and store data.
I'm building an app which gets finds the nearest laser tag site to where you are. The adderss data is stored (due to some bad design) in one field in an external database, with it's country coming from another table (told you it was a bad design). I may change where it's getting it's data from if I can, as it currently requires a lot more JSON parsing.
Anyway, as I've got the address, I need to get the locations and put them on the map. As there's several hundred sites to add, it takes a while to request the co-ordinates for all of them, and process them etc.
So my question is this: would it be better to use the google georeader on android, which I'm having problems with, or to use a web server to send the requests via google's http geocoding requests.
Also, would it be better to then store the data on the phone and check for updates every time it loads, or just not store anything on the phone and get all the data every time it loads?
Ta muchly!
So my question is this: would it be
better to use the google georeader on
android, which I'm having problems
with, or to use a web server to send
the requests via google's http
geocoding requests.
It's not like laser tag locations move terribly frequently. Do a one-time geocoding lookup on your server, cache the data in a local database, and serve out of the cache. However, there may be legal implications here, see below...
Also, would it be better to then store
the data on the phone and check for
updates every time it loads, or just
not store anything on the phone and
get all the data every time it loads?
That's actually more of a question of law than of coding, IMHO. From a coding standpoint, if the database is small, keeping it on-device and updating it periodically would be OK. However, if it is not your data, it will depend on whether or not it is legal for you to hold onto it and copy it around.