Develop Reference

How to get collection from Meteor server with Android DDP?

Let me to start explain my problem. There is repository with some explanations, but there are no methods how to get collection or json file from Meteor server(only insert). Also author did not explain properly methods onDataChanged, onDataAdded etc.
public class Login extends Activity implements MeteorCallback{
public static Meteor mMeteor;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_login);
mMeteor = new Meteor(this, "some_socket_it_doesn't_matter");
mMeteor.setCallback(this);
}
}
public class ListOfElements extends ListFragment implements MeteorCallback{
#Override
public void onActivityCreated(Bundle savedInstanceState) {
super.onActivityCreated(savedInstanceState);
String subscriptionId = Login.mMeteor.subscribe("notifications");
Log.d("Log", subscriptionId);
}
}
I didn't understand how i have to use subscription or how to get collection from server. Why there are only insert methods in github repository and no get? I really have no idea how make the code to get collection, use subscribe and so on. There are no any understandable explanations in the network. Please, can you help me with this by explaining how to realize getting, subscribing in this code.

There are two special things about Meteor: It works asynchronously and it has been designed specifically for real-time applications. Thus it has a few different concepts for retrieving data and for some other tasks.
In a synchronous application, you would just call insert(...) and immediately get the method's return value, e.g. a boolean value for success/error or a numeric value for the number of rows that have been inserted.
You would call get(...) and immediately receive a collection of rows as the method's return value.
But in Meteor, everything is asynchronous. This means that you get the results not immediately, but a few (milli)seconds later, in a callback method.
When you call insert(...), this is not so important, as you have noticed. You just call this method and often forget about the result, i.e. you don't wait and check for the result because insertions are usually successful. But this method is still asynchronous and you could (and sometimes should) listen for the result which will arrive a few (milli)seconds later, again.
When you want to call get(...), this would be possible in theory, with the important point again being that it's asynchronous. So you would say "get me all chat messages from the last 5 minutes". There would be no result or return value, as usual, but the result would arrive a short time later, asynchronously, in a callback method that you define. This is what onDataAdded(...), onDataChanged(...) and onDataRemoved(...) are for.
Now it's not clear, yet, why you can't call get(...) and wait for data to arrive in those methods.
The answer to that question is Meteor being designed for real-time applications. This is why you can't say "get me all chat messages from the last 5 minutes". Instead, you have to say "I want to subscribe to all chat messages from the last 5 minutes and always be updated about changes".
So, in Meteor, you subscribe to data sets instead of requesting them via get(...).
All in all, this means the following:
If you want to get some messages, you subscribe to your data set that holds those messages.
When the initial rows are sent (!) and whenever new rows are added to the collection, you receive those in your onDataAdded(...) callback. When rows are modified, you receive those changes in your onDataChanged(...) callback. And, finally, when rows are deleted, you are informed about those deletions in your onDataRemoved(...) callback.
When you don't want to get updates for your data set anymore, you unsubscribe from that set. This is optional.
With the Android-DDP library in your Android application, it translates to the following:
final String subscriptionId = mMeteor.subscribe("chats");
public void onDataAdded(String collection, String docID, String json) { ... }
mMeteor.unsubscribe(subscriptionId);
As you can see, what you have to learn is really Meteor and not the library Android-DDP. Meteor has some new concepts that one has to understand. But when you know how Meteor works, translating those things to Android-DDP is really simple and only a matter of looking up the method names.

Query efficiency with Parse using local store in Android

My question is very simple, what is the best approach to work with Parse using the local store at the time I want to query the saved objects.
Is it better to trigger several queries to the local store directly on the main thread and avoid nesting a lot of anonymous classes or using a background thread?
It's important thing to notice is that this method is going to be called very frequently and the pattern will be repeated in several places with different queries. I'm evaluating both efficiency and code quality in readability. These methods will be called synchronously so we can assume the data will be consistent at any time.
As the objects are being saved locally I would expect the queries to be very fast in response. Here's a rough sample of how the code would look like in both cases.
Option one:
public void processBatches() {
ParseQuery<Batch> batchQuery = Batch.getQuery();
int batchCount = batchQuery.fromLocalDatastore().count();
List<Batch> batches = batchQuery.fromLocalDatastore().find();
for(Batch b : batches) {
// do whatever I need to do
}
}
Option two:
public void processBatches() {
ParseQuery<Batch> batchQuery = Batch.getQuery();
int batchCount = batchQuery.fromLocalDatastore().countInBackground(new CountCallback() {
#Override
public void done(int i, ParseException e) {
if (i > 0) {
batchQuery.findInBackground(new FindCallback<Batch>() {
#Override
public void done(List<Batch> list, ParseException e) {
for (Batch batch : list) {
// do whatever I need to do
}
}
});
}
}
});
}

Well since in option one you are blocking the UI thread, there could be a delay in the user's ability to interact with your application. This is not a very good option since even if it is for just a moment, users don't want to be waiting unless they know operations are happening. But, if you know that at any time there will be little to no delay, go ahead and do it.
Nevertheless, I argue that option two is going to be the best option. This is because, in general, all network operations should be performed in the background. Although in your case you are performing local datastore queries, suppose a user has gone to their application task manager and cleared the data (very rare this will happen) what happens now when you perform the find from local data store and processing of Batch objects? Well, the app crashes. Again, this is not a very good option for the usability for your application.
Choose the second option, and allow an AsyncThread to run the find() and count() query operations to the network if there is nothing found from local data store queries. Also, from the Parse documentation for find:
public Task<List<T>> findInBackground()
Retrieves a list of ParseObjects that satisfy this query from the source in a background thread.
This is preferable to using ParseQuery.find(), unless your code is already running in a background thread.
Returns:
A Task that will be resolved when the find has completed.
Parse's creators prefers that the users of their API use a background thread to perform operations.

It really depends.
Is the user triggering the update? If so then do it on the main thread because you don't want them waiting
If not, then is the data access a result of fetching data from the web (and hence you should already be on a background thread) so could probably just remain on the background thread
Also what happens in "// do whatever I need to do"? Is it an update to the UI or more background processing?

Recommended way / order to read data from a webservice, parse that data and insert it in a SQLite db

I'm one to start mentioning that I'm totally new to Android, I've just finished reading a quick, introductory book and now I have to implement my very first app.
This app is going to be use to take orders. Among all the data I'm going to store in a local db, two tables are the most important: Customers and Articles, being the latter the largest of all the tables (aprox 20000 records)
One of the main process in my app, fetches all the data that my app need to work off-line when the user press a button that starts the daily operations on the device.
Well, the process consists of the following steps :
a. Read a restful service to retrieve the Customers Data
b. Parse the response to Json Objects
c. Insert those customers to the Customers Table
d. Read a restful service to retrieve the Articles Data
e. Parse the response to Json Objects
f. Insert those articles to the Articles Table
This is what I've planned to do:
Write a helper class that handles all the HTTP GET requests. Then call this class whenever I need to download all the Customers and Articles data
Since all this process might take a lot of time, I need to do it the background. So based on some suggestions I'm going to put all this code inside a Bound Service.
While all this long processing is taking place in the background I'll have to show some sort of indicator (a ProgressDialog) This is the reason I opted for using a Bound Service
Though I think I've got the general idea of how to do most of these thing separately, I think that putting the all together is quite a different story.
So these are the questions I've got now that I have to put the puzzle together:
Do you think the order in which I'm executing all the 6 steps of the process described is correct / efficient? If you had to make some changes, what would you change?
If the activity that started the service is explicitly cancelled or is hidden by another activity, the service has to have a way to let the user know that the process has finished. How could I implement that?
Is it possible/ recommended to write to the SQLite DB within the service? Is it the same as when I do so within an activity?
In J2ME I've done something similar, and when I put something like the 6 steps I mentioned above all of them are executed sequentially, that is , one after the other. Is it the same in Android?
I hope I'm not asking too many questions. I just put them together because they are all related.
Basically in this question I'm not asking for working code ( though it'd be OK if you could provide some sample code) What I'm really after is some suggestions, some guidance. Something like "Hey, I think this might help you with point number 3" or "You might find this article useful", "I think you'd better off using this instead of that". That kind of thing.
I decided to come to you because you're the experts and I really need someone to put me in the right direction.
Thank you very much.
P.S. Please do not close this question, if your think I need to change something just let me know and I'll do it.

I decided to come to you because you're the experts
first i am not expert and also i am not knowledgeable you can find more expert people than me but this is my opinion hope to give you some help.
first of all forget to use AsyncTask to download because it must be used for short background jobs not like yours i think the amount of file you want to download is pretty large.(i think so)
check downloadmanager of google to see how it works it may help you.
http://developer.android.com/reference/android/app/DownloadManager.html
http://blog.vogella.com/2011/06/14/android-downloadmanager-example/
if you want to use service use unbound service because you do not want the service to be destroyed by android or user when the user close the apps do you? i think you want to get your data any way.
in order to be efficient i recommend these steps:
a. Read a restful service to retrieve the Customers Data
b. when you get Customer data do :
create another service or thread to Read a restful service to retrieve the Articles Data
Parse the response to Json Objects on your old service or thread
now you have 2 services or threads that run concurrently so follow the steps that obvious insert parse and so, on each service or thread.
why do not i combine a and d? because i think user do not like to wait much time behind download progress bar.
in order to insert your data to database use transaction and i recommend you use:
http://greendao-orm.com/
it is more efficient ORM than others for database and you get free from db implementation.
If the activity that started the service is explicitly cancelled or is hidden by another activity, the service has to have a way to let the user know that the process has finished. How could I implement that?
use notification:
http://developer.android.com/training/notify-user/build-notification.html
http://www.tutorialspoint.com/android/android_notifications.htm
http://www.vogella.com/tutorials/AndroidNotifications/article.html
While all this long processing is taking place in the background I'll have to show some sort of indicator (a ProgressDialog) This is the reason I opted for using a Bound Service`
how can I update the UI from an Unbound Service?`
Use a LocalBroadCastManager, or in general BroadCastReciever
Android update activity UI from service
In J2ME I've done something similar, and when I put something like the 6 steps I mentioned above all of them are executed sequentially, that is , one after the other. Is it the same in Android?
this is depends on your steps, if you follow my idea you run concurrently and if you run your idea you will run sequentially.
Good Luck.

I did something like yours.
In first step I get data from webservice in HTTP GET of POST method using AsyncTask like this:
public class GetService extends AsyncTask<String, String, String> {
private String mRestUrl;
private ServiceCallback mCallback;
private final HttpClient Client = new DefaultHttpClient();
private String Content;
private String url;
private String Error;
private ProgressDialog barProgressDialog;
private ProgressDialog Dialog;
public GetService(String restUrl, ServiceCallback callback) {
this.mRestUrl = restUrl;
this.mCallback = callback;
this.url = restUrl;
Dialog = new ProgressDialog(AppContext.CurrentContext);
}
#Override
protected void onPreExecute() {
super.onPreExecute();
}
#Override
protected String doInBackground(String... urls) {
Content = null;
BufferedReader reader = null;
try {
StringBuilder builder = new StringBuilder();
HttpClient client = new DefaultHttpClient();
HttpGet get = new HttpGet(this.url);
HttpResponse response = client.execute(get);
int status = response.getStatusLine().getStatusCode();
if (status == 200) // sucess
{
HttpEntity e = response.getEntity();
// String data = EntityUtils.toString(e);
InputStream content = e.getContent();
reader = new BufferedReader(new InputStreamReader(content));
String line;
while ((line = reader.readLine()) != null) {
builder.append(line);
}
Content = builder.toString();
} else if (status == 401) {
return "-Auth Failed Error Code 400";
} else {
return "-Error Code: " + status;
}
} catch (Exception ex) {
Error = ex.getMessage();
} finally {
Dialog.dismiss();
try {
reader.close();
}
catch (Exception ex) {
ex.printStackTrace();
}
}
return Content;
}
#Override
protected void onPostExecute(String result) {
try {
GetService.this.get(20000, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
} catch (TimeoutException e) {
e.printStackTrace();
}
mCallback.onTaskComplete(result);
super.onPostExecute(result);
}
}
and my callback class is:
public abstract class ServiceCallback{
public abstract void onTaskComplete(String result);
}
I call AsyncTask in my code everywhere I want to get data from webservice:
new GetService(url, new ServiceCallback() {
public void onTaskComplete(String response) {
// Parse Response of WebService
}
}).execute();
In second step I Parse response of WebService in onTaskComplete method using json helper Libraries like Gson or Jackson. for example in jackson:
List<YourClass> data = new ObjectMapper()
.readValue(
response,
new TypeReference<List<YourClass>>() {
});
At the end I store Data in Database. for connecting to DB I Prefer to use GreenDao as my ORM. In this way storin data in DB can be done in one line code like this:
//after converting json to object
YourORMDaoClass.insertOrReplaceInTx(data);
To Use GreenDao ORM this link is very helpful;

Do you think the order in which I'm executing all the 6 steps of the
process described is correct / efficient? If you had to make some
changes, what would you change?
It depends. If this data is related and cannot exists without each other then you should change the order like this:
a. Read a restful service to retrieve the Customers Data
b. Parse the response to Json Objects
d. Read a restful service to retrieve the Articles Data
e. Parse the response to Json Objects
c. Insert those customers to the Customers Table
f. Insert those articles to the Articles Table
Steps c and f should be combined in transaction.
Otherwise, the order does not matter. If data is not related then separating these processes and running them in sequence might be a good idea.
If the activity that started the service is explicitly cancelled or is
hidden by another activity, the service has to have a way to let the
user know that the process has finished. How could I implement that?
I suggest to start with implementation of the IntentService class. It handles for you background thread and works like a queue of events where single Intent delivers a data to process.
Actually you could implement one of the patterns presented by Google on one of their IO conferences. I have implemented an option A shown on the video. It works for me really well. The trick is that using ContentProvider from the background automatically updates UI which listen for changes thanks to CursorAdapter.
To update UI progress you can use LocalBroadcastManager or event bus libraries e.g. Otto.
You can also extend your tables and store status and progress, updating tables would automatically update UI as well, just keep in mind that these updates should be rare e.g. control in the service background how often table progress is updated calculating the progress first and checking with service local variable if it's changed.
In case your app is in the background, post status notification. User should be able to navigate back to your app clicking on the notification.
Is it possible/ recommended to write to the SQLite DB within the
service? Is it the same as when I do so within an activity?
You can do it within the Service. Actually, if you follow the pattern I have mentioned above, you would do it in the Processor tier on the background service thread.
In J2ME I've done something similar, and when I put something like the
6 steps I mentioned above all of them are executed sequentially, that
is , one after the other. Is it the same in Android?
It's completely up to you how communication with the server will work. If you decide to use IntentService class then it will work in a sequence which is not a bad idea on Android. On the other hand you may extend Service class directly and implement own thread executor with a thread pool. You can also have dedicated IntentService classes for unrelated operations.
I also recommend to read lessons:
Transferring Data Without Draining the Battery
Transferring Data Using Sync Adapters
If you don't want to play directly with HTTP connection implementation then consider using Retrofit or Volley
If you just need JSON parsers then these 2 are the best:
GSON
Jackson

Fetching big amount of data, what is the best way to go?

I have severals URLs I need to get data from, this should happen in order, one by one. The amount of data returned by requesting those URLs is relatively big. I need to be able to reschedule particular downloads which failed.
What is the best way to go? Shall I use IntentService, Loaders or something else?
Additional note: I would need not only to download, but also post process the data (create tables in db, fill it with data, etc). So DownloadManger can't be of help here.

I would use an IntentService.
It has a number of advantages that are suitable for your needs, including being able to download the data without your application running and supporting automatic restart of the service using setIntentRedelivery().
You can set a number of identifiers for the particular job, you need to perform using Intent extras, and you can keep track of the progress using SharedPreferences - that way you can also resume the work if it's been cancelled previously.

The easiest way is probably to use the system DownloadManager http://developer.android.com/reference/android/app/DownloadManager.html
(answering from my phone, so please excuse the lack of formatting)

I would suggest a service for this. Having service resolves many problems
It would allow reporting of progress asynchronously to the application so you can enable or disable a specific gui in application based on the download status of data
It will allow you to continue the download even if the user switches to other application or closes the application.
Will allow you to establish independent communication with server to prioritize downloads without user interaction.

Try a WakefulIntentService for creating a long-running job that uses wakelocks to keep your task alive and running https://github.com/commonsguy/cwac-wakeful .
Also, if your whole app process is getting killed, you may want to look into persisting the task queue to disk, using something like Tape, from Square

I think the way to go is loading urls in an array, then starting an AsyncTask, returning a boolean to onPostExecute indicating if the operation has success or not. then, keeping a global int index, you can run the AsyncTask with the next index if success, or the same index otherwise. Here is a pseudocode
private int index=0;
//this array must be loaded with urls
private ArrayList<String> urlsArray;
new MyDownloaderAsyncTask().execute(urlsArray.get(index));
class MyDownloaderAsyncTask extends AsyncTask<String,String,Boolean>{
#Override
doInBackground(String... input){
//downlaod my data is the function which download data and return a boolean
return downloadMyData();
}
#Override
onPostExecute(Boolean result){
if(result)
new MyDownloaderAsyncTask().execute(urlsArray.get(++index));
else
new MyDownloaderAsyncTask().execute(urlsArray.get(index));
}
}
hope this help

I have just completed an open source library that can do exactly what you need. Using droidQuery, you can do something like this:
$.ajax(new AjaxOptions().url("http://www.example.com")
.type("GET")
.dataType("JSON")
.context(this)
.success(new Function() {
#Override
public void invoke($ droidQuery, Object... params) {
//since dataType is JSON, params[0] is a JSONObject
JSONObject obj = (JSONObject) params[0];
//TODO handle data
//TODO start the next ajax task
}
})
.error(new Function() {
#Override
public void invoke($ droidQuery, Object... params) {
AjaxError error = params[0];
//TODO adjust error.options before retry:
$.ajax(error.request, error.options);
}
}));
You can specify other data types, which will return different object types, such as JSONObject, String, Document, etc.

Similar to #Murtuza Kabul I'd say use a service, but it's a little complicated than that. We have a similar situation related to constant internet access and updates, although ours places greater focus on keeping the service running. I'll try to highlight the main features without drowning you in too much detail (and code is owned by the company ;) )
android.permission.RECEIVE_BOOT_COMPLETED permission and a BroadcastReceiver listening for android.intent.action.BOOT_COMPLETED to poke the service awake.
Don't link the service to the Activity, you want it running all the time. eg we call context.startService(new Intent(context.getApplicationContext(), OurService.class))
The service class is just a simple class which registers and calls an OurServiceHandler (as in our case we fire off repeated checks and the Handler manages the 'ticks')
We have an OurServiceRunnable which is a singleton which is checked and called by the Handler for each test. It protects against overlapping updates. It delegates to an OurServiceWorker to do the actual lifting.
Sounds heavy handed, but you want to ensure that the service is always running, always ticking (via the Handler) but only running a single check at a time. You're also going to run into database issue if you use the standard SqlLite DbHelper paradigm, as you can't open the DB on multiple threads and you definitely want the internet access off the main thread. Our hack was a java.util.concurrent.locks.ReentrantLock protecting access to the DB, but you could probably keep DB access on the UI thread and pass DB operations via the Handler.
Beyond this it's just a matter of keeping the downloads atomic in terms of "get task, download task, complete task" or enabling it to pick up from a failed state eg downloaded OK, attempt to complete.

You should take a look at the volley library :
http://www.javacodegeeks.com/2013/06/android-volley-library-example.html
There is also an interesting video of the author that took place at google io 2013 :
http://www.youtube.com/watch?v=yhv8l9F44qo
Mainly because it eases the process of managing a lot of these fastidious tasks that are connection checking, connection interruption, queue management, retry, resume, etc.
Quoting from the javacodegeeks "Advantages of using Volley :
Volley automatically schedule all network requests. It means that Volley will be taking care of all the network requests your app executes for fetching response or image from web.
Volley provides transparent disk and memory caching.
Volley provides powerful cancellation request API. It means that you can cancel a single request or you can set blocks or scopes of requests to cancel.
Volley provides powerful customization abilities.
Volley provides Debugging and tracing tools"
Update from dennisdrew :
For large file, better use a variant of volley which authorize using another http client implementation. This link gives more details :
The volley article about this modification :
http://ogrelab.ikratko.com/android-volley-examples-samples-and-demos/
The github file detail :
https://github.com/ogrebgr/android_volley_examples/blob/master/src/com/github/volley_examples/toolbox/ExtHttpClientStack.java

public class FetchDataFromDBThread implements Runnable {
/*
* Defines the code to run for this task.
*/
#Override
public void run() {
// Moves the current Thread into the background
android.os.Process
.setThreadPriority(android.os.Process.THREAD_PRIORITY_BACKGROUND);
FetchDataFromDB();
}
}

Best method for saving data - preferences, sqlite, serializable or other?

I've been investigating alternative methods for saving my game's data between turns, and wonder if anyone can point me in the right direction.
I have approximately 32k of data which must be saved during onPause. I ruled out preferences due to the sheer quantity of data. I spent a few days playing around with SQLite but couldn't get the data to save in less than two seconds (although the time certainly hasn't been wasted).
I've decided that I'll use the database for loading constant data at the beginning of the game. This will certainly make it easier to tweak various parameters and default values in the game. But this still leaves me looking for the ideal method for writing data.
The data that needs to be saved is basically nine occurrences of class A and nine occurrences of class B. I'm an intensive month into the learning curve of Android (and the nuances of Java, coming from a C++ background) and have been googling like crazy. This brought two possibilities to mind -
1) Serialization (ObjectOutputStream)
I thought this would be the perfect solution but, having read several other posts regarding the subject, gather that it isn't highly recommended on the Android platform due to speed and memory allocations provoking the garbage collector into a potential rage.
2) DataOutputStream class
My current thought is to add Load and Save functions to both classes and to use DataOutputStream and DataInputStream calls in them to write and read the data respectively.
The data in the classes are primitives (strings and ints mostly) and arrays of primitives, so there's nothing too complicated in there to break down. Would this second solution seem a good, viable one? Or are there other solutions that I am unaware of as yet?

You should use an Async task to save the data, I used this method to fetch highscores at the start a game:
new HighscoreTask().execute(this);
the Async task looks like this:
public class HighscoreTask extends AsyncTask<MainView, Void, Void> {
protected void onPreExecute() {
}
protected void onPostExecute(final Void unused) {
}
#Override
protected Void doInBackground(MainView... params) {
HighScoreFactory.syncScores();
return null;
}
}
All the database interaction happens in HighScoreFactory.syncScores() this can take as long as it needs because it happens in the background. In my case it sends an HTTP request to an external server and loads these into a database. It's never caused any problems and works seamlessly.

Why do you have a 2 second limit on your database write? If it is purely for the sake of UI responsiveness, then there is another approach you can take.
You don't actually have to perform the save within your onPause method itself, you could just kick off a new Thread that actually does the save for you.
private void backgroundSave(){
Thread backgroundThread = new Thread() {
#Override
public void run() {
//do save here
}
};
backgroundThread.start();
}
#Override
protected void onPause() {
super.onPause();
backgroundSave();
}
You could alternatively use an AsyncTask for this.
You might have to consider the case when a user attempts to restart your app before the save is complete, but that shouldn't be too hard to take into account.

Have you tried insert data to the database in transaction?
try{
db.beginTransaction();
//here insert data to database
db.setTransactionSuccessful();
} finally {
db.endTranscation();
}
That can speed up operation.

Create a new Thread that does the data writing using Context.openFileOutput(String name, int mode) with this as the context. You can then write it in the background with the new thread and retrieve it with: Context.openFileInput(String name) again with this as the context. Hopefully this helps.