My Realtime Database has achieved over 1GB data stored, so in order to trim it,save storage and optimize daily use once a month i run a routine to delete old and irrelevant data.
it goes like this:
App.getDatabaseInstance().getReference("store/orders/historic/").orderByChild("creationTs").limitToLast(500).endAt(System.currentTimeMillis() - (90L * ONE_DAY_MILLIS)).addListenerForSingleValueEvent(new ValueEventListener() {
#Override
public void onDataChange(#NonNull DataSnapshot dataSnapshot) {
if (dataSnapshot.hasChildren()) {
for (DataSnapshot historicDs : dataSnapshot.getChildren()) {
historicDs.getRef().removeValue();
}
cleanHistoricBranch();
} else
System.out.println("FINISHED!!!");
}
#Override
public void onCancelled(#NonNull DatabaseError databaseError) {
}
});
The query run over a few thousands nodes (NOT MILLIONS) in the database but it takes HOURS to complete. i guess the problem is that the data must be downloaded and deleted ONE BY ONE
i tried different approaches but didnt work well.
App.getDatabaseInstance().getReference("store/orders/historic/").orderByChild("creationTs").limitToLast(500).endAt(System.currentTimeMillis() - (90L * ONE_DAY_MILLIS)).addListenerForSingleValueEvent(new ValueEventListener() {
#Override
public void onDataChange(#NonNull DataSnapshot dataSnapshot) {
if (dataSnapshot.hasChildren()) {
dataSnapshot.getRef().removeValue(); //deletes the whole branch, even the nodes that doesnt match the query.
cleanHistoricBranch();
} else
System.out.println("FINISHED!!!");
}
#Override
public void onCancelled(#NonNull DatabaseError databaseError) {
}
});
so does anyone have any better approach to trim large number of nodes in the database hierarchy? each node has very few data, but i have around 20 to 50 thousands of nodes candidates to removal.
If the time is mostly spent reading the data, the common approaches are:
Run the process more frequently, so that you have to do less data each time.
Set up integrated backups of your database, and use that backup to determine the keys to delete offline. Then send the write operations to the online database.
Related
I have a firebase database consisting of a bunch of cases. I want to loop through all these cases and find the count of Male cases only, male is represented by "M".
Picture of my database.
How I am trying to query for this data:
databseCOVIDCases = FirebaseDatabase.getInstance().getReference();
databseCOVIDCases.addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(#NonNull DataSnapshot snapshot) {
for (DataSnapshot data : snapshot.getChildren()) {
if (data.child("Sex").getValue(String.class) == "M") {
numMaleCases++;
}
}
}
#Override
public void onCancelled(#NonNull DatabaseError error) { }
});
When I set the text of the text view it shows 0 and then crashes with an out of memory error.
Instead of looping through all cases and counting the ones where Sex is M, I'd recommend using a query to only read those nodes. That saves you (and your users) the bandwidth of loading all the nodes where Sex is not M.
In code that'd be:
Query query = databseCOVIDCases.orderByChild("Sex").equalTo("M");
query.addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(#NonNull DataSnapshot snapshot) {
Log.i("Cases", "M case count: "+snapshot.getChildrenCount());
}
#Override
public void onCancelled(#NonNull DatabaseError error) {
throw error.toException(); // Never ignore errors
}
});
While the above works, a few things to keep in mind:
Any code that needs the count has to be inside the onDataChange method or be called from there. Other code may not run when you expect it to run.
For more on this, see
getContactsFromFirebase() method return an empty list
and
Setting Singleton property value in Firebase Listener
Reading all these nodes to only show the count it wasteful. It is much more practical (though unusual if you come from a SQL background) to store the actual count you need in the database, and update it whenever you add/remove a M node. While your write operations becomes more complex when you do this, your reads become much simpler, cheaper, and more scalable.
For more on this, see How to get size of an element/list in Firebase without get it all? and Is the way the Firebase database quickstart handles counts secure?
I know there are lot similar questions but nothing works for me. I tried so many ways. Here are my code.
so Basically If the current user uid is equals to child of joined_users I want to show text "Joined"
Query ref = FirebaseDatabase.getInstance().getReference().child("joined_users").orderByChild("uid").equalTo(myUid);
ref.addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(#NonNull DataSnapshot dataSnapshot) {
if (dataSnapshot.exists()){
holder.tag_room.setText("Joined");
}
}
#Override
public void onCancelled(#NonNull DatabaseError databaseError) {
}
});
This code:FirebaseDatabase.getInstance().getReference().child("joined_users") looks at the node joind_users directly under the root of your database. That is not where the data in your screenshot is, which explains why the query finds no matching nodes.
If you want to read the data in your screenshot, you'll need to start with FirebaseDatabase.getInstance().getReference().child("/Rooms//51008/joined_users").
There is the performance issue when i request the Firebase again and again.grid view update very slowly. so what should i do in this case i am thinking to store the path of the posts in the database. is this good idea or to store in the internal storage.(Caching)
private void setuptempGrid(View view){
setupImageGrid(ProfileActivity.images,view);
FirebaseDatabase.getInstance().getReference()
.child(getString(R.string.db_user_posts))
.child(application.getUser().getUserId()).addListenerForSingleValueEvent(new ValueEventListener() {
#Override
public void onDataChange(#NonNull DataSnapshot dataSnapshot) {
for (DataSnapshot d:dataSnapshot.getChildren() ) {
FirebaseDatabase.getInstance().getReference()
.child(getString(R.string.db_posts)).child(getString(R.string.db_public))
.child(d.getKey()).addListenerForSingleValueEvent(new ValueEventListener() {
#Override
public void onDataChange(#NonNull DataSnapshot dataSnapshot) {
gridImageAdapter.add(dataSnapshot.getValue(Posts.class).getProfilePic());
}
#Override
public void onCancelled(#NonNull DatabaseError databaseError) {
}
});
}
}
#Override
public void onCancelled(#NonNull DatabaseError databaseError) {
}
});
}
I have this code this take to much time to get the images and every time it loads when my fragment is created.
Use
FirebaseDatabase.getInstance().setPersistenceEnabled(true);
Acc. to docs
By enabling persistence, any data that the Firebase Realtime Database client would sync while online persists to disk and is available offline, even when the user or operating system restarts the app. This means your app works as it would online by using the local data stored in the cache.
and also do -
DatabaseReference db = FirebaseDatabase.getInstance().getReference()
.child(getString(R.string.db_user_posts))
.child(application.getUser().getUserId());
db.keepSynced(true);
Use Guava caches as they are optimized and also easy to implement. And before hitting firebase check if you have the data required for the specified node or not.
In my Application I am using Firebase to retrieve the mobilnumbers of the Users. Therefore I use this code:
databaseUsers.orderByChild("uid").addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
users.clear();
for (DataSnapshot postSnapshot : dataSnapshot.getChildren()) {
User contactlists = postSnapshot.getValue(User.class);
users.add(contactlists);
}
ContactList contactAdapter = new ContactList(ContactListActivity.this, users);
listViewContacts.setAdapter(contactAdapter);
}
#Override
public void onCancelled(DatabaseError databaseError) {
}
});
Now I have the following question: If the number of users is high, is there a possibility to send only the mobilnumbers of your Phonebook maybe in a list? Otherwise I think the traffic to Firebase might be not so efficient?!
The other opportunity would be to send each number individually but this might be quite complex if the user has many contacts.
At the moment I get all numbers from the server but I need to filter for the right contacts AND I need to display the names of the contact.
What is the best solution to use Firebase as efficient as possible and also get the names of the contacts?
Thank you in advance!
You'll have to:
Loop through the local phone book to find the phone number of each contact.
Execute a query to Firebase for each number.
Add the resulting contact (if any) to the list/adapter and update the view.
So say you've done step 1 and have a list of phone numbers. You'd then loop through those and for each:
for (String phonenumber: phonenumbers) {
Query query = databaseUsers.orderByChild("phonenumber").equalTo(phonenumber);
query.addListenerForSingleValueEvent(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
User contactlists = postSnapshot.getValue(User.class);
users.add(contactlists);
adapter.notifyDataSetChanged();
}
#Override
public void onCancelled(DatabaseError databaseError) {
throw databaseError.toException(); // don't ignore erors
}
});
}
The call to notifyDataSetChanged() ensures that the adapter know that it needs to update the view.
While the code gets a bit convoluted, it is not as slow as you may initially fear, since Firebase pipelines the requests over a single connection. The performance will mostly depend on the number of users you have in the database, but up to a few hundreds of thousands this should be fine.
I have been looking for a way to get one child object's data in Android Firebase.
I have found things like Firebase retrieve child Android. All the solutions are suggesting using a "ChildEventListener", however I need to get this data at this moment, not when it is moved, deleted, updated, etcetera.
My data is kept in https://.firebaseio.com/users//creation as a string. I figure there must be some simple way to access that without needing to do too much, because if I copy the exact URL to my browser, I can see the: 'creation: "2015/05/31 21:33:55"' right there in my "Firebase Forge Dashboard".
How can I access this without a listener?
Firebase listeners fire for both the initial data and any changes.
If you're looking to synchronize the data in a collection, use ChildEventListener. If you're looking to synchronize a single object, use ValueEventListener. Note that in both cases you're not "getting" the data. You're synchronizing it, which means that the callback may be invoked multiple times: for the initial data and whenever the data gets updated.
This is covered in Firebase's quickstart guide for Android. The relevant code and quote:
FirebaseRef.child("message").addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot snapshot) {
System.out.println(snapshot.getValue()); //prints "Do you have data? You'll love Firebase."
}
#Override
public void onCancelled(DatabaseError databaseError) {
}
});
In the example above, the value event will fire once for the initial state of the data, and then again every time the value of that data changes.
Please spend a few moments to go through that quick start. It shouldn't take more than 15 minutes and it will save you from a lot of head scratching and questions. The Firebase Android Guide is probably a good next destination, for this question specifically: https://firebase.google.com/docs/database/android/read-and-write
You don't directly read a value. You can set it with .setValue(), but there is no .getValue() on the reference object.
You have to use a listener. If you just want to read the value once, you use ref.addListenerForSingleValueEvent().
Example:
Firebase ref = new Firebase("YOUR-URL-HERE/PATH/TO/YOUR/STUFF");
ref.addListenerForSingleValueEvent(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
String value = (String) dataSnapshot.getValue();
// do your stuff here with value
}
#Override
public void onCancelled(FirebaseError firebaseError) {
}
});
Source: https://www.firebase.com/docs/android/guide/retrieving-data.html#section-reading-once
just fetch specific node data and its working perfect for me
mFirebaseInstance.getReference("yourNodeName").getRef().addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
for (DataSnapshot postSnapshot : dataSnapshot.getChildren()) {
Log.e(TAG, "======="+postSnapshot.child("email").getValue());
Log.e(TAG, "======="+postSnapshot.child("name").getValue());
}
}
#Override
public void onCancelled(DatabaseError error) {
// Failed to read value
Log.e(TAG, "Failed to read app title value.", error.toException());
}
});
I store my data this way:
accountsTable ->
key1 -> account1
key2 -> account2
in order to get object data:
accountsDb = mDatabase.child("accountsTable");
accountsDb.child("some key").addListenerForSingleValueEvent(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot snapshot) {
try{
Account account = snapshot.getChildren().iterator().next()
.getValue(Account.class);
} catch (Throwable e) {
MyLogger.error(this, "onCreate eror", e);
}
}
#Override public void onCancelled(DatabaseError error) { }
});