How to detect android contact updated and sync to firestore - android

I am trying to get all the android contacts which been updated.
I am saving on firebase the last contact id i added and the last updated timestamp
I am using the next function to get back a cursor of all the updated contacts to compare with firebase server
private fun getUpdatedContacts(): Cursor? {
val projection = arrayOf(
ContactsContract.Contacts._ID,
ContactsContract.Contacts.DISPLAY_NAME,
ContactsContract.Contacts.HAS_PHONE_NUMBER,
ContactsContract.Contacts.CONTACT_LAST_UPDATED_TIMESTAMP)
val selection = ContactsContract.Contacts.CONTACT_LAST_UPDATED_TIMESTAMP + " > ? AND " +
ContactsContract.Contacts._ID + "<= ?"
val selectionArgs = arrayOf(mFireContactDetails!!.lcu_ms.toString(), mFireContactDetails!!.lcid.toString())
val sortOrder = ContactsContract.Contacts.CONTACT_LAST_UPDATED_TIMESTAMP + " ASC"
return mContentResolver.query(
ContactsContract.Contacts.CONTENT_URI,
projection,
selection,
selectionArgs,
sortOrder)
}
But when i change one contact in my phone this cursor is returned MANY un-related contacts that i never used and mark them as changed. Last time when i just added a phone number to an existing contact, i got back from this cursor more than 50 contacts as been updated.
What is going on Android?? I am trying to sync contacts for the past 3 months now. Why is it so hard???

This is almost the same question with the same answer as your other question: When deleting a contact on android, other random contacts id's being changed
You have some assumptions on Contact IDs that you can't make - no one guarantees Contact IDs are incremental, and no one guarantees Contact IDs are stable, in fact they are definitely not.
You can use queried contact IDs while you're app is running, there is very small chance of them being changed within some minutes, but there is some chance of having IDs changed for existing users every once in a while.
Not only that, but the same ID can point to some contact today, and point to a completely different contact tomorrow.
If you keep some clone of the local contacts in the cloud, you should use the following composite IDs to reference contacts:
Contacts.CONTACT_ID, Contacts.LOOKUP_KEY, Contacts.DISPLAY_NAME
See my answer here for more details: How to uniquely identify a contact on ContactsContract.Contacts table
It's not a perfect solution, but it's the best we have

I been testing this solution for couple of days and it seems OK but i think i need to test it much more. If you using this method, do your own testing and above all, PLEASE LET ME KNOW IF I MISSED ANYTHING and don't be hurry to downgrade. Thx!
I built an App class that extents Application and implements the
ActivityLifecycleCallbacks. In which i create a ContactSync class for the
first time and activate it everytime the app goes to foregound
In ContactSync class, i am using Kotlin withContext(Dispatchers.IO) to suspend any code for easier flow
I use .get() to get all the contacts from firestore related to current user
at the .get() addOnSuccessListener, i add all the contacts to a HashMap with the normalized phone number as key and name + firestore id as values (using internal class)
While making the HashMap i also make sure there are no duplicates on firestore with smae phone number and if so delete them (using batch)
i then retrieve all the contacts from android phone. I sort them by NORMALIZED_NUMBER first and DISPLAY_NAME (will explain later)
I am now creating a batchArray with index and count to avoid exceeding the 500 limit
I start scanning through the contacts cursor,
I first get the normalized number, if not available (null) i create it my self using a function i made (it might be that a null value is only returned for phone numbers not in correct format, not sure)
I then compare the normalized number with previous cursor value. If the same i ignore it to avoid duplicates in firestore (remember the cursor is sorted by NORMALIZED_NUMBER)
I then check if the normalized number already in HashMap.
If in HashMap: i compare the name in HashMap to the cursor name. if different, i conclude the name was changed and i update the firestore contact in batch array (remember to increment counter and if exceeds 500 increase index). I then remove the normalized number from the HashMap to avoid its deletion later
If not in HashMap: i conclude the contact is new and i add it to firestore via batch
I iterate through all the cursor until completed.
When cursor complete i close it
Any remaining records found in HashMap are ones that were not found on firestore hence deleted. I iterate and delete them using batch
sync is done on the phone side
Now, since making the actual sync needs access to all users, i user firebase functions in node. I create 2 functions:
function that fires when new user is created (signed via phone)
function that fires when new contact document is created.
Both functions compare the users to the normalized number in document and if matching, writing the uid of that user to the firestore document "friend_uid" field.
Note you might have errors in these functions if you try to use them in free firebase plan. I suggest changing to Blaze plan and limit the charging to couple of dollars. By changing to Blaze, google also gives you free extras and avoid actual payment
By that, the sync is completed. The sync takes only couple of seconds
To display all the contacts which are users to the app, query all user contacts with "friend_uid" that are not null.
Some extra notes:
The .get() will retrieve all the contacts every time a sync is made. That might be a lot of reads if user has couple of hundreds contacts. To minimize, i use .get(Source.DEFAULT) when launching the app and .get(Source.CACHE) for the other times. Since these documents name and number only modified by user, i believe it will not be a problem most of the times (still testing)
To minimize the sync process as much as possible, i initiate it only if any contact changed its timestamp. I save the last timestamp to SharedPreferences and compare it. I found it mostly saves sync when app re-opened fast.
I also save the last user logged in. If any change in user, i re-initialize the current user contacts
Some source code (still testing, please let me know if any error):
private fun getContacts(): Cursor? {
val projection = arrayOf(
ContactsContract.CommonDataKinds.Phone._ID,
ContactsContract.CommonDataKinds.Phone.NUMBER,
ContactsContract.CommonDataKinds.Phone.NORMALIZED_NUMBER,
ContactsContract.CommonDataKinds.Phone.DISPLAY_NAME,
ContactsContract.CommonDataKinds.Phone.CONTACT_LAST_UPDATED_TIMESTAMP)
//sort by NORMALIZED_NUMBER to detect duplicates and then by name to keep order and avoiding name change
val sortOrder = ContactsContract.CommonDataKinds.Phone.NORMALIZED_NUMBER + " ASC, " +
ContactsContract.CommonDataKinds.Phone.DISPLAY_NAME + " ASC"
return mContentResolver.query(
ContactsContract.CommonDataKinds.Phone.CONTENT_URI,
projection,
null,
null,
sortOrder)
}
private suspend fun syncContactsAsync() = withContext(Dispatchers.IO) {
if (isAnythingChanged() || mFirstRun) {
if (getValues() == Result.SUCCESS) {
myPrintln("values retrieved success")
} else {
myPrintln("values retrieved failed. Aborting.")
return#withContext
}
val cursor: Cursor? = getContacts()
if (cursor == null) {
myPrintln("cursor cannot be null")
mFireContactHashMap.clear()
return#withContext
}
if (cursor.count == 0) {
cursor.close()
mFireContactHashMap.clear()
myPrintln("cursor empty")
return#withContext
}
var contactName: String?
var internalContact: InternalContact?
val batchArray = mutableListOf(FirebaseFirestore.getInstance().batch())
var batchIndex = 0
var batchCount = 0
var normalizedNumber:String?
var prevNumber = ""
var firestoreId: String
while (cursor.moveToNext()) {
normalizedNumber = cursor.getString(COLUMN_UPDATED_NORMALIZED_NUMBER)
if (normalizedNumber == null) {
normalizedNumber = cursor.getString(COLUMN_UPDATED_PHONE_NUMBER)
normalizedNumber = Phone.getParsedPhoneNumber(mDeviceCountryIso,normalizedNumber,mContext)
}
//cursor sorted by normalized numbers so if same as previous, do not check
if (normalizedNumber != prevNumber) {
prevNumber = normalizedNumber
contactName = cursor.getString(COLUMN_UPDATED_DISPLAY_NAME)
internalContact = mFireContactHashMap[normalizedNumber]
//if phone number exists on firestore
if (internalContact != null) {
//if name changed, update in firestore
if (internalContact.name != contactName) {
myPrintln("updating $normalizedNumber from name: ${internalContact.name} to: $contactName")
batchArray[batchIndex].update(
mFireContactRef.document(internalContact.id),
FireContact.COLUMN_NAME,
contactName)
batchCount++
}
//remove to avoid deletions
mFireContactHashMap.remove(normalizedNumber)
} else {
//New item. Insert
if (normalizedNumber != mUserPhoneNumber) {
myPrintln("adding $normalizedNumber / $contactName")
firestoreId = mFireContactRef.document().id
batchArray[batchIndex].set(mFireContactRef.document(firestoreId),
FireContact(firestoreId, -1, contactName,
cursor.getString(COLUMN_UPDATED_PHONE_NUMBER),
normalizedNumber))
batchCount++
}
}
if (BATCH_HALF_MAX < batchCount ) {
batchArray += FirebaseFirestore.getInstance().batch()
batchCount = 0
batchIndex++
}
}
}
cursor.close()
//Remaining contacts not found on cursor so assumed deleted. Delete from firestore
mFireContactHashMap.forEach { (key, value) ->
myPrintln("deleting ${value.name} / $key")
batchArray[batchIndex].delete(mFireContactRef.document(value.id))
batchCount++
if (BATCH_HALF_MAX < batchCount ) {
batchArray += FirebaseFirestore.getInstance().batch()
batchCount = 0
batchIndex++
}
}
//execute all batches
if ((batchCount > 0) || (batchIndex > 0)) {
myPrintln("committing changes...")
batchArray.forEach { batch ->
batch.commit()
}
} else {
myPrintln("no records to commit")
}
myPrintln("end sync")
mFireContactHashMap.clear()
mPreferenceManager.edit().putLong(PREF_LAST_TIMESTAMP,mLastContactUpdated).apply()
mFirstRun = false
} else {
myPrintln("no change in contacts")
}
}
private suspend fun putAllUserContactsToHashMap() : Result {
var result = Result.FAILED
val batchArray = mutableListOf(FirebaseFirestore.getInstance().batch())
var batchIndex = 0
var batchCount = 0
mFireContactHashMap.clear()
var source = Source.CACHE
if (mFirstRun) {
source = Source.DEFAULT
myPrintln("get contacts via Source.DEFAULT")
} else {
myPrintln("get contacts via Source.CACHE")
}
mFireContactRef.whereEqualTo( FireContact.COLUMN_USER_ID,mUid ).get(source)
.addOnSuccessListener {documents ->
var fireContact : FireContact
for (doc in documents) {
fireContact = doc.toObject(FireContact::class.java)
if (!mFireContactHashMap.containsKey(fireContact.paPho)) {
mFireContactHashMap[fireContact.paPho] = InternalContact(fireContact.na, doc.id)
} else {
myPrintln("duplicate will be removed from firestore: ${fireContact.paPho} / ${fireContact.na} / ${doc.id}")
batchArray[batchIndex].delete(mFireContactRef.document(doc.id))
batchCount++
if (BATCH_HALF_MAX < batchCount) {
batchArray += FirebaseFirestore.getInstance().batch()
batchCount = 0
batchIndex++
}
}
}
result = Result.SUCCESS
}.addOnFailureListener { exception ->
myPrintln("Error getting documents: $exception")
}.await()
//execute all batches
if ((batchCount > 0) || (batchIndex > 0)) {
myPrintln("committing duplicate delete... ")
batchArray.forEach { batch ->
batch.commit()
}
} else {
myPrintln("no duplicates to delete")
}
return result
}

Related

Firestore query StartAt(DocumentReference) not giving proper result

In my firestore database,there are 12+ documents.I am getting the first 3 documents correctly by calling the below function on button click. But on the secondclick, though the documentReference is passed correctly, its not retrieving any data.The querySnapshot size is coming 0. What could be the problem.
Given below is the declaration
private val db: FirebaseFirestore = FirebaseFirestore.getInstance()
private val colRef: CollectionReference = db.collection("Notebook")
private var lastResult: DocumentReference? = null
private lateinit var query: Query
and below is the onButtonClick code :
private fun loadNoteNew() {
#Suppress("SENSELESS_COMPARISON", "LiftReturnOrAssignment")
if (lastResult == null) {
query = colRef.orderBy("priority")
.limit(3)
} else {
Log.i(TAG, "Start ${lastResult!!.id}")
query = colRef.orderBy("priority")
.startAfter(lastResult)
.limit(3)
}
Log.i(TAG, "before get")
query.get()
.addOnSuccessListener { querySnapshot ->
var data = ""
Log.i(TAG, "querySnapshot Size : ${querySnapshot.size()}")
if (lastResult != null) {
Log.i(TAG, "querySnapshot ID : ${lastResult!!.id}")
}
for (snapshot in querySnapshot) {
val note = snapshot.toObject(Note::class.java)
note.id = snapshot.id
val title = note.title
val desc = note.description
val priority = note.priority
data += "${note.id} \nTitle =$title \nDescription = $desc\nPriority : $priority\n\n"
}
if (querySnapshot.size() > 0) {
data += "---------------\n\n"
textView_loadData.append(data)
lastResult = querySnapshot.documents[querySnapshot.size() - 1].reference
Log.i(TAG, lastResult!!.id)
}
}
}
Given below is the logcat for first click
I/FireStoreExample: before get
I/FireStoreExample: querySnapshot Size : 3
I/FireStoreExample: P9hIw4Ai7w4IHP6H3ew3
and given below is the logcat of second click
I/FireStoreExample: Start P9hIw4Ai7w4IHP6H3ew3
I/FireStoreExample: before get
I/FireStoreExample: querySnapshot Size : 0
I/FireStoreExample: querySnapshot ID : P9hIw4Ai7w4IHP6H3ew3
Please help me find out,where i am getting it wrong.
Thanks
The second query result is empty because of a misunderstanding on the semantics of query pagination using startAt and startAfter methods.
Let's say the Notebook collection contains N documents. When you make the first query you're asking for the first 3 documents ordered by the priority field so the query is returning documents 1..3. Then upon the second click you're expecting the query to return the next 3 results so indeed you're expecting documents 4..6. The keypoint here is that both startAt and startAfter paginate based on the value of the ordered field rather than with the last document retrieved. Overall the semantics of startAt and startAfter are roughly as follows.
orderby(X).startAt(Y) => Return documents whose X field is greater than or equal Y
orderby(X).startAfter(Y) => Return documents whose X field is strictly greater than Y
With that in mind, let's examine what the code is actually doing when you make the second query:
// At the end of the first query...
lastResult = querySnapshot.documents[querySnapshot.size() - 1].reference
// Second query
query = colRef.orderBy("priority")
.startAfter(lastResult)
.limit(3)
In the code above you're asking for the documents whose "priority" field is greater than document reference "P9hIw4Ai7w4IHP6H3ew3" and indeed there are no documents greater than that, therefore the result set is empty. Here is api reference for both.
There is yet another thing to note. Because these methods filter upon the fields value the position of the cursor could be ambiguous. For instance, if you have 4 documents with priority 3 and already retrieved the leading three if you set startAfter(3) you'll be missing a document. Similarly, if startAt(3) were to be made you'll get back the same three documents. This is also pointed out in the documentation. All in all you have a couple of options to make this work as intended:
Add another orderby in another field so that documents are uniquely identified by the combination so to prevent any cursor ambiguity and be able to use startAfter with guarantees. Next snippet build upon the doc samples and your code.
// first query
query = colRef.orderBy("priority")
.orderBy("AnotherField")
.limit(3)
// Save last document
lastResult = querySnapshot.documents[querySnapshot.size() - 1]
// Second and next queries
query = colRef.orderBy("priority")
.orderBy("AnotherField")
.startAfter(lastResult)
.limit(3)
Lastly remember that it might be simpler to just query all the documents if they're not many and delay optimizations until they become a performance issue.

How to remove duplicate entries in Android Contacts App which gets added when adding new contacts programatically?

I have an app which inserts/updates a contact in the phone book using below code, the code works but the problem I am facing is if there is similar contact with the same phone number eg. a WhatsApp, dou, Viber etc. contact then those app contact takes over my contacts DisplayName and merges together but due to this, we are having multiple duplicate entries in some versions of android like (Samsung, LG, MI A5 stock android) etc.
But works on some phones like MI Max and a few others, does anyone have a solution to this problem or is there I am missing some fields that need to be present to avoid duplicate contacts.
private fun insertContact(contact: Contact): Boolean {
try {
val operations = ArrayList<ContentProviderOperation>()
ContentProviderOperation.newInsert(RawContacts.CONTENT_URI).apply {
withValue(RawContacts.ACCOUNT_NAME, "abcd#gmail.com")
withValue(RawContacts.ACCOUNT_TYPE, "google.com")
operations.add(build())
}
ContentProviderOperation.newInsert(Data.CONTENT_URI).apply {
withValueBackReference(Data.RAW_CONTACT_ID, 0)
withValue(Data.MIMETYPE, StructuredName.CONTENT_ITEM_TYPE)
withValue(StructuredName.GIVEN_NAME, contact.firstName)
withValue(StructuredName.FAMILY_NAME, contact.lastName)
withValue(StructuredName.SUFFIX, "AppName")
operations.add(build())
}
addUpdatePhone(operations, contact.phoneNumbers)
//similar function for other fields email, address, birthday, profilePic etc
val results = context.contentResolver.applyBatch(ContactsContract.AUTHORITY, operations)
return true
} catch (e: Exception) {
LOG.e( "Error inserting contact")
return false
}
}
private fun updateContact(contact: contact, rawContactId: String): Boolean {
try {
val operations = ArrayList<ContentProviderOperation>()
ContentProviderOperation.newUpdate(Data.CONTENT_URI).apply {
val selection = "${Data.RAW_CONTACT_ID} = ? AND ${Data.MIMETYPE} = ?"
val selectionArgs = arrayOf(rawContactId, StructuredName.CONTENT_ITEM_TYPE)
withSelection(selection, selectionArgs)
withValue(StructuredName.GIVEN_NAME, contact.firstName)
withValue(StructuredName.FAMILY_NAME, contact.lastName)
withValue(StructuredName.SUFFIX, "AppName")
operations.add(build())
}
addUpdatePhone(operations, contact.phoneNumbers, true, rawContactId)
//similar function for other fields email, address, birthday, profilePic etc
context.contentResolver.applyBatch(ContactsContract.AUTHORITY, operations)
return true
} catch (e: Exception) {
LOG.e("Error updating contact")
return false
}
}
private fun addUpdatePhone(operations: ArrayList<ContentProviderOperation>, phoneNumbers: List<PhoneNumber>, isUpdate: Boolean = false, rawContactId: String = "") {
if(isUpdate) {
//delete old data with the given raw_contact_id
ContentProviderOperation.newDelete(Data.CONTENT_URI).apply {
val selection = "${Data.RAW_CONTACT_ID} = ? AND ${Data.MIMETYPE} = ? "
val selectionArgs = arrayOf(rawContactId, Phone.CONTENT_ITEM_TYPE)
withSelection(selection, selectionArgs)
operations.add(build())
}
}
phoneNumbers.forEach {
//add new rows of phone number for the given raw_contact_id
ContentProviderOperation.newInsert(Data.CONTENT_URI).apply {
if(isUpdate) {
withValue(Data.RAW_CONTACT_ID, rawContactId)
} else {
withValueBackReference(Data.RAW_CONTACT_ID, 0)
}
withValue(Data.MIMETYPE, Phone.CONTENT_ITEM_TYPE)
withValue(Phone.TYPE, it.type)
withValue(Phone.LABEL, it.label)
withValue(Phone.NUMBER, it.value)
withValue(Phone.NORMALIZED_NUMBER, it.value.normalizeNumber())
operations.add(build())
}
}
}
//similar functions as above for other fields
private fun otherUpdateFunAsAbove() {}
After adding a new RawContact, Android needs to make a decision if this is a brand new Contact or if there's already an existing contact that represents the same person.
Android has an algorithm, which changes and evolved with different OS versions, and might have been tweaked by makers like Samsung, but usually it looks for a very similar name with some other item (such as phone or email) that is either identical or very close.
In that case it'll merge the two contacts using RawContact Aggregation.
Apps can control this process via AggregationExceptions in which an app can state "Keep these two RawContacts separate" or "Keep these two RawContacts merged" regardless of that algorithm.
So what you're explaining would be normal behavior, your original name should still had been kept within the RawContacts.
In any way, I would not recommend trying to find a RawContact (i.e. in your selection clause) using an ID + Name, instead you should use just the RawContact ID.
If you can't find that RawContact ID, this still doesn't mean you should create a new one, instead use the RawContact's lookupUri that you need to store in your app, and get a possibly new ID from that, in a way similar to the approach suggested for Contacts.

Android Kotlin Executing SQLite query in MainActivity

I'm trying to figure out how to execute a SQL query from my MainActivity. I'm currently using a large portion of this example. Upon clicking my Acknowledge button, it finds the selected rows, gets the jsonID of said row, then I'm wanting to update a value in my SQLite table of those rows.
So my question is, how do I execute my "jsonQuery" string, to my SQLite database upon clicking my Acknowledge button?
Here's the code I have in my MainActivity:
acknowledge.setOnClickListener {
var jsonData_text: String = ""
for (i in 0 until JSONParser.MrAdapter.public_modelArrayList!!.size) {
if (JSONParser.MrAdapter.public_modelArrayList!!.get(i).getSelecteds()) {
jsonData_text = jsonData_text + JSONParser.MrAdapter.public_modelArrayList!!.get(i).getJSONID() + ","
}
}
val jsonSQLQuery = jsonData_text.dropLast(1)
val jsonQuery = "update Notes set acknowledged = 1 WHERE jsonID in ("+jsonSQLQuery+")"
Log.d("logged_json", jsonQuery)
}
Thanks for any help!

How to check if record exists or not using Anko?

I am learning to fetching data from sqlite using anko. I can print the data successfully (if the record exist) but my application always crash when the data doesn't exist.
the error says:
parseSingle accepts only cursors with a single entry
I know exactly the meaning of error, I just dont know how to solve it.
here is the code for query:
fun getUserByUid(uid: Int): UserModel
{
val data = context.database.use {
val db = context.database.readableDatabase
val columns = UserModel.COLUMN_ID + "," + UserModel.COLUMN_NAME + "," + UserModel.COLUMN_API_KEY
val query = db.select(UserModel.TABLE_NAME, columns)
.whereArgs("(uid = {userId})",
"userId" to uid)
query.exec {
val rowParser = classParser<UserModel>()
parseSingle(rowParser) // this line that trigger error exception
}
}
return data
}
I tried to find count function in query or rowParser variable to check if the record exist or not but could not find it.
From the wiki page.
https://github.com/Kotlin/anko/wiki/Anko-SQLite#parsing-query-results
Parsing query results
So we have some Cursor, and how can we parse it into regular classes? Anko provides functions parseSingle, parseOpt and parseList to do it much more easily.
Method Description
parseSingle(rowParser): T Parse exactly one row
parseOpt(rowParser): T? Parse zero or one row
parseList(rowParser): List Parse zero or more rows
Note that parseSingle() and parseOpt() will throw an exception if the received Cursor contains more than one row.

Xamarin.Mobile Contacts API takes up a lot of time while fetching contacts?

I'm using Xamarin.Mobile Component for Android to fetch contacts using the code:
var book = new AddressBook (Activity) {PreferContactAggregation = true};
var contData = data.Data;
var cur = Activity.ManagedQuery (contData, null, null, null, null);
Contact myContact = null;
var lookupKeyList = new List<string> ();
while (cur.MoveToNext ()) {
lookupKeyList.Add (cur.GetString (cur.GetColumnIndexContactsContract.Contacts.InterfaceConsts.LookupKey)));
}
myContact = book.Where (c => c.Id == lookupKeyList [0]).First ();
This code is part of picking a contact from the phone book and receiving the data on OnActivityResult method.
Unfortunately, this code is taking up too much time on some devices and is instantaneous on others. I guess its related to Contact Aggregation but I am not sure. Any pointers?
Sounds like an Android issue, not necessarily a Xamarin issue. Take a look at the question Getting name and email from contact list is very slow. One of the things this answer does is use a Projection in the query to get all the columns in one go.

Categories

Resources