I have a simple view with mapper that emits docs with some keys.
com.couchbase.lite.View view = database.getView(VIEW_NAME);
if (view.getMap() == null) {
Mapper map = new Mapper() {
#Override
public void map(Map<String, Object> document, Emitter emitter) {
if ("user".equals(document.get("type"))) {
emitter.emit(document.get("name"), document);
}
}
};
view.setMap(map, null);
}
having this View, i can than create Queries on it, with certain parameters like setKeys, startKey, endKey, setDescending, setDescending, setSkip and others as described in couchbase manual.
If i write
Query query = view.createQuery();
List<Object> keys = new ArrayList<>();
keys.add("User Name");
query.setKeys(keys);
that query would return all docs matching "User Name" key.
But I couldn't find a simple way to write queries that excludes (omits) docs with certain keys (like opposite to setKeys() function)
one hack was found in ToDoLite Example
The code looks like this:
public static Query getQuery(Database database, final String ignoreUserId) {
com.couchbase.lite.View view = database.getView(VIEW_NAME);
if (view.getMap() == null) {
Mapper map = new Mapper() {
#Override
public void map(Map<String, Object> document, Emitter emitter) {
if ("user".equals(document.get("type"))) {
if (ignoreUserId == null ||
(ignoreUserId != null &&
!ignoreUserId.equals(document.get("user_id")))) {
emitter.emit(document.get("name"), document);
}
}
}
};
view.setMap(map, null);
}
Query query = view.createQuery();
return query;
}
Note that the view will only exclude key ignoreUserId you passed to it during the first call, and will ignore all the others during next calls (because it will create view only once during the first call)
So you need to create new view for every key they want to omit.
But if you have a lot of keys you want to exclude or does it often, it would be inefficient and boilerplate.
Do you know any better solution or hack ?
Any help appreciated
Thanks in advance
CouchBase is not designed for the types of queries you want to issue here. It is designed to use map/reduce under the hood to identify specific documents with a range (down to a range of 1 document), not exclude specific documents. What you are asking is not going to be efficient in CouchBase. You are using the wrong technology if your goal is to do these kinds of queries efficiently.
If your hands are tied and you are locked into CouchBase Lite, though, you have to work within the types of queries you have. Excluding a particular value can be restated as including all other values. If you want to do that in CouchBase, you use range queries with ranges designed to not include the value you want to exclude.
Here is a brief conceptual example. Suppose your view has documents with keys of "A", "B", "C", "D", "F", and "X" and you want to issue a query whose result excludes document "D". You could get the result you desired by first issue a range query for "A"-"C" and then issuing a second range query for "E"-"Z". The two results combined would be everything excluding "D".
Of course, those are simple keys. The more complex your keys, the more complex your range endpoints become to exclude specific values. The more keys you want to exclude, the more queries you have to perform (you have to perform N + 1 queries for N excluded terms). You are probably going to have a more efficient system by querying for all values in the view and filtering them in code yourself.
Related
I have been able to achieve a basic search capability in Android Room + FTS with the following query as an example in my Dao:
#Query("SELECT * FROM Conversation JOIN ConversationFts ON Conversation.id == ConversationFts.id WHERE ConversationFts.title LIKE :text GROUP BY Conversation.id")
public abstract DataSource.Factory<Integer, Conversation> search(String text);
Where the text is passed along between percentage characters, as such %lorem%.
This example works perfectly for the search of a single word and I want to expand this to be able to search for one or more words with the condition that they do not need to be in the order they are entered, but they must have a match for all the words. This means this has to be an AND and not OR situation.
I've found a few examples of SQL queries, but they require a specific query tailored for each case, for example: LIKE text AND LIKE another AND LIKE, etc... which is not a solution.
How can this be achieved? And to make it clear, I am not looking for a raw query solution, I want to stick to Room as much as possible, otherwise, I'd rather just use this as it is than to resort to raw queries.
EDIT: Adding an example per request
I search for do the results returned include all matches that contain do in the title, even if it is a partial match
I search for do and test the results returned include all matches that contain do and test, but in no specific order, even if they are partial matches.
However, if just one of them cannot be found in the text then it will not be returned in the results. For example, if do is found, but test is not then it will not be part of the results.
I think there is no way to do that except creating a raw query dynamically. You can write another method, something like this:
public abstract class ConversationDao {
public DataSource.Factory<Integer, Conversation> search(String text) {
StringBuilder builder = new StringBuilder();
String[] words = text.split("\\s+");
if (words.length > 0) {
builder.append(" WHERE ");
}
for (int i = 0; i < words.length; i++) {
builder.append("ConversationFts.title LIKE %").append(words[i]).append("%");
if (i < words.length - 1) {
builder.append(" AND ");
}
}
SupportSQLiteQuery query = new SimpleSQLiteQuery(
"SELECT * FROM Conversation JOIN ConversationFts ON Conversation.id == ConversationFts.id"
+ builder.toString()
+ " GROUP BY Conversation.id"
);
return search(query);
}
#RawQuery
public abstract DataSource.Factory<Integer, Conversation> search(SupportSQLiteQuery query);
}
In my app, I have a multiple-choice dialog with various filter options that the user should be able to choose in order to filter the database based on the rarity field of the documents. Since there are many options in the filter dialog, covering each case by hand would take ages if we take into account all the possible combinations of the filters. With that in mind, I tried creating a starting query as you can see below and then I iterate through the list of filters selected by the user and try to add a whereEqualTo("rarity",filter) operation to the query for each filter. I noticed that you can't concatenate queries like with normal variables e.g. var i += 5 so i would like to know if there is any solution to this kind of issue. Can you actually apply multiple whereEqualTo operations in the same query in steps/pieces without overriding the previously applied operations on that same query?
Here's what I've tried after receiving the filters selected by the user in my FilterActivity.kt class:
class FilterActivity : AppCompatActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_filter)
val db = FirebaseFirestore.getInstance()
val filters:ArrayList<String>? = intent.getStringArrayListExtra("filterOptions")
Log.d("FilterActivity", "filter options $filters")
var query = db.collection("Cards").orderBy("resID")
for(filter in filters!!) {
query = query.whereEqualTo("rarity",filter)
}
query.get().addOnSuccessListener { querySnapshot ->
if(querySnapshot.isEmpty) Log.d("FilterActivity","is empty")
for(doc in querySnapshot.documents) {
Log.d("FilterActivity", "${doc.getString("name")} - ${doc.getString("rarity")}")
}
}
}
}
Basically you are trying a do OR operation, where you are retrieving all documents, in which rarity fields matched any of the value in array.
You are try new firebase whereIn operation where you can pass array of values, but theres a limitation of max 10 values in filter
FirebaseFirestore.getInstance()
.collection("Cards")
.orderBy("resID")
.whereIn("rarity",filters!!.toList())
.get().addOnSuccessListener { querySnapshot ->
if (querySnapshot.isEmpty) Log.d("FilterActivity", "is empty")
for (doc in querySnapshot.documents) {
Log.d("FilterActivity", "${doc.getString("name")} - ${doc.getString("rarity")}")
}
}
filters arraylist can contain max 10 values
Can you chain multiple whereEqualTo operations in one query in pieces in Firestore
You can chain as may whereEqualTo operations as you need.
The problem in your code += operator. There is no way you can make an addition/concatenation of two Query objects. To solve this, please change the following line of code:
query += query.whereEqualTo("rarity",filter)
to
query = query.whereEqualTo("rarity",filter)
Good day all, I have a list of Objects (Let's call them ContactObject for simplicity). This object contains 2 Strings, Name and Email.
This list of objects will number somewhere around 2000 in size. The goal here is to filter that list as the user types letters and display it on the screen (IE in a recyclerview) if they match. Ideally, It would filter where the objects with a not-null name would be above an object with a null name.
As of right now, the steps I am taking are:
1) Create 2 lists to start and get the String the user is typing to compare to
List<ContactObject> nameContactList = new ArrayList<>();
List<ContactObject> emailContactList = new ArrayList<>();
String compareTo; //Passed in as an argument
2) Loop though the master list of ContactObjects via an enhanced for loop
3) Get the name and email Strings
String name = contactObject.getName();
String email = contactObject.getEmail();
4) If the name matches, add it to the list. Intentionally skip this loop if the name is not null and it gets added to the list to prevent doubling.
if(name != null){
if(name.toLowerCase().contains(compareTo)){
nameContactList.add(contactObject);
continue;
}
}
if(email != null){
if(email.toLowerCase().contains(compareTo)){
emailContactList.add(contactObject);
}
}
5) Outside of the for loop now as the object lists are build, use a comparator to sort the ones with names (I do not care about sorting the ones with emails at the moment)
Collections.sort(nameContactList, new Comparator<ContactObject>() {
public int compare(ContactObject v1, ContactObject v2) {
String fName1, fName2;
try {
fName1 = v1.getName();
fName2 = v2.getName();
return fName1.compareTo(fName2);
} catch (Exception e) {
return -1;
}
}
});
6) Loop through the built lists (one sorted) and then add them to the master list that will be used to set into the adapter for the recyclerview:
for(ContactObject contactObject: nameContactList){
masterList.add(contactObject);
}
for(ContactObject contactObject: emailContactList){
masterList.add(contactObject);
}
7) And then we are all done.
Herein lies the problem, this code works just fine, but it is quite slow. When I am filtering through the list of 2000 in size, it can take 1-3 seconds each time the user types a letter.
My goal here is to emulate apps that allow you to search the contact list of the phone, but seem to always to it quicker than I am able to replicate.
Does anyone have any recommendations as to how I can speed this process up at all?
Is there some hidden Android secret I don't know of that only allows you to query a small section of the contacts in quicker succession?
I currently have a statement which reads
if(Arrays.asList(results).contains("Word"));
and I want to add at least several more terms to the .contains parameter however I am under the impression that it is bad programming practice to have a large number of terms on one line..
My question is, is there a more suitable way to store all the values I want to have in the .contains parameters?
Thanks
You can use intersection of two lists:
String[] terms = {"Word", "Foo", "Bar"};
List<String> resultList = Arrays.asList(results);
resultList.retainAll(Arrays.asList(terms))
if(resultList.size() > 0)
{
/// Do something
}
To improve performance though, it's better to use the intersection of two HashSets:
String[] terms = {"Word", "Foo", "Bar"};
Set<String> termSet = new HashSet<String>(Arrays.asList(terms));
Set<String> resultsSet = new HashSet<String>(Arrays.asList(results));
resultsSet.retainAll(termSet);
if(resultsSet.size() > 0)
{
/// Do something
}
As a side note, the above code checks whether ANY of the terms appear in results. To check that ALL the terms appear in results, you simply make sure the intersection is the same size as your term list:
resultsSet.retainAll(termSet);
if(resultSet.size() == termSet.size())
You can utilize Android's java.util.Collections
class to help you with this. In particular, disjoint will be useful:
Returns whether the specified collections have no elements in common.
Here's a code sample that should get you started.
In your Activity or wherever you are checking to see if your results contain a word that you are looking for:
String[] results = {"dog", "cat"};
String[] wordsWeAreLookingFor = {"foo", "dog"};
boolean foundWordInResults = this.checkIfArrayContainsAnyStringsInAnotherArray(results, wordsWeAreLookingFor);
Log.d("MyActivity", "foundWordInResults:" + foundWordInResults);
Also in your the same class, or perhaps a utility class:
private boolean checkIfArrayContainsAnyStringsInAnotherArray(String[] results, String[] wordsWeAreLookingFor) {
List<String> resultsList = Arrays.asList(results);
List<String> wordsWeAreLookingForList = Arrays.asList(wordsWeAreLookingFor);
return !Collections.disjoint(resultsList, wordsWeAreLookingForList);
}
Note that this particular code sample will have contain true in foundWordInResults since "dog" is in both results and wordsWeAreLookingFor.
Why don't you just store your results in a HashSet? With a HashSet, you can benefit from hashing of the keys, and it will make your assertion much faster.
Arrays.asList(results).contains("Word") creates a temporary List object each time just to do linear search, it is not efficient use of memory and it's slow.
There's HashSet.containsAll(Collection collection) method you can use to do what you want, but again, it's not efficient use of memory if you want to create a temporary List of the parameters just to do an assertion.
I suggest the following:
HashSet hashSet = ....
public assertSomething(String[] params) {
for(String s : params) {
if(hashSet.contains(s)) {
// do something
break;
}
}
}
I try to make a dictionary using Quick Search Box in Android. As shown in the SearchableDictionary tutorial, it loads all (999 definitions)data and uses them as matches to the input text to get the search suggestion. in my case, I have 26963 rows of data that need to be suggest while user input a word on QSB. therefore, I want to grab the char data one by one from the QSB, so that it will be efficiently load necessary suggestion. how can i do this?
here's the code i use...
bringit(200);
if (Intent.ACTION_VIEW.equals(intent.getAction())) {
// from click on search results
//Dictionary.getInstance().ensureLoaded(getResources());
String word = intent.getDataString();
//if(word.length() > 3){bringit(10);}
Dictionary.Word theWord = Dictionary.getMatches(word).get(0);
launchWord(theWord);
finish();
} else if (Intent.ACTION_SEARCH.equals(intent.getAction())) {
String query = intent.getStringExtra(SearchManager.QUERY);
//SearchManager.
//String bb =
mTextView.setText(getString(R.string.search_results, query));
WordAdapter wordAdapter = new WordAdapter(Dictionary.getMatches(query));
//letsCount(query);
mList.setAdapter(wordAdapter);
mList.setOnItemClickListener(wordAdapter);
}
Log.d("dict", intent.toString());
if (intent.getExtras() != null) {
Log.d("dict", intent.getExtras().keySet().toString());
}
}
private void letsCount(String query) {
// TODO Auto-generated method stub
for(int i=0; i<query.length(); i++){
definite[i] = query.charAt(i);
}
}
public void bringit(int sum) {
// TODO Auto-generated method stub
String[] ss = new String[10];
Log.d("dict", "loading words");
for(int i=1; i<=sum; i++){
KamusDbAdapter a = new KamusDbAdapter(getApplicationContext());
a.open();
Cursor x = a.quick(String.valueOf(i));startManagingCursor(x);
if(x.moveToFirst()){
ss[0] = x.getString(1);
ss[1] = x.getString(2);
}
Dictionary.addWord(ss[0].trim(), ss[1].trim());
Log.v("Debug",ss[0]+" "+ss[1]);
//onStop();
}
}
I use SQLite to collect data. and the other code is just same as the tutorial...
Retrieving a cursor is generally slow. You only want to retrieve one cursor which contains all the matching results.
You should perform the searching using SQL rather than fetching everything. A FULL_TEXT search is usually fastest for text matching, it is however slightly more complicated to implement than a simple LIKE, but I highly recommend you give it a try.
So you want to execute an SQL statement like:
SELECT * FROM my_table WHERE subject_column MATCH 'something'
See SQLite FTS Extension for more information. You can also use wild-cards to match part of a word.
In terms of search suggestions there is really no point returning more than around ~100 results since generally no users ever bother to scroll down that far, so you can further speed things up by adding a LIMIT 0, 100 to the end of your SQL statement.
If possible only start getting cursors once the user has entered more than X number of characters (usually 3 but in you're case this may not be appropriate). That way you're not performing searches that could potentially match thousands of items.
You seem to be leaving lots of cursors open until the application closes them even though you don't actually need them anymore: instead of calling startManagingCursor just make sure to call x.close() after your if (x.moveToFirst()) { ... } - this will free up memory faster.
On an unrelated note: please don't name your variables and methods things like ss or bringIt() as it makes code hard to read -- what is ss and what does bringIt() bring exactly?
You could have a look at the full text search extension in SQL Lite. Idea is to have a SQL query that fetches only the matching results, not all the results and then filter.
There is also a sample for the Android SDK: com/example/android/searchabledict/DictionaryDatabase