In my chat application, I store the participants of a chat as their UIDs in a Map so I can so I can do queries like this:
.whereEqualTo("participantUIDs.$currentUserUid", true)
.whereEqualTo("participantUIDs.$partnerUid", true)
The problem is when I try to use this with orderBy
.whereEqualTo("participantUIDs.$currentUserUid", true)
.orderBy("lastMessageSentTimestamp")
I have to create a custom index. But this index will contain that specific user UID and I can't create an index for every user in my app. How can I circumvent this problem?
You can order the documents on the client after an unordered query. This should not be very taxing on the client app when the number of documents is less than 10,000.
Regarding:
I can't create an index for every user in my app.
That's definitely not an option, as there are some limitations when it comes to Cloud Firestore indexes:
https://firebase.google.com/docs/firestore/quotas#indexes
However, even if you manage to stay below these limits, that's not an option to manually create an index for each and every user that joins your app.
In my opinion, for your particular use-case, you should consider augmenting your data structure to allow a reverse lookup. Meaning that you should create a participantUIDs collection where you should keep the lists for each user. This technique is called denormalization and is a common practice when it comes to NoSQL databases like Cloud Firestore or Firebase Realtime Database.
But remember, there is "no perfect database structure":
What is the correct way to structure this kind of data in Firestore?
It's a little old, but I think this video might also help:
https://www.youtube.com/watch?v=u3KwKQddPoo
More info regarding why you need an index:
Why does this firestore query require an index?
P.S. You can also rely on Firebase Realtime Database when Cloud Firestore may become a little expensive. Both work really well together.
Info:
Array or Subcollection for storing events user uploaded
I am using firebase Cloud Firestore for my Android and iOS apps. I am stuck with how to structure my database.
Basically I have an "organization" collection that contains many users, and for every user, I want to save attendance time and leave time for every day.
The problem is I want to generate reports that allow me to get every day-attendance for each user, single-day attendance for all users, and all days attendance for all users.
so I tried this: inside the user I would have an "attendance" collection then each document is Unix timestamp of that day (to make sure that it's unique). then save fields like attend_time and leave_time...etc. the path is like that "/organization/android/users/3PRs42gRFzZQhLKcUhpf4wPMRV43/attendance/1590271200"
then I needed to get attendance for a single day for all users, so I did this: now I have another path "/organization/android/attendance" and inside the attendance, I store the Unix timestamp of the day, then the user ID then his attendance. and now I am saving attendance twice.
but I still can't get attendance for all days for all users!
this would be easy in Relational Database like SQL. any idea how to do it in firebase?
If you want to track attendance across all users, you're looking for a collection group query. This allows you to query documents from all collections named attendance.
Since a query in Firestore can only see data in the collection(s) it queries, you may have to duplicate some data from parent collections (your users and organizations) into each attendance document to allow the query. This type of data duplication is quite normal in NoSQL databases too.
Finally: if you need to perform many ad-hoc queries, you might want to consider using a database for those that is more suited to that use-case. For example, it is quite common to use Firestore to handle the direct-to-client interactions that require scaling to massive number of users, but then use BigQuery for the ad-hoc querying of that data. There is even a Firebase Extension that automatically exports to BigQuery to make this easier.
I have created a SQLite database in my android app. I want the data of every user in a centralized database. How can i setup the server and the sychronization that will allow me to get the data from every user?
What are the steps that need to be followed ?
Data synchronization is very complicated process. In document above there are some sample patterns which you may use.
http://www.dre.vanderbilt.edu/~schmidt/PDF/PatternPaperv11.pdf
You need to know what kind of troubles you can meet and include solutions for them in your project.
E.g.
User1 and User2 have synchronized data. User1 is modyfying one row in table Data. In the same time User2 has deleted this row. Your job is to handle this case.
I guess replication is what you are looking for.
Replication is a set of technologies for copying and distributing data and database objects from one database to another and then synchronizing between databases to maintain consistency. Using replication, you can distribute data to different locations and to remote or mobile users over local and wide area networks, dial-up connections, wireless connections, and the Internet.
http://msdn.microsoft.com/tr-tr/library/ms151198.aspx
I have an Android application which stores user's data in a SQLite database (containing 4 tables). Each table has an Integer Primary Key (Auto-incremented).
To get an idea of the scale:
3 of those tables have less than 7 fields and aren't expected to have more than 10 entries.
1 table having 8 fields, gets roughly 5 entries per day on an average.
What I want is the database to be synced to cloud - for many reasons, including
The data to be in sync across all of user's devices
To collect anonymous data, for research purposes, if the user has opted in.
I realize I would require a multi-tenant database. I want to know an efficient way to do this.
If I use a MySQL database on a web server, would this be efficient? How do I approach multi-tenancy in this case?
Or, do I need to use a cloud service like Google App Engine? (Which is completely new to me)
Yes, you should consider using GAE and its datastore. It also supports SQL, but I think your application is well suited to the datastore.
Your datastore would have 5 'kinds' (these are like tables) - the four you listed above and a User kind. Your four kinds would have a key path that would group them beneath their respective user. This would take care of the multi-tenancy and it would be very efficient: retrieving and writing these groups (defined by the key path hierarchy) is efficient and cheap.
For a quick intro to these concepts of the datastore I recommend this page of the objectify docs.
GAE supports Java, so not only can you use the same language on client and server, but you can also share code. It requires a bit of care when organizing your code (and watch out for logging) but this has considerable benefits particularly if Android is your only client.
I know SQLite Data Base is used in mobile devices (Android, iPhone) and it is light, takes only Kb space. Is there any limitation in SQLite? I want to know how they are different.
Every SQL database uses its own implementation of the language that varies slightly. While basic queries are almost universal, there are notable nuances between MySQL, PostgreSQL, Microsoft SQL Server, Oracle Database, etc.
What's particularly notable about SQLite is that unlike all the others mentioned above, this database software doesn't come with a daemon that queries are passed through. This means that if multiple processes are using the database at once, they will be directly altering the data through the SQLite library and making the read / write data calls to the OS themselves. It also means that the locking mechanisms don't deal with contention very well.
This isn't a problem for most applications that where one would think of using SQLite -- the small overhead benefits and easy data retrieval are worth it. However, if you'll be accessing your database with more than one process or don't consider mapping all your requests through one thread, it could be slightly troublesome.
Sqlite is very light version of SQL supporting many features of SQL. Basically is been developed for small devices like mobile phones, tablets etc.
SQLite is a third party ,open-sourced and in-process database engine. SQL Server Compact is from Microsoft, and is a stripped-down version of SQL Server.They are two competing database engines.
SQL is query language. Sqlite is embeddable relational database management system.
Edit : ( Source from following comment on my answer )
Sqlite also doesn't require a special database server or anything. It's just a direct filesystem engine that uses SQL syntax. ( By : Adam Plocher )
Techinically, SQLite is not open-source software but rather public domain. There is no license. ( By : Larry Lustig )
SQL is a query language. SQLite is an embeddable relational database management system.
Unlike other databases (like SQL Server and MySQL) SQLite does not support stored procedures.
SQLite is file-based, unlike other databases, like SQL Server and MySQL which are server-based.
What is SQLite?
SQLite is an open-source, zero-configuration, self-contained, stand-alone, transaction relational database engine designed to be embedded into an application.
Python SQLite can be defined as a C Library developed using ANSI-C, light-weight disc based database; doesn't demand for an extra or any other separate server process.
What are some SQLite characteristic?
SQLite does NOT require a server to run (RDBMS such as MySQL, PostgreSQL, etc., requires a separate server process to operate).
Is self-contained, it requires minimal support from the operating system or external library. This makes SQLite usable in any environment especially in embedded devices like iPhones, Android phones, game consoles, handheld media players.
Does not use any configuration files.
Is ACID-compliant. It means all queries and changes are Atomic, Consistent, Isolated, and Durable, all changes within a transaction take place completely or not at all even when an unexpected situation like application crash, power failure, or operating system crash occurs.
Capable of creating in-memory databases that are very fast to work with.
Uses dynamic types for tables. It means you can store any value in any column, regardless of the data type.
Allows a single database connection to access multiple database files simultaneously.
SQLite & ACID
SQLite guarantees all the transactions are ACID compliant even if the transaction is interrupted by a program crash, operation system dump, or power failure to the computer.
Atomic: a transaction should be atomic. It means that a change cannot be broken down into smaller ones. When you commit a transaction, either the entire transaction is applied or not.
Consistent: a transaction must ensure to change the database from one valid state to another. When a transaction starts and executes a statement to modify data, the database becomes inconsistent. However, when the transaction is committed or rolled back, it is important that the transaction must keep the database consistent.
Isolation: a pending transaction performed by a session must be isolated from other sessions. When a session starts a transaction and executes the INSERT or UPDATE statement to change the data, these changes are only visible to the current session, not others. On the other hand, the changes committed by other sessions after the transaction started should not be visible to the current session.
Durable: if a transaction is successfully committed, the changes must be permanent in the database regardless of the condition such as power failure or program crash. On the contrary, if the program crashes before the transaction is committed, the change should not persist.
Credit Sqlitetutorial.net
The most basic difference between SQLite and SQL is :
SQL is a query language which is used by different SQL databases. It is not a database itself.
SQLite is a database management system itself which uses SQL.
SQL is a database querying language and SQLite is a database (RDBMS) which uses SQL specifications. SQLite can be said as competitor to Microsoft's SQL Server.
The name itself suggests that it is the light version of SQL RDBMS. It is used in most of the small and portable devices like Android and iOS devices.
SQLite: Database Management System (DBMS).
SQL: Structured Query Language is a computer language, used to Create, edit and get data from DBMS via queries.