Sorry if this question is very basic but I started seeing no sql storages for an android app and I found App engine endpoints and servlets that as far as i understood, expose an api to do CRUD operations.
The thing that it is difficult to grasp for me is the format of the objects stored and it's efficiency. I read that by using libraries like objectify/gson the objects are stored in json.
Now, suppose I have a class Person that has an attribute Friends that is a list of Person. If two different persons share the same friend it will be stored in two different json objects, duplicating the information. Should I make a different class for the storage that keeps the id of the friends and then when loading a person find his/her friends from a hash map of Person? But that would imply requesting all the Persons to the Web service to construct that map even though I only want to find a Person without friends.
Another way would be to make the 'query' in the server side, return the friends objects of the person that is requested and put them in a hash map for future 'queries' of other Persons. In this way I would transfer less data each time but i would consume more times the webservice which can lead to exhaust the daily request limit quota.
Again sorry for the beginner question. I would appreciate any directions, patterns to solve this problem, in a nutshell how to efficiently -in the sense of space- store objects and efficiently retrieve them -in the sense of the amount of queries and data transfer from the web service- in a nosql database.
Indeed Google Cloud Endpoints will allow you to do CRUD operations through one or more API(s). But, as detailed in the documentation (https://cloud.google.com/appengine/docs/java/endpoints/) it allows you to do much more than that, e.g. "all of the services and features available in App Engine, such as Google Cloud Storage, Mail, Task Queues" etc.
You can use Objectify when the back end of your endpoints is Datastore. Objectify is the open-source API for Java which is recommended by Google. However, note that the data is not store as json but as Data objects that are called "Entities" which can have properties of different data types. See https://cloud.google.com/appengine/docs/java/datastore/entities for more info.
The approach to NoSQL database is very different from relational database when it comes to data modelling. You should not care about normalizing your data and storing the same data multiple times is quite a common approach.
In your case, if two persons share the same friend, you would save the friend information two times, in each person entity. In such a way, when you will query the list of friends for one person you just have to get the person entity through Objectify in your endpoint: it will include the list of friends and it will automatically be transformed to JSON when sent to the front-end.
I would suggest you try the Google examples (https://cloud.google.com/appengine/docs/java/endpoints/helloworld-java-maven) or even better follow the Udacity MOOC which will help you understanding the entire stack https://www.udacity.com/course/developing-scalable-apps-in-java--ud859
The tutorials from Romin Irani are also an excellent entry point to this technology https://rominirani.com/google-cloud-endpoints-tutorial-part-1-b571ad6c7cd2#.p4h8rmkt3 There are tutorials for Eclipse as well as Android Studio (I recommend using the second one).
Related
I'm developing an android application that will allow users to find informations about the streets in my city. They will be able to register and save some streets as "favourites".
My question is, how can I use Firebase to store the streets data considering that they won't change overtime?
The Firebase data model is not well suited to storing arrays (or Java List objects). See this blog post explaining the behavior you get. Instead of storing a List, follow the Firebase documentation's approach for storing collections. This will indeed store it as a map, which is the correct approach precisely .
That depends mostly on how you want to read the data.
If you always load all of the street data, then you might as well store it as a file on Firebase Hosting. That'll be a lot cheaper, and perform better (since the data is cached on a CDN).
If the app loads parts of the data, but it's not very dynamic, you could split the data into multiple files and still store those on Firebase Hosting.
If users sometimes update the data, but it's still hardly queries by the apps, consider storing it in Cloud Storage through the Firebase SDK. The files are available to all of your users that way.
If you want advanced querying of the data, consider storing it in the Firebase Realtime Database or in Cloud Firestore.
I need to do a application that incorporates a database and I'm thinking about using the Firebase Realtime database.
Basically the application is an information app. It has different categories, Shops, Restaurants, Attractions, ect. I've almost completed the Udacity course on firebase and It can do what I need, I'm just not sure it's the most efficient.
They way they explain the database is structured is having a key so my key would either be Shop, Restaurant or Attraction. Below the Shop key there would be "Shop1","Shop2","Shop3", ect. Now this is where my problem comes in, inside shop1 I'm planning to have the shop name, longitude, latitude, description, and other details about the shop. Each time I want to add a new shop I'm going to have to add a child of the shop then under the new child I'm going to have to manually type the key and value.
This will get very time consuming and I was wondering if this is the correct way of adding data to the database ?
Thanks
Is Firebase the best way to store data for an information app?
This question is primarily opinion based, but since Firebase gives you so many choices under the hood, it is becoming a pretty decent service. Being scalable and the quick response of the database is a great option.
Read more about a good Database Schema here and about populating the database, you can write cloud functions and that would be a pretty advanced level of implementation.
May be your question is "What is the best way to store data for an information app in Firebase?"
In your case, it will be difficult to save sequential key without keeping track the number of shop.
I would rather suggest using push(shopModel). If you're using push, the key will be automatically generated and it's guaranteed to be unique and order by time inserted.
I am developing a social networking android app. I am currently using parse as my back end. I am using Parse.com to store the text messages and fetch those messages.
The first problem with parse is that it runs very-2 slow.
The second problem is that I have to set limit to 1000 users. How can I access large number of users or data?
How can i fetch the results faster with large number of users?
Should i consider using any other backend like google app engine, etc.
I want to fetch and store results quickly, just like facebook and WhatsApp? I would really appreciate your ideas/feedback/suggestion. Thanks
The problem is not that parse is slow, but more likely that your data model is not optimized for use on a NoSQL database service like parse. You probably have SQL experience and created your data model like you would for a SQL database.
With NoSQL (and especially mobile apps), you need to model for queries; not normalisation and consistency.
When designing your backend, first create a list of the most commonly used queries your app will have. Then design your model around how to minimize the number of queries necessary for retrieving the dataset you need.
Why do you need to get 1000 or more users in one query?
I recommend you take a look at the data model behind the Anypic app on parse.com. The model is very simple, yet extremely versatile.
If you are a SQL-person (like I was), you need to "unlearn" your relation thinking and start thinking about query speed :-)
For a better answer you should probably paste some of your code to know how is the query you are making and how is your data model. That is the only way you can optimize your query because you cannot make Parse itself faster and you cannot remove its limits.
By the way, I work in Backbeam. It is a fast backend as a service and it has support for complex queries such as making joins. Maybe you want to give it a try.
I recently started using Kinvey as a backend for my Android app. The documentation doesn't have a lot of info about Collections. I want to know if it's possible to create Collections using the same concepts applied to MySQL tables for example:
A Collection called Users will hold a User ID, Username, User Email
And another Collection called Items corresponding to users -> Item ID, Item Name, User ID.
Has anyone successfully created Collections like this using Kinvey?
kinvey.com
I have also contacted their support team about this bu no reply yet.
I'm an engineer at Kinvey and can help you at this. Kinvey uses a NoSQL store on the back end, so the concepts are a little different than those of a relational database system like MySql, but in general the same thought process can apply. A Collection is similar to a table, although it is Schema-less. This means that attributes (columns in MySql terms) can be added dynamically as needed. You simply create the collection, and then start saving data objects to it. For more info on our Android library specifically, take a look at our Data Store User Guide.
I am developing an android app for building and sharing a database of bike trails.
Users will be able to add their own locations and trails to their local copy of the database, or edit existing descriptions, details, etc.
I would like some mechanism where all users of the app could share their data with one another. For instance, through a central web-based database or something.
It doesn't really work to just upload the entire database, because I am anticipating there will be times when several users will want to make edits at the same time, possibly to the same object.
Is there a defined "best practice" for accomplishing this kind of data-sharing?
You should create a server application which will handle all your client's data exchanges. Your server application have to be linked to your database. Also your client application(ak: Users) will communicate only with the server application which will refresh your other clients applications. Take a look at Java socket and remember to put them in a AsyncTask class (similar to Thread, but used only for android).
Take a look at Jackson: http://wiki.fasterxml.com/JacksonDataBinding
If you have a standard Java model inside your app, Jackson will help you convert it to JSON, which you can send easily to a central server.
When you pull down new routes from your central server, you'll simply use Jackson to deserialize the JSON back into your Java model.
I think the algorithm should be something like this:
Note: you need a good way of determining whether 2 locations are the same or not. Presumably a single location could have different names or spellings, and slightly different GPS coordinates. Also, multiple trails might start at the same GPS coordinates.
Iterate through the remote records, one by one
If the location doesn't exist locally
create it.
else if the record is identical
ignore it
else
if only one record changed
copy it to the opposite database
else
merge the data from the 2 records together somehow
Finally, you would need to upload any locally created records to the remote database.