I plan to write an Android app that allows the users to share little pieces of information about a place, e.g., a bar, restaurant or the like. I'm now looking into what is the best way to store such data. I'm particularly concerned about retrieval: When showing a (Google) Map, I'd like to overlay it with said information, so I'll need to be able to make geospatial queries to the database.
Is this possible with the Google Cloud Datastore, for example? What are the alternatives?
Datastore does not have support for geospatial properties per se, but you can easily create a convenience class to work with it. In App Engine, there is a property GeoPt just for this purpose both in the Java and Python libraries.
If you are interested in geo queries there is not much that Datastore can do for you, but there is an interesting lib in Python to deal with that, geomodel. The problems are still on performance and costs. The operations performed there, although optimized, are heavy.
There's one last alternative. Search API allows you to perform geo queries. The only drawback of this are costs and limited free tier.
You can try a quadkey. Its similar to a geohash or a geocell. Translate the points to a binary and interleave it and treat it as base-4 number. It comes it looks like a space filling curve and it has the same properties. Here is a good article about quadkeys:https://msdn.microsoft.com/en-us/library/bb259689.aspx.
You might want to look at Postgresql and even PostGIS for storing geo data. They are fairly efficient, scalable and fast. I've used Postgresql with geodata for many years and on very busy websites and Android apps. If all you need is to locate things near other things, Postgresql works great. If you want to find things within polygons or do more complex queries, PostGIS has a lot of great functions.
Postgresql and PostGIS are both open source and free.
If you want amazing performance, use an SSD and a machine with enough memory to hold the entire database. You'll be amazed at how fast that setup is.
Related
I want to develop a Nutrition Recommender Smart Phone App. Details of project are following.
This app will:
•
guide patients about the choices of food and diet plan according to
their health issues.
provide a list of dieting plans
recommend healthy food choices for men, women, toddlers, kids etc.
according to their age, weight and health condition
• have the calorie calculator
have some good articles about diet and foods
inform people about the nutrition importance of different
vegetables, fruits, beverages, grains, oils, dairy etc.
share some innovative ideas about breakfast, lunch and dinner.
But I don’t know how to design database to implement above functionality. Can someone guide me how many tables should I create and what tables relationship exist?.
Introduction
First of all, this is very basic what you wrote us. You should start specifiing those things and get more into details. For example for a diet plan you can think about the properties such plan could have. Properties are e.g. the length of the plan, the nutrition you need and maybe some sport-excercises. Just some samples. Then you may split up the nutrition in one table and so on. Now you can think about if you want to do all the database stuff on your own, or use a framework. Actually the whole database stuff seen in MySQL and so on is done on a webserver, because of security leaks on the client (decompiling, traffic reading, ...). If you want to save data on the local storage or in a local db (SQLite would be you choice) then you should also think about if you want to use an API for that (e.g. RoomAPI by Android Jetpack)
General thing
You should get your self a clear mind which architecture you want to use. You can store data on a webserver and transfer the data via web-interfaces e.g. REST with JSON or XML. You can also store data on the local device in a file or in a database (which is basically also a file with specific interface requirements). You have to decide and consider you require the exchange on multiple devices or just for one device on it's own.
Planning your structures
After you know which architecture you want to use, you can start planning. First of all I would suggest you to get an abstract view of your project. That means just writing down what you want to have and then start writing down the corresponding properties. For a diet-plan this may be the name of the plan, the length, required nutrition and so on... After that you may know, oh okay, I also need a table with the food and you will see (automatically) that there will be a connection. If you see the connection just draw a line connecting those. In an ideal situation you already know the relation of this connection (One to many, Many to one, Many to Many, One to One, see this stackoverflow article: Difference Between One-to-Many, Many-to-One and Many-to-Many?).
Implementing your structures
Anything I do is handcrafted
In the case you want to do all on your own you may use a gui tool for databases to design your database. You can also use commands to do that, what need much more effort. You can decide on your own and on your knowledge.
Pro for gui
Easy to use
Fast creation of the tables
Contra for gui
May use some cryptic names for indexes and keys
"May don't let you look behind the scenes"
I rely on APIs
You can also rely on APIs. That means often, that you can program the class according to your plan, then annotate it and the connections/relations would be managed by the api. Such abstraction-layer is Room for Android or Doctrine for PHP, those are just samples and there are much more.
Conclusion
How you get the data into the database and out of the database is your thing. You could rely on easy use of APIs or get your things done by yourself. It should be a little guide for you to understand database design.
P.S.: If someone has things to edit, do that! I would appreciate that!
Cheers Tarik.
I have seen a lot of NoSQL libraries for mobile platform, but the main thing comes in mind, why we are trying to use NoSQL in mobile systems which are already having very limited space.
Please clarify the scenarios, where they could be beneficial over Sqlite3.
Mobile Application Development (Android, iOS) platforms having complex & rigid structure with relational database. For simplicity of handling of data.As developers come up with new ideas and features for their applications, making changes becomes a time-consuming task because constant changes have to be made to the database schema.
Another issue with mobile applications that NoSQL addresses is the need for constant updates. After an application has been released, maintenance becomes a major concern, among other things to consider. Because NoSQL is document based, fixing certain types of bugs and other problems doesn’t require a complete overhaul of the database, because the changes made by developers don’t necessarily affect every other aspect of the application.
Finally, NoSQL is well known for its scalability. Unlike relational databases, NoSQL databases scale outward rather than vertically. Now a days there are consideration of NoSQL. For More you can read below articles :-
Why NoSQL Trumps Relational Databases for Mobile Applications
Why NoSQL is Better for Mobile Apps
Hope this helps you !
NoSQL is best leveraged for unstructured and semi-structured data, thus, enter mobile application development. I would break down the benefits as such:
Low-Latency Data Storage: NoSQL databases are built to serve predictable, low-latency requests and provide a highly available experience to end users.
Scale for Peak Loads: peer to peer NoSQL databases make it easy to add additional capacity and scale out quickly versus scaling vertically with relational DB's.
Riak (by Basho) is a good one to take a look at.
not all nosql databases need much space, today we have some lightweight solutions, like iBoxDB.net and iBoxDB.java.
and what are the benefits of using nosql, there has a scenario
A database for Q&A and RPG games
I've found that in creating and using a non-sql key/value data store (https://github.com/AaronBratcher/SimpleDB) that I work in a style that uses less memory at runtime. Instead of keeping an array of objects for my long tables, I only keep an array of keys, getting the object values as needed.
I've also found that as I prototype and change the format of my classes, something I've been doing a lot of as the dynamics of my project changes, I don't need to spend as much time refactoring my db layer class because I simply change the class init/value code and spend more time working on more important aspects of my project.
My app is growing very fast, and we have used Parse.com as the main database.
We want to create a web version of this to work together with the app for iOS and Android.
The only limitation that we know from Parse.com is that the queries max limit is 1000, so after it you need to create a function to do more queries.
So talking about a very large database, performance on server-side, Users doing queries using 3g network and scalability, is a good way move it to mysql?
Not considering time and cost.
Only talking about performance, scalability and queries to work with Android, iOS and web.
Thank you
In terms of scalability I guess you really need to worry about that if you are managing your own servers. Worse case, you go with a professional hosting service but use VPS.
Parse uses mongoDB which is entirely different than MySQL. The 1000 limit should really be an issue regardless of what database you use because you should be structuring your app that you don't grab anywhere near 1000 items at once. That is where lazy loading comes into play.
Parse is a REST API. This means you're using HTTP requests to get information. You will need to write an API or find an open source option to put on your server. You need to make sure that your code runs efficiently so you can get results fast and put as little strain as possible on your servers.
Another thing is to try and prevent having 1000 queries. If there's a way where you can download data that can be reused in the app instead of re-downloading it each time then do it. Less strain on servers, quick response, and little data going back and forth.
That's really all I have to offer.
Not sure if this would help you, but I'm moving stuff out as well, but you are comparing two different databases to each other. I'm moving my data to Cassandra or MongoDB, that's because the data is key value. As for mysql, I hope you keep in mind, a lot of companies use it for large scale problems and they run it fine. I hope this helps.
How the app works
Currently an app is in the works which utility is to explore activities in 5 regions. Each activity is represented as an JS object with a fair amount of properties. Activities can be viewed through different filters in their respective tab, for example categories or a map. Inside each main filter, there are options to filter on date, region, accessibility etc.
The challenge
There is a lot of JSON that needs to be stored on the device, and support is required for both iOS and android.
In the best case scenario the data needs to be in sync with the database, and all data needs to be available on the device. The app will need to be snappy for a good experience, this means that fetching data needs to be as fast as possible. Furthermore, filtering data needs to be as snappy as possible.
Viable solutions considered so far (which don't quite cut it yet)
MongloDB with the MongloDB Titanium Store adapter, silver bullet?
This approach at first seemed be the silver bullet. Although the project seems promising, it is maintained by one heroic hacker, and the project is in need of some documentation. I have inspected the source, and hacked my way through the API, but to no avail, console.log and jasmine tests won't cut it this time. More important still, it is not quite finished yet, and features compared to MongoDB are missing. A great project, I hope it will catch on more and be capable enough to assist desperate titanium developers in the future.
JSONDB, only for iOS
This app really needs to work on both platforms, iOS and android, so no reason for trying this. Moreover, JSONDB works within a single context only, which would be a serious concern as well.
Ti Filesystem and JSON.stringify + JSON.parse, not memory efficient
A viable solution for saving a small list of saved items, which is also a feature in the app. But in other posts issues over memory limits with the use of JSON methods have been noticed for android. Though this might not be the least of my problem, memory efficiency overal will be a huge problem. Never have I seen benchmarks for performance with file reading and writing for Titanium, so I am not sure how big of an impact reading and writing would be. Filtering big objects is a huge concern as well, underscore won't manage this kind of big data. Iterating big objects is a huge problem no matter what approach I will choose.
Big ass global object
Practically the same approach as a Filesystem, only keep it in a global. This has the same issues and is just a plain unethical practice.
SQLite, yuck
Highly document oriented JSON data to SQlite, it sounds worse than samsung galaxy fanboys. Any feedback on this?
Multiple files + SQLite to maintain + lazy load, unicorns and rainbows?
Desperate for a solution, I might be onto something in the course of writing this post. There are probably something 10-16 main categories which each 1 to 4 subcategories. Keep all the activities for a subcategory in it's own file, which is a quite slim JSON. Browsing through categories, each subcategory is rendered in it's own TableViewSection, each subcategory be appended independently to the table based on how much the user is scrolled down, effectively lazy loading the content. There is only one quite quick file read. Within this view adding more subfilters effects only the already loaded items, and iterating this items is reasonably affordable.
Updating the data is also quite effective, only files that are subject to change are updated. A SQLite database can maintain the dates of all activities which have a expiry date, it can dynamically build it's own JSON file for the upcoming seven days or month. This will make the calendar view quite smooth for most usage. Picking future dates will be a nightmare though.
Still the map is an issue...
If you have read all of this, thank you. If you have experience with something similar, or might be onto something, feel free to reply! I have to quit writing, quit coding and start sleeping.
Sorry for the crappy monglodb docs. I developed it for some internal projects and really wanted to share it with the community, but the lack of docs does make it hard to use. But great news I have docs now lol also slimmed down and cleaned up the source code. Hope it works better for you now. http://monglodb.com
I'm the original author of JSONDB and thought I'd drop in and provide an answer for anyone finding this question via Googlefu.
JSONDB is now deprecated software - it's been replaced by another project called SculeJS. SculeJS aims to provide a full featured NoSQL database written in pure JavaScript for use in Titanium, NodeJS, and web apps.
JSONDB was originally only available for use in iOS applications due to limitations in the way Ti native modules were built - the current versions of JSONDB and SculeJS are compatible with both iOS and Android apps.
In a lot of ways MongloDB and SculeJS are similar, where they diverge is in the way SculeJS has been engineered. SculeJS is intended more to provide powerful, generic data structures with a rich query layer rather than being a straight port of MongoDB. No insult to Monglo - it looks like great software, I just wanted to point out the difference in intent between the two projects.
As a side note - all pure JavaScript modules are limited to to a single execution context within Titanium applications.
For what you're building I think MongloDB, JSONDB, SculeJS and TaffyDB would all do the job, the details of the implementation would just be slightly different.
I was encountering the same problem. I had about 5mb's of data which I wanted to store with the app, and not let it download.
I finally ended up with an SQLite database, with high performance. It is not as bad as you think. It might not be a nice solution ,but for the lack of choice it is a very good one IMHO.
Just create a couple of tables, and functions to parse them to database, and the other way around, and I promise you, you will be happy.
DO NOT store the JSON in the database, but store the values appropriately.
I'm working on an Android application which is fetching data from internet among other things. Actually, the project was started by someone else which is not here anymore,
and now that I have to turn it into a light client application and implement the server side (in Java), I'm wondering what would be the best tools/patterns to use to fit my needs.
Let's say I have to deal with several models (class representing a category) of objects which all inherits from one class : they have common attributes (such as name, attache thumbnail...) but specific properties too.
Because of this,you can understand that I can't afford to manage one specific table to map each single class.
However, I still want to be able to cache my objects somewhere in the Android device to populate the views of the application when working in offline mode.
Currently, the solution used by the previous developer was to store data directly into a TEXT field in the SQLIite database, as serialized objets.
This should be ok on the server side but I've read that the usual Java serializaton was very slow on the Android platform, although it is not really noticeable now because I work with around ~50 objects, I was looking for more performant alternatives for the future.
I've came across the JSON solution which can easily handle complex structures and Jackson library seems very interesting with its simplified data binding to POJO objects and its well-known performance.
But then, how should I store my Json objects ?
Is it possible to keep a json string in a TEXT field of a SQlite table ?
Or should I rather store them as .json file for each object ?
Which one is the more efficient to retrieve later lot of data?
Plus, I was thinking that JSON would be a very good exchange format between the Android client application and my server whould is in charge of processing the information from internet third-parties apis and exposing this data with webservices. (rather than trying to implement some RMI-like solution)
Is using the usual Apache HTTPClient enough on Android to communicate with the server?
For those who successfully developped client-server application (which seems very common to me) is this a good approach for Android ?
It seems to me that with mobile platforms, you can't really use the approach that you've learned for more classic J2EE app and such...
Any advice would be greatly appreciated because I'm a student and Android beginner who really want to improve her mobile development skills !
Thanks :)
That's open to discussion, so SO is probably not the best place to ask. In general, before declaring something is too slow (or fast), measure, compare and pick the one that works best for you. Yes, you can save JSON in a DB, an it will generally be faster than having separate files on the FS. But, again, benchmark and compare.
BTW, most J2EE 'approaches' (patterns) are overkill for any platform, let alone mobile.