How the app works
Currently an app is in the works which utility is to explore activities in 5 regions. Each activity is represented as an JS object with a fair amount of properties. Activities can be viewed through different filters in their respective tab, for example categories or a map. Inside each main filter, there are options to filter on date, region, accessibility etc.
The challenge
There is a lot of JSON that needs to be stored on the device, and support is required for both iOS and android.
In the best case scenario the data needs to be in sync with the database, and all data needs to be available on the device. The app will need to be snappy for a good experience, this means that fetching data needs to be as fast as possible. Furthermore, filtering data needs to be as snappy as possible.
Viable solutions considered so far (which don't quite cut it yet)
MongloDB with the MongloDB Titanium Store adapter, silver bullet?
This approach at first seemed be the silver bullet. Although the project seems promising, it is maintained by one heroic hacker, and the project is in need of some documentation. I have inspected the source, and hacked my way through the API, but to no avail, console.log and jasmine tests won't cut it this time. More important still, it is not quite finished yet, and features compared to MongoDB are missing. A great project, I hope it will catch on more and be capable enough to assist desperate titanium developers in the future.
JSONDB, only for iOS
This app really needs to work on both platforms, iOS and android, so no reason for trying this. Moreover, JSONDB works within a single context only, which would be a serious concern as well.
Ti Filesystem and JSON.stringify + JSON.parse, not memory efficient
A viable solution for saving a small list of saved items, which is also a feature in the app. But in other posts issues over memory limits with the use of JSON methods have been noticed for android. Though this might not be the least of my problem, memory efficiency overal will be a huge problem. Never have I seen benchmarks for performance with file reading and writing for Titanium, so I am not sure how big of an impact reading and writing would be. Filtering big objects is a huge concern as well, underscore won't manage this kind of big data. Iterating big objects is a huge problem no matter what approach I will choose.
Big ass global object
Practically the same approach as a Filesystem, only keep it in a global. This has the same issues and is just a plain unethical practice.
SQLite, yuck
Highly document oriented JSON data to SQlite, it sounds worse than samsung galaxy fanboys. Any feedback on this?
Multiple files + SQLite to maintain + lazy load, unicorns and rainbows?
Desperate for a solution, I might be onto something in the course of writing this post. There are probably something 10-16 main categories which each 1 to 4 subcategories. Keep all the activities for a subcategory in it's own file, which is a quite slim JSON. Browsing through categories, each subcategory is rendered in it's own TableViewSection, each subcategory be appended independently to the table based on how much the user is scrolled down, effectively lazy loading the content. There is only one quite quick file read. Within this view adding more subfilters effects only the already loaded items, and iterating this items is reasonably affordable.
Updating the data is also quite effective, only files that are subject to change are updated. A SQLite database can maintain the dates of all activities which have a expiry date, it can dynamically build it's own JSON file for the upcoming seven days or month. This will make the calendar view quite smooth for most usage. Picking future dates will be a nightmare though.
Still the map is an issue...
If you have read all of this, thank you. If you have experience with something similar, or might be onto something, feel free to reply! I have to quit writing, quit coding and start sleeping.
Sorry for the crappy monglodb docs. I developed it for some internal projects and really wanted to share it with the community, but the lack of docs does make it hard to use. But great news I have docs now lol also slimmed down and cleaned up the source code. Hope it works better for you now. http://monglodb.com
I'm the original author of JSONDB and thought I'd drop in and provide an answer for anyone finding this question via Googlefu.
JSONDB is now deprecated software - it's been replaced by another project called SculeJS. SculeJS aims to provide a full featured NoSQL database written in pure JavaScript for use in Titanium, NodeJS, and web apps.
JSONDB was originally only available for use in iOS applications due to limitations in the way Ti native modules were built - the current versions of JSONDB and SculeJS are compatible with both iOS and Android apps.
In a lot of ways MongloDB and SculeJS are similar, where they diverge is in the way SculeJS has been engineered. SculeJS is intended more to provide powerful, generic data structures with a rich query layer rather than being a straight port of MongoDB. No insult to Monglo - it looks like great software, I just wanted to point out the difference in intent between the two projects.
As a side note - all pure JavaScript modules are limited to to a single execution context within Titanium applications.
For what you're building I think MongloDB, JSONDB, SculeJS and TaffyDB would all do the job, the details of the implementation would just be slightly different.
I was encountering the same problem. I had about 5mb's of data which I wanted to store with the app, and not let it download.
I finally ended up with an SQLite database, with high performance. It is not as bad as you think. It might not be a nice solution ,but for the lack of choice it is a very good one IMHO.
Just create a couple of tables, and functions to parse them to database, and the other way around, and I promise you, you will be happy.
DO NOT store the JSON in the database, but store the values appropriately.
Related
I am a new guy to full stack web application development. I want to design a web application which has data stored in say back end databases. Now I want to design a desktop web client as well as android application which will be able to fetch data from back end. So how do I need to start? What APIs can be used or how can I expose data from same back end to multiple clients?
Also I want to handle massive amount of request. How to design such a system? What to use in back end to store data and handle requests efficiently.
Any video / document / reference containing useful information will be much appreciated.
Wow, you have a whole forest of questions to settle. You are going to need to go do your own research on such things as algorithms and data flow for your application before you can make any reasonable choice of platform. Here are a couple of basic ideas to get you going: 1) look at Java and Node.js. There are lots of other possible platforms but chances are you will end up using one of those two. Try to think about what the actual code you will generate in each of those will look like. A little or a lot? 2) Just store your data in files, most probably using JSON. Maybe you will end up doing something more fancy after you figure out where you are going with your project, but you will be surprised how well the simple file-based solution will scale.
When you have done a bunch more research, and maybe even coded up a few ideas on your platform of choice, then come back and massively edit your question. Only then will specific suggestions for tool choices be possible.
I plan to write an Android app that allows the users to share little pieces of information about a place, e.g., a bar, restaurant or the like. I'm now looking into what is the best way to store such data. I'm particularly concerned about retrieval: When showing a (Google) Map, I'd like to overlay it with said information, so I'll need to be able to make geospatial queries to the database.
Is this possible with the Google Cloud Datastore, for example? What are the alternatives?
Datastore does not have support for geospatial properties per se, but you can easily create a convenience class to work with it. In App Engine, there is a property GeoPt just for this purpose both in the Java and Python libraries.
If you are interested in geo queries there is not much that Datastore can do for you, but there is an interesting lib in Python to deal with that, geomodel. The problems are still on performance and costs. The operations performed there, although optimized, are heavy.
There's one last alternative. Search API allows you to perform geo queries. The only drawback of this are costs and limited free tier.
You can try a quadkey. Its similar to a geohash or a geocell. Translate the points to a binary and interleave it and treat it as base-4 number. It comes it looks like a space filling curve and it has the same properties. Here is a good article about quadkeys:https://msdn.microsoft.com/en-us/library/bb259689.aspx.
You might want to look at Postgresql and even PostGIS for storing geo data. They are fairly efficient, scalable and fast. I've used Postgresql with geodata for many years and on very busy websites and Android apps. If all you need is to locate things near other things, Postgresql works great. If you want to find things within polygons or do more complex queries, PostGIS has a lot of great functions.
Postgresql and PostGIS are both open source and free.
If you want amazing performance, use an SSD and a machine with enough memory to hold the entire database. You'll be amazed at how fast that setup is.
I am asking for your advice and a sanity check, please. For a couple of years I have thought about building a mobile site for my own purposes (initially) that will allow me consistent interface on the web and on a mobile phone. My initial target is Android 4 and above.
The central idea is to make notes via web interface. It is easy to
edit on a full PC. Basic formatting is necessary - bold, italic,
lists. I should have some basic editing / adding capability from the phone.
Possibly I might need to add images. This at a later stage.
I will need to use tables.
View these categorised notes on my phone when I need to access that information quickly.
Information should be searchable.
I should be able to categorise that information both hierarchically and by labels.
On the phone the last copy of the information should be available even when there is no internet access. That is the key bit of this project - clear, well structured information, easily available when needed.
So far I have done some 7 websites 6 - 9 years ago and this year I did a responsive design interface with jQuery and HTML5. I have access to Apache on a linux hosting site. I have looked into jQuery Mobile. I think jQuery mobile should be great for my front-end. From my past work with LAMP servers I think I have to have a database backend to this project. What I have looked into so far suggests that the best would be to invest some time in learning Ruby on Rails (which looks very interesting to me) and use that as a back end interface with a database. What database I should use, I don't know. I have in the past used mySQL, also programmed Access and a bit DB2. I have heard a lot of good stuff about postgres. However considering the mobile phone, I think I should go with SQLite probably? What should be the general mechanism for updating the phone from the net?
How does the above look?
Maybe there are some projects which already do something like this?
I don't think I should use some big CMS systems with their bloat and all, because I really want this thing to work for me. Same goes for generic note taking systems such as Evernote. So I am wanting to build it from the ground up, unless there is some similar lightweight project that I could build upon. I am not in a particular hurry, but I am about to get started soon.
A great type of answer for me would be one which tells what major components I should use for this project. It also would be good to know if I am overlooking some important bits.
I appreciate that what I am asking is a large question, but I am not expecting a Wikipedia in response. A sanity check and pointers in the right direction is what I need.
Thank you for your time and thoughts.
What a wall of text... 1.-7. are all achievable by using jquery mobile and HTML5 and a good database architecture as backend. I don't see any BIG problems in your idea. What database you use is up to you, your requirements are not that special, that it would require some specialized product. Any established database backend should do the job.
For updating the content on your phone take a look on HTML Offline Cache and implement an update feature in your Frontend.
First time posting and first time working on Android, so go easy if I am breaking any rules :)
Anyways, I just got into making an Android app and I'm trying to create a simple trivia game. I plan to have many questions (hopefully about 5000+ questions) made. No data manipulation made, just straight up reading the questions and presenting it to the user. I am now faced with the dilema in how to store the questions.
I have two choices:
1.) Bundle the questions with the app: Possibly store the information in SQLite. Originally, for demo purposes, I placed the questions in an XML file, but I quickly realized how inefficient it will be once the questions start piling up. First, I am concerned if opening up such a huge XML file would suck up Android's memory. Secondly, I am worried how large the app would be if it contains 5000+ questions. I read here about the pros of XML vs SQLite. In that example, the user has 70,000 entries so maybe my 5,000 questions would be enough?
OR
2.) Host the questions on a server: I believe the upside is the app wouldn't need to be bundled with a vast amount of questions and wouldn't need to worry about the logic of opening and assembling the questions. It would just hit a PHP page and depending on the parameters sent, the PHP page would return the questions in XML format. The downside is the user would need to be online (to retrieve the questions) in order to play the game and my server would need to be up and running 24/7.
Has anyone encountered this design issue of how and where to store vast amount of data in an Android app?
Thanks, any help would be much appreciated!
From a marketing perspective, I think the right approach is #1. Just looking at the top app list, it's filled with apps that work offline. I don't know why most people fail to mention this one important criteria when they talk about marketing apps... especially in Android, where lots of people don't have a data plan. Plus, a trivia app sounds like something someone would probably use during a commute in a train as opposed to something like Facebook Chat that they'd use when they're online.
From a technical perspective, storing 5000+ questions really won't take a huge amount of space. There's this app called "MyFitnessPal". It stores maybe over 30,000 foods in a SQlite database, with nutritional information. So don't overestimate how much space it will take.
The advantage of going with #2 is if the questions or answers change often, you might want to go with that approach.
I think that the best option for you is to store the questions in SQLite.This way the users can have access to your game even if they are offline,which is one of the important things as Hisoka mentioned. So I think you should try using SQLite.
I would like to create a custom open source information management application, that I could adapt to practical needs of a psychiatrist. (and also would really like to avoid inventing the wheel). Could someone advise me what would be a good way to start? I have some amateur programming experience (see below), but I have little experience with java and eclipse (which I have set up with Android SDK plugin on my linux PC).
What I would like to achieve is a simple app where I could relatively easily re-program/rearrange information presenting screens. I would like the major development stages to be something like:
Basic framework of the app is done Main information is added
through the programming environment
basic formatting (bold, color, lists) should be supported
I imagine this would be through HTML/CSS markup Hierarchical (not
too deep) screens with concise information about conditions,
treatments; easy, not too cumbersome navigation is a must.
Possibility to attach own notes from android device
on every piece of information (e.g. on medical condition, on medication,
list of differential diagnoses, etc)
A desktop application that allows for easy adding and editing
of the content
initially could be even sqlite manager or something similar
possibility for adding images,
possibility to add rating scales, which would calculate scores
maybe also possibility to print through some android print driver
* in essence I envision this as a sort of an integrated website with a relatively simple database behind it (sqlite?) running on android. My models would be epocrates and medscape apps, but the content would be more specific to everyday psychiatric practice. It would include aspects of psychological therapies as well.
In about a years time, I think, the content would be interesting enough to see whether the app should go on the market.
My (limited) experience:
I am a MD - a Psychiatrist. I have some amateur development experience (php, html, css, Perl, bash, sed, sql, Access and some time ago - AS/400). I would like to start an app that I could modify as I go - in order to have the most useful info at my fingertips, when I need it in my work. I have considered some databases (e.g. HandDB) but my experience with them is that you get very constricted with their user interface and this hinders fast information retrieval when you need it. Adding information can be clumsy as well.
Why I am asking for help here
Most of tasks above seem pretty standard. I imagine that people with lots of programming experience here, probably would be able to point me towards some information that would help me to get started.
Many thanks for considering this :)
Are you going to develop this app? Everything you mentioned can be easily done by reading the android documentation. Although if its you doing the coding i recommend buying a book
You will find a lot of samples and other support in the android doc
Printing can be also setup using Google Cloud print
If you want to open source your application you can always use google code.
You have asked a really broad question here. But i guess the above information is good to get you started.