I am looking to build a mobile app that will send 100 mb of data per instance to a "cloud" system where the data will be manipulated, machine learning models applied to it (not built upon), then the summarised data should be sent back to the users phone. I have written much of my scripts on my local machine and now I need to begin to deploy it. Both in terms of deploying in the cloud, and in app development, I have no experience.
I initially planned to run this all on GCP. However loading models is a bit of a challenge. Whilst working through this, someone suggested that perhaps what I need is a VPS. Initial views suggest yes a VPS could host my data and do my processing for me and a significant cost reduction. However, GCP has the benefit of having Firebase linked to it, and so there is then the question of how I could set up the VPS to receive the data and send it back out to the app.
I know that this is very generic and please ask me more questions so I can clarify as far as I can. My challenge is how to get started in this whole area which I do not have experience in. Links to relevant tutorials / courses which would help me on this path, would be of great help.
In terms of size and frequency of my set up:
One push of data is approximately 100mb. The data sent back will be much less (currently undefined but its summary statistics)
This will then happen up to 5 times per week, and I would hope for 5000-10000 customers
Appreciate any help and hints you can give me, and if this is deemed inappropriate because its not specific enough, could you advise where I could seek the help I am after.
Thank you,
James
You can use Google Compute Engine to create a VM for hosting your app.
To get started, you can create a GCP account which is provided with a credit and follow this quickstart guide to see how it works.
As for the additional resources to help you become familiar with the platform, I can recommend GCE tutorials and Quicklabs.
Related
I have an idea for a mobile app that I would like to make. I am experienced in programming locally on the device, but not so much at making something that has a server in it.
I have (rather hastily) rented a server at HostGator and rented the domain name.
I made a project for university before and did the server-side scripting, but it had flaws, wasn't well polished and took a lot of time. So I'd like to avoid it as much as possible.
I have been searching the internet from morning till night for services that would take the load of server-side from my bac.
I'm planning to build an iPhone and an android app that will actually be "wrappers" to my data on the server. I will need user accounts and databases in general. The app will basically connect the users in some way and organize events for them. I saw a lot of names like "Parse" and "Kinvey" but I'm not really sure if that's what I'm looking for or not. Would I need to build an API myself, put it on the server, and wrap the apps around it? Or is there (hopefully) another solution? I'm looking for general guidance and what to research exactly.
What you're looking for are BaaSs (Backend-as-a-Service) and there are plenty to choose from. I highly recommend Firebase as a starting point as it's pretty easy to work with and pretty fast to get up and running, though there may be a learning curve in grasping how you should define your data in an efficient way. Parse is now defunct and you'll probably still have to wrangle with trying to get it deployed to a server, which may be a daunting task if you don't have experience with that kind of stuff as it's now just an open source project that doesn't give you any server infrastructure to work with out of the box like they use to. Other promising BaaSs is Realm's mobile platform which I haven't tried yet.
I am planning to build a web application and android app, which will manage huge numbers of notification (push notification), and can work in slow internet connection too. I need to send and get instant notifications. Number of users can be thousands or millions, application will have multiple servers (web farm), multiple database. Now I need to decide that which database will be best for this kind of application and which language should I use for programming. Please help me out. Any suggestions will be appreciated
Well, first you need to decide what your immediate needs are. Are you going to use this on a platforms that could potentially have hundreds of people accessing information at the same time? Then you need to estimate your future needs.
This will help you to decide your database system.
As per my experience i am suggesting you to use MYSQL database.
I Blindly Suggest you to Use Parse Cloud Database,as it provides SDK for All mobile Environments like Android and IOS for easy implementation and also it recently Launched a Javascript SDK to use.Its free for Trial.MultiPlatform Support and Secure
Check it out Here: https://www.parse.com/
Are you sure you going to get to thousands and millions users ? Everyone starts from scratch (read: zero users, except some friends). By this I mean, that you have to concentrate on what's the real issue within your development (growing app user base is different story):
Creation of Android app and it's lifecycle (updates, support of previous versions & etc).
Back-end. Will I also work on Back-end. Working on 2 'projects' (Android app and it's back-end) isn't easy. Not everyone is experienced enough to work on multiple assignments at the same time.
Valuate an option of using SaaS/Paas backend. Most of the have trial or free version for developer.
Third option is great. Get cheap/free web host. Store there configuration, that your Android app will download when it starts. In configuration you should declare what's the back-end and how to communicate with it. You can use any of known services like https://www.firebase.com or https://parse.com/plans or even use Google App engine free tier / AWS free tier.
About developing app for Android - if your app doesn't need any complex calculation or libraries - just write it with JavaScript. It's fast enough. Though, Java apps are always faster and easier to debug.
Good luck !
Never built an app before.
Downloaded & installed SDK & Eclipse.
For my 'learning experience' I would like to build an app for my Galaxy S II (4.0.4) that will take me to a web site and then log me in, eg: my Gmail Account or my Voip.ms Account
Can someone point me to a tutorial or suggest how I should start such an app?
Is this too complicated to begin with?
Personally, I think this is way too complicated to start as a learner. You may end up spending a lot more time in debugging some code which does not work because of some silly stuff rather than "learning" how to code Android.
I started with a simple app that has few screens, stores some info in local SQLite DB and sends some across to an remote SQL Server over internet. I managed to complete this in 2 weeks and got quite a good hold on Android basics. Now I am working on a much complex app and am not getting stuck in fundamental issues.
Developer.android.com offers some good app examples and tutorials and is the first go-to location when you want to see how to do something Android-related.
As for what you want to do, it will really depend on the website. Going to the website in question, searching for a developer section is the way to go.
There they will explain which API they offer to devs. OAuth + REST are widely adopted but there is no universal solution, it really depends on the website.
A final note : embedding a webpage in an Android app is a very bad idea from an user-experience perspective, Using an API to offer a native app will always result in a better experience.
Actually I have a Client-Server based application developed on vb.net/C#.net now i want to port or publish that client-server Application on Android. I knew the tools for making it useful for my Android device. And I have also make out some important point to find out the cost of that project, but i'm newbie so i can't understand how to give assign or fix a amount for particular task in android. My Points are:
Develop a jazzy look user Interface for displaying a data.
Check and use the Device's network connection state.
Use the web service for accessing the method/functionality of
client-server application.
Also check and use the state of web service whether it is available
or not and use the functionality or fetch the data on the basis of
state.
Also the main thing is to maintain the server for storing the data
and fetching the data via web service.
Any help is appreciated for finding and assigning the cost/price/amount to the Android application.
This usually depends on the size of your target market. If you have a large target market, you can let the app go at a low price, because volume and economies of scale will kick in bringing your costs down while your revenues go up.
If you have a smaller target market, you're going to have to bump up the price of the application in order to recover costs of delivering to such a small market and move into the positive for profit loss.
For the most part, you can ignore the cost of developing your app. Development is mainly a one-time fixed cost that disappears over time. There will be some maintenance of the application itself that will cost money over time, but it isn't enough to worry about. Your server maintenance will be the brunt of your costs, but provided you're able to outsource most of the storage and bandwidth onto Amazon's extremely cheap pay as you go servers, you're running costs will be relatively low as well.
The greatest majority of your budget will be spent on marketing (depending on the type of app you are developing). Should you get a shoe in with the press, or market through ads on mobile, Facebook and the like. This is where most of your money will be going. Make sure to offset the cost of conversion with the revenue generated by conversion.
If you gave me more details, I could get more specific and help you out some more, but with what I have to go on, I have to stay broad. I wish you the best of luck with your venture.
I have been trying to find an answer to this vital question for a few months but to no avail. So I am posting it here since stackoverflow is the best place for technical questions.
I made an Android app, name of which I'll not mention at this stage. It syncs with a central MySQL database via a webservice on LAMP, which also I created. It all works just great, no problems whatsoever. As the users use the app and perform certain actions, data from their devices gets uploaded to this central server which is running in a data center. As other mobile devices syncs with the database, they get updated with the newer data. At the initial install, app gets a few hundred K of data downloaded via few hundred queries.
Now the problem is that the app has started to get so many downloads that I can foresee I'll not be able to keep up with the sync. For a while I have been optimizing the code both on the Android end and on Web end (with iPhone version still under development), and also using caching techniques, but still, the communication between thousands of mobiles devices and one single database is reaching its limits. Everyday the number of downloads are more than previous day. Now I am extremely worried how to handle this situation.
I have been looking into using Amazon DynamoDB, have also went over its SDK, tutorials, etc. and am ready to spend a few days and switch this whole sync thingy to Amazon. But this will eventually cost me heavily, for which I'll have to charge for the app or use a better advertisement scheme. But that's another issue.
Unfortunately Googling for weeks hasn't helped me on figuring out how to efficiently handle large syncs with thousands of mobile devices, which might get into tens of thousands or probably hundreds of thousands.
In short, what to do to, lets say, deal with 100K devices using my app and in need of keeping in sync with a central database?
Kindly answer if you have faced a similar situation and know how to deal with it in real life.
Thanks in advance.
A good friend of mine Robert, the server admin at Shopify, guided me in this regard and I thought I should briefly share it here.
Its true that for a large number of users a large number of servers is required. The challenge here is how to efficiently manage this multiple server scenario.
I also had this question asked in a Google interview which surprisingly I answered correctly, or that's what the interviewer said. I also worked on this type of problem for a company, who was working on a project for Verizon, dealing with hundreds of thousands of customers accessing the server farm via a mobile app.
I noticed same approach for all of the above. This gave me a relief that my own method was also not much different.
So if you are using e.g. Amazon services, use their ELB, i.e. Elastic Load Balancing solution. Use it by cloning the main server, each being turned on or off based on the load on the previous server. You'll also need a DNS server with multiple CNAME records to direct load to multiple IPs. This is a very interesting technology in itself, but Amazon makes it much simpler. DNS in itself is a complicated technology, however a little reading on using CNAME and using it for routing and balancing via Amazon will help very quickly.
Now there is obviously also a database involved in any given architect. In my case it has always been MySQL. I tried and gave up on Amazon's DynamoDB (a NoSQL solution) due to reasons of cost and complexity. Also one of my ex-bosses who was much better than me in understanding differences between SQL and NoSQL explained me once why NoSQL was a poorer choice than SQL. So in case of our problem, we setup one master database, and then its slaves. Each instance of the REST server answering our REST queries interacts only with its own MySQL slave, which in turn is getting updated from the main master database. Its true that this will create a lot of server instances, and will cost a decent amount of money on Amazon, but this is how it is done.
In the Verizon scenario, the solution was from Microsoft and each server was supposed to take load from 25000 users before switching on the next server for another 25000 users. However as soon I hear the word Microsoft, I lose all my confidence on any proposed solution. I could see the issues they had with the whole .NET architecture and licensing. I rather stick with Linux based solutions.
I have seen data centers in Singapore servicing North American market, and speeds are pretty bad. Amazon also offers data centers in different parts of the world. It is really important that your servers are where your audience is. For your blog or similar stuff you can use service from CloudFlare for example, but for your app, better implement your own intelligent architecture with strategically placed servers.
And in the end, I rely heavily on Amazon cloud services and now they have many new solutions to make things even easier for dealing with large number of requests from large number of apps. I don't know all of these solutions but the main architecture remains the same as I described here.
Good luck with your next great app :)
Option 1) If assume your application has lots of reads from the MySQL DB, in that case first option is to add Read replica's(RR), You can add upto 5 Read replica's
Option 2) Have ElastiCache Clusters sit between your webservice and MySQL DB
Option 3) Look at implementing DynamoDB. It is not a replacement for your MySQL and requires good amount of coding effort if your data architecture is tightly dependent on MySQL