What I have: A trained recurrent neural network in Tensorflow.
What I want: A mobile application that can run this network as fast as possible (inference mode only, no training).
I believe there are multiple ways how I can accomplish my goal, but I would like you feedback/corrections and additions because I have never done this before.
Tensorflow Lite. Pro: Straight forward, available on Android and iOS. Contra: Probably not the fastest method, right?
TensorRT. Pro: Very fast + I can write custom C code to make it faster. Contra: Used for Nvidia devices so no easy way to run on Android and iOS, right?
Custom Code + Libraries like openBLAS. Pro: Probably very fast and possibility to link to it on Android on iOS (if I am not mistaken). Contra: Is there much use for recurrent neural networks? Does it really work well on Android + iOS?
Re-implement Everything. I could also rewrite the whole computation in C/C++ which shouldn't be too hard with recurrent neural networks. Pro: Probably the fastest method because I can optimize everything. Contra: Will take a long time and if the network changes I have to update my code as well (although I am willing to do it this way if it really is the fastest). Also, how fast can I make calls to libraries (C/C++) on Android? Am I limited by the Java interfaces?
Some details about the mobile application. The application will take a sound recording of the user, do some processing (like Speech2Text) and output the text. I do not want to find a solution that is "fast enough", but the fastest option because this will happen over very large sound files. So almost every speed improvement counts. Do you have any advice, how I should approach this problem?
Last question: If I try to hire somebody to help me out, should I look for an Android/iOS-, Embedded- or Tensorflow- type of person?
1. TensorflowLite
Pro: it uses GPU optimizations on Android; fairly easy to incorporate into Swift/Objective-C app, and very easy into Java/Android (just adding one line in gradle.build); You can transform TF model to CoreML
Cons: if you use C++ library - you will have some issues adding TFLite as a library to your Android/Java-JNI (there is no native way to build such library without JNI); No GPU support on iOS (community works on MPS integration tho)
Also here is reference to TFLite speech-to-text demo app, it could be useful.
2. TensorRT
It uses TensorRT uses cuDNN which uses CUDA library. There is CUDA for Android, not sure if it supports the whole functionality.
3. Custom code + Libraries
I would recommend you to use Android NNet library and CoreML; in case you need to go deeper - you can use Eigen library for linear algebra. However, writing your own custom code is not beneficial in the long term, you would need to support/test/improve it - which is a huge deal, more important than performance.
Re-implement Everything
This option is very similar to the previous one, implementing your own RNN(LSTM) should be fine, as soon as you know what you are doing, just use one of the linear algebra libraries (e.g. Eigen).
The overall recommendation would be to:**
try to do it server side: use some lossy compression and serverside
speech2text;
try using Tensorflow Lite; measure performance, find bottlenecks, try to optimize
if some parts of TFLite would be too slow - reimplement them in custom operations; (and make PR to the Tensorflow)
if bottlenecks are on the hardware level - goto 1st suggestion
Maybe you should try this lib, it can run on android and ios devices.
https://github.com/Tencent/TNN
Related
I am just starting to learn OpenCV for Android, I have played around with it a bit and it all works fine.
I installed the NDK and managed to run some of the sample apps which included it.
I am not clear on what we need the NDK for. I could not find it referenced anywhere in the documentation.
Is there OpenCV functionality which is not available in the regular library 2.4.8 ?
Is it just so we can use modules written in c++ that others have made available, without rewriting them in java ?
I have been using NDK for my application and following are my
observations.
Yes, it gives the advantage of using plenty of c++
modules which are made available.
If you already have a bit of
experience in computer vision application programming using openCV
library in C++, you don´t have to learn new syntaxes in java(can be quite
irritating some times).
Using NDK for your app can give you a slight upperhand in terms of
performance when image processing you do is computationally
demanding (I am not quite sure about this because the openCV
library made available for Android is just a wrapper around the
same header files and stuff and should almost give the same performance as NDK. I have never really compared the
performance myself but have read in various blogs that NDK is faster).
One thing
you really have to be careful when using NDK is calls from JAVA to
NDK side or the other way round, these calls can be really expensive in terms of performance(Needs careful planning).
Passing few parameters like array of MAT from JAVA to NDK are a bit
of headache, but you can find few workarounds.
Based on these and other factors you might have found out from various sources and also by your programming strengths you can decide if you want to use NDK or not. There was never really a set of guidelines i could find that says you can use NDK if so and so conditions are satisfied, people just start with which ever programming style they are more comfortable with.
In Android documentation they advice to use NDK in case of
# CPU-intensive operations that don't allocate much memory,
# such as signal processing, physics simulation, and so on.
If everything you want to do can be computed with OpenCV built'in functions, you may not need NDK as processing routines of OpenCV are already in C/C++.
However, if you have to process the images matrix intensively (I mean direct access to pixels), you will improve performances using the NDK.
I created an Android app. While creating one specific app was an interesting challenge, I'm now looking into creating a group of similar apps.
I'd like to create a group of similar Android apps and then move on to creating the same on tablets and iOS... (anything mobile).
I've considered doing so with a product called PhoneGap or doing a web based mobile app. Both of these options seem less than ideal. Doing the Android app I've been frustrated by Java's lack of control and low level constructs. Moving to something like a web based app seems like the exact wrong direction.
C++ is my language of choice. It has the ability to work at a low level, is highly portable across platforms, and has significant support for generic coding which would be useful for generating a group of similar apps. However, the Android documentation suggests to not use C++ unless your goal is porting existing code or dealing with computationally heavy tasks.
I'm leaning towards using C++ anyway, but are there other options I've not considered?
Thanks
You could in theory write your logic in C++ and then have UI layers on top that make use of it. If you are really comfortable with C++ that might be the way to go.
Almost any other parts (networking, UI, animation, etc) are better off being done in the native language of the platform. Use of cross platform solutions always limits you in some way, and usually leads to an application that is not as good as it could be for any platform.
Well, Google's recommendation to not use C++ is based on the following, I believe. C++ is low level, so you can get extra performance out of it if you know what you are doing. Google makes the reasonable assumption that many programmers do not. It is easier for an inexperienced programmer to do harm in C++ then to get a performance boost.
But, if you know what you are doing, it can help you. UI elements on both iOS and Android are implemented in their main language (obj-c, and Java respectively) so there is not a great way around that, but you can write core logic and other functions in C++ and it will be portable between them (iOS can use C++ directly and Android can use it via the Native Development Kit).
There are a few other options available. The one I ended up using is Appcelerator Titanium but please stay away from it. If your project gets complicated or large at all you will hate yourself for choosing it, as I did. Another interesting one that uses C++ instead of Javascript is Marmalade. I haven't used it though, so I can't comment on it.
A non-free solution that I hear good things about is Xamarin, who have ported both environments to C# and a .NET using Mono. However, you still have to write two versions of your code for the UI as far as I can tell.
My work consists of porting an iPad application to Android (so from Objective-C to Java). I've developed in the Android environment, but never in iOS. Currently I am looking for advice -- I'd like to know if is there a way, method, or process which can help me to do this more easily.
At this stage generally the answer is no, but I've got some suggestions that might help.
Use UIWebView/Webkit extensively - baring any HTML5 media (and SVG), there's little porting required when you render HTML. What you render in one, generally, renders well on the other.
If your iOS app is a basic show-this-edit-that style of app, you may find PhoneGap, jQuery Mobile, Titanium or the new Adobe suite a better time investment.
Because Android lacks a consistent device base, which makes developing animation-rich UIs difficult, you might find that there is no 1-1 UI comparison. Instead think about the features you offer, and their underlying data and view models.
You should be able to create a similar UIViewController/Activities structure although the tying logic behind the scenes will be platform specific. Map this out on some paper - it makes a really good what-the-hell-do-i-do-next plan for your UI skeleton.
Prefer an intermediate abstraction between your 'in data' and 'out data' so you can exchange parsers/kits/apis/frameworks without hacking everything to bits.
Where you have custom draw routines and graphics, ensure that you've got filler gradients or colours. Android uses a box model to support the many many screen sizes - iOS only has 4 resolutions to worry about. Recreating the same look might take too much time - contrasting pastel colours are a good placeholder until you can justify making the artwork.
There's not a lot more that can say that hasn't already been said before.
Hope this helps!
Few advices:
You will in most cases need to do redesign of UI as iPad, being tablet PC and having a larger screen as compared with most Android devices. Advice: Check if there is a version of the application dedicated for iPhone as it could be very helpful during redesign stage.
Check if iPad code contains beside Objective-C code also C/C++ code. If there is significant amount of code you might consider using Android NDK which allows C/C++ code to be used together with Java.
If the iPad application was using one of the popular cross platform frameworks (e.g. PhoneGap) that are based on HTML5/CSS/JavaScript combination you might be able to reuse most of the code. Both iPad and Android browser/ui elements responsible for rendering HTML5 are based on same engine (WebKit) and generally generate similar experience. Note: Some frameworks allow custom access to native functionalities which will require rewrite in almost all cases.
Do not try to create exact same UI on Android if it conflicts with user interface guidelines. As you probably know the guidelines are available at: http://developer.android.com/guide/practices/ui_guidelines/index.html
Android users are used to certain interface and they should be getting similar experience as other apps available for Android. Also it might be more than painful to create similar controls for Android that exist on iPad. If there is pressure to make same "look and feel" here is a post that explains how to defend from such pressures: Porting iPhone applications to Android? How to convince them NOT to
Numerous features existing on iPad will not be available on Android (e.g. Android application splash screen using just an image). So analyse the iPad application and decompose it to set of functionalities. Once you have them then you can map them to Android API and start coding.
There are no Objective-C to Java converters at the moment (this might change in the future) but for general feeling what are differences between Objective-C and Java you might want to read "Porting Objective-C to Java" by Theresa Ray of Tensor Information Systems Inc
I was using opencv for some time for programming in Android, and I now see that the Gimp library is much stronger. Where can I find a starting point to learn Gimp?
I also want to know the basic concepts behind of Gimp plugins. In the past, I used C APIs in opencv. How could I write the code for android?
Also, what packages do I need to install in windows to start using Gimp?
ALthough GIMP dows have some standalone libraries that perform some image manipulation, most image manipulation is done either by GIMP's core program or through GIMP's plug-ins. Both approaches need to have the entire program installed and running (though not necessarily usin a display).
I know nothing on Andorid progrmaing, and don't knwo how can one install ordinary native code in C and call it from Android apps - if you are very familiar with it, you might have a chance in your attempt.
However GIMP itself relies on a extensive ecosystem of libraries, including, but not limited to, glib, gtk+, cairo, pango, gegl - and each of these in turn might have other pre-requisites. Since Windows does not have a working package manager to authomatically install libraries and header files of these various libraries, working with these natively on Windows, though the code of each of them is multiplatform and can run on Windows and other OSses,is very hard. So hard that hthe people who build GIMP for Windows themselves do so in a Linux environment, from where they cros-compile GIMP for Windows.
Making all of these libraries work on an Android is probably not hard if you are using the GNU ecosystem around the Android's Linux kernel , and not just the bare Android environment (I don't know enough about android to even know if that is possible).
All in all: it will be though for you, and demand a whole lot of research.
One of GIMP's libraries, the GEGL (Generic Graphics Library) has a lot less prerequistes, and can be used as an ordinary library. I think you can probably build it with just glib and Babl as prerequisites. This is the library that will replace current's GIMP core, and reimplement the operations of most existing plug-ins -- so it might be enough for you.
If you can get GEGL running and usable from an Android system share that with the World --it would be , in itelsef, a project worth of a Google Summer of Code project. (And still would be about an order of magnitude easier than getting GIMP code in there to be used as a library from other applications).
Finally -- if you want just a couple of GIMP's effects, if the effect is implemented as a Plug-in in GIMP, the plug-ins' code is quite straightforward. So, while it would be hard to get the whole GIMP environment inside Android, copying the functions that actually perform the pixel manipulation from GIMP's source tree and converting them to work in a java method inside your app would not be hard. Just remember to comply with the license in this case: GIMP's plugins code is under GPLv3. (the GEGL library is only LGPL)
In short: no, you can't use GIMP's "libraries" as native code from an Android app -if you can use OpenCV, you have a good chance of being able to use GEGL instead. Only orting the algorithms of certain plugins to manipulate pixels in your app would be easier.
However -- if your application would allow delegating Image Processing to an internet based server, setting up an HTTP application to receive a image, use GIMP to process it, and stream it back would be a simple thing to do.
(So, you could not apply effects in real time, but would allow one to, for example, take a photo, select a series of effects from menus, and send it to the server for processing)
GIMP uses quite a bit of memory when loading brushes. If you drop all of the useless plug-ins, and build it from source. You may be able to get it working but you will have to build ALL of the linked libraries directly into the executable.
In other words; build linked libraries directly into the code as a static build. In this manner things may function properly unless one of those linked libraries call another linked library.
Getting the libraries themselves to work on the OS may provide additional programs opportunities to use them. Additionally, GTK+ (GIMP Tool Kit), GIMP's interface is also rather bloated and ugly.
If all else fails, you'll simply have to settle for a smaller program with the features you're looking for on the fly ( Levels, Curves, the clone tool, dodge and burn, etc. ) Layers are also nice, but editing a a large megapixel image begins to eat up memory rather quickly and most android device don't have a swap partition.
I'm doing some initial research on smart phone development, and I noticed that Android and Windows Mobile both support c++ for application development. I was curious if anyone had any experience trying to manage shared files between both Android and Windows Mobile, and to what extent that code can be shared? e.g. no user interface can be shared, but web service and business logic classes can be shared, etc.
I can't speak to the WinMo side of things, but on the Android side you should really really really avoid using native code for anything except performance-critical processing algorithms. JNI/NDK stuff does not play nicely with the normal Dalvik lifecycle and can be a source of all sorts of ugly unpleasant bugs and memory leaks. From what I understand there also aren't on-board NDK libraries for more complex high-level functionality like HTTP (just more basic/performance-oriented libs like libz and OpenGL), so you'd probably have to compile that stuff and ship it with the app as well. I would definitely not recommend coding your web service classes in C++, even if it's technically possible; it'll be less buggy and nicer to write C#/Java and you should be able to make mostly the same architectural decisions for consistency.
That said, if you have a performance-critical bit of image processing code or the like, it actually can be fairly straightforward to get that working across Android to other platforms (I've seen it done quite well with some image-processing C code used in an iPhone app and then used via the NDK in an Android app).
Check the documentation on the NDK for details on what it can (and can't) do, and see similar SO threads like this one.