Develop Reference

Fastest way to run recurrent neural network (inference) on mobile device

What I have: A trained recurrent neural network in Tensorflow.
What I want: A mobile application that can run this network as fast as possible (inference mode only, no training).
I believe there are multiple ways how I can accomplish my goal, but I would like you feedback/corrections and additions because I have never done this before.
Tensorflow Lite. Pro: Straight forward, available on Android and iOS. Contra: Probably not the fastest method, right?
TensorRT. Pro: Very fast + I can write custom C code to make it faster. Contra: Used for Nvidia devices so no easy way to run on Android and iOS, right?
Custom Code + Libraries like openBLAS. Pro: Probably very fast and possibility to link to it on Android on iOS (if I am not mistaken). Contra: Is there much use for recurrent neural networks? Does it really work well on Android + iOS?
Re-implement Everything. I could also rewrite the whole computation in C/C++ which shouldn't be too hard with recurrent neural networks. Pro: Probably the fastest method because I can optimize everything. Contra: Will take a long time and if the network changes I have to update my code as well (although I am willing to do it this way if it really is the fastest). Also, how fast can I make calls to libraries (C/C++) on Android? Am I limited by the Java interfaces?
Some details about the mobile application. The application will take a sound recording of the user, do some processing (like Speech2Text) and output the text. I do not want to find a solution that is "fast enough", but the fastest option because this will happen over very large sound files. So almost every speed improvement counts. Do you have any advice, how I should approach this problem?
Last question: If I try to hire somebody to help me out, should I look for an Android/iOS-, Embedded- or Tensorflow- type of person?

1. TensorflowLite
Pro: it uses GPU optimizations on Android; fairly easy to incorporate into Swift/Objective-C app, and very easy into Java/Android (just adding one line in gradle.build); You can transform TF model to CoreML
Cons: if you use C++ library - you will have some issues adding TFLite as a library to your Android/Java-JNI (there is no native way to build such library without JNI); No GPU support on iOS (community works on MPS integration tho)
Also here is reference to TFLite speech-to-text demo app, it could be useful.
2. TensorRT
It uses TensorRT uses cuDNN which uses CUDA library. There is CUDA for Android, not sure if it supports the whole functionality.
3. Custom code + Libraries
I would recommend you to use Android NNet library and CoreML; in case you need to go deeper - you can use Eigen library for linear algebra. However, writing your own custom code is not beneficial in the long term, you would need to support/test/improve it - which is a huge deal, more important than performance.
Re-implement Everything
This option is very similar to the previous one, implementing your own RNN(LSTM) should be fine, as soon as you know what you are doing, just use one of the linear algebra libraries (e.g. Eigen).
The overall recommendation would be to:**
try to do it server side: use some lossy compression and serverside
speech2text;
try using Tensorflow Lite; measure performance, find bottlenecks, try to optimize
if some parts of TFLite would be too slow - reimplement them in custom operations; (and make PR to the Tensorflow)
if bottlenecks are on the hardware level - goto 1st suggestion

Maybe you should try this lib, it can run on android and ios devices.
https://github.com/Tencent/TNN

Using PyQt5 to interact with HTML content in Android app

As the title says, I need a way to use PyQt5 to interact with HTML/JS web elements from within an app that should be deployed (also) to Android. I started looking at QWebEngine and QWebChannel and the result in terms of interaction is exactly what I need. Sadly, I realized that QWebEngine is not supported by Qt for Android, and I am kind of stuck right now.
I've been reading that is somehow possible to use the deprecated QWebKit instead of QWebEngine, but:
One would need to compile PyQt with QWebKit (and I don't know how to do that)
I have no idea how to replace the functionality of QWebChannel (is this required or should I be able to use it along with QWebKit?)
I don't know if it worths the effort, given that QWebEngine is newer and possibly it will be supported in the (hopefully near?) future.
Last, I should add I don't know anything about QML and its interactions through PyQt, if this is important in this scenario.
Can anybody suggest if using QWebKit now is the way to go, or an alternative path to follow? Should I consider giving up using PyQt for my purpose at present?
-- EDIT1 --
I think I found something interesting regarding using QtWebView.
Qt WebView provides a way to display web content in a QML application
without necessarily including a full web browser stack by using native
APIs where it makes sense.
This is useful on mobile platforms such as Android, iOS, and WinRT;
especially on iOS, where policy dictates that all web content is
displayed using the operating system's web view.
Unfortunately, this imples understanding a bit of QML and using QQuickWidget to embed the QML webview inside a widget (as suggested here).
Does anybody have experience in using QtWebView with PyQt5 and can provide some suggestions? Would this be a reasonable path?

C++ vs native implementation of iOS/android

I'm developing an app for iOS and android that identifies songs. We have a matching engine in c++ that works well, but my question is about the rest of the app. We have a user interface design that allows the user to record, match and save, as well as share, edit, and see their recordings.
I'm not a coder - just a designer- and I'm getting conflicting advice about how to implement. One person wants to do the majority of the interface in c++ with just a thin native GUI skin, while the other says we should write in native IOS (swift) and Android and only have the engine in c++. The argument for c++ is it will be a single code base to maintain - it would include doing the sharing, storing history, displaying sonograms, etc. The argument against is that in fact it will not be easier to maintain, and will also create a lot of problems making calls, for example, between the interface and the sharing module.
Hope this is clear - it's a very hard thing for me to assess as ignorant as I am! Any advice would be greatly appreciated.

In my opinion, the way to go is to write the GUI the way it was meant to be. Native iOS/Android will yield a much better user experience and will allow you to use the platform latest and greatest way of implementing a UI.
Also, I am assuming that the engine will probably not be developped by the same people that develop the GUI itself, so the argument that it would be simpler to have a single code base doesn't really make sense.
Another solution could be to use Xamarin, so that you can consolidate iOS and Android development, while still having a common C++ engine.
Edit: Typo

Cross Mobile Options

I created an Android app. While creating one specific app was an interesting challenge, I'm now looking into creating a group of similar apps.
I'd like to create a group of similar Android apps and then move on to creating the same on tablets and iOS... (anything mobile).
I've considered doing so with a product called PhoneGap or doing a web based mobile app. Both of these options seem less than ideal. Doing the Android app I've been frustrated by Java's lack of control and low level constructs. Moving to something like a web based app seems like the exact wrong direction.
C++ is my language of choice. It has the ability to work at a low level, is highly portable across platforms, and has significant support for generic coding which would be useful for generating a group of similar apps. However, the Android documentation suggests to not use C++ unless your goal is porting existing code or dealing with computationally heavy tasks.
I'm leaning towards using C++ anyway, but are there other options I've not considered?
Thanks

You could in theory write your logic in C++ and then have UI layers on top that make use of it. If you are really comfortable with C++ that might be the way to go.
Almost any other parts (networking, UI, animation, etc) are better off being done in the native language of the platform. Use of cross platform solutions always limits you in some way, and usually leads to an application that is not as good as it could be for any platform.

Well, Google's recommendation to not use C++ is based on the following, I believe. C++ is low level, so you can get extra performance out of it if you know what you are doing. Google makes the reasonable assumption that many programmers do not. It is easier for an inexperienced programmer to do harm in C++ then to get a performance boost.
But, if you know what you are doing, it can help you. UI elements on both iOS and Android are implemented in their main language (obj-c, and Java respectively) so there is not a great way around that, but you can write core logic and other functions in C++ and it will be portable between them (iOS can use C++ directly and Android can use it via the Native Development Kit).
There are a few other options available. The one I ended up using is Appcelerator Titanium but please stay away from it. If your project gets complicated or large at all you will hate yourself for choosing it, as I did. Another interesting one that uses C++ instead of Javascript is Marmalade. I haven't used it though, so I can't comment on it.
A non-free solution that I hear good things about is Xamarin, who have ported both environments to C# and a .NET using Mono. However, you still have to write two versions of your code for the UI as far as I can tell.

Speeding up the porting of an iPad application to Android

My work consists of porting an iPad application to Android (so from Objective-C to Java). I've developed in the Android environment, but never in iOS. Currently I am looking for advice -- I'd like to know if is there a way, method, or process which can help me to do this more easily.

At this stage generally the answer is no, but I've got some suggestions that might help.
Use UIWebView/Webkit extensively - baring any HTML5 media (and SVG), there's little porting required when you render HTML. What you render in one, generally, renders well on the other.
If your iOS app is a basic show-this-edit-that style of app, you may find PhoneGap, jQuery Mobile, Titanium or the new Adobe suite a better time investment.
Because Android lacks a consistent device base, which makes developing animation-rich UIs difficult, you might find that there is no 1-1 UI comparison. Instead think about the features you offer, and their underlying data and view models.
You should be able to create a similar UIViewController/Activities structure although the tying logic behind the scenes will be platform specific. Map this out on some paper - it makes a really good what-the-hell-do-i-do-next plan for your UI skeleton.
Prefer an intermediate abstraction between your 'in data' and 'out data' so you can exchange parsers/kits/apis/frameworks without hacking everything to bits.
Where you have custom draw routines and graphics, ensure that you've got filler gradients or colours. Android uses a box model to support the many many screen sizes - iOS only has 4 resolutions to worry about. Recreating the same look might take too much time - contrasting pastel colours are a good placeholder until you can justify making the artwork.
There's not a lot more that can say that hasn't already been said before.
Hope this helps!

Few advices:
You will in most cases need to do redesign of UI as iPad, being tablet PC and having a larger screen as compared with most Android devices. Advice: Check if there is a version of the application dedicated for iPhone as it could be very helpful during redesign stage.
Check if iPad code contains beside Objective-C code also C/C++ code. If there is significant amount of code you might consider using Android NDK which allows C/C++ code to be used together with Java.
If the iPad application was using one of the popular cross platform frameworks (e.g. PhoneGap) that are based on HTML5/CSS/JavaScript combination you might be able to reuse most of the code. Both iPad and Android browser/ui elements responsible for rendering HTML5 are based on same engine (WebKit) and generally generate similar experience. Note: Some frameworks allow custom access to native functionalities which will require rewrite in almost all cases.
Do not try to create exact same UI on Android if it conflicts with user interface guidelines. As you probably know the guidelines are available at: http://developer.android.com/guide/practices/ui_guidelines/index.html
Android users are used to certain interface and they should be getting similar experience as other apps available for Android. Also it might be more than painful to create similar controls for Android that exist on iPad. If there is pressure to make same "look and feel" here is a post that explains how to defend from such pressures: Porting iPhone applications to Android? How to convince them NOT to
Numerous features existing on iPad will not be available on Android (e.g. Android application splash screen using just an image). So analyse the iPad application and decompose it to set of functionalities. Once you have them then you can map them to Android API and start coding.
There are no Objective-C to Java converters at the moment (this might change in the future) but for general feeling what are differences between Objective-C and Java you might want to read "Porting Objective-C to Java" by Theresa Ray of Tensor Information Systems Inc