I am trying to create an OCR app for android. I want to be able to do it on the device itself rather than sending it to a server and then see the results. Is there a library available for it? I can consider buying as well.
Thanks.
While I know zilch about Android development, what I could offer you is to explore the option of porting 'tesseract'. Have used it a little, and few other FOSS OCR tools (some actually based on tesseract), and have found it to be best of the FOSS lot. AFAIR, it does only basic recognition, and you need to massage the document prior to feeding it to tesseract. Where commercial s/w surpasses FOSS tools, is the parapharnalia of preprocessing/massaging they are able to do, and even some postprocessing -- guessing tables and fitting text/unrecognized graphics back into positionally accurate locations on page, and converting into other document formats.
Tesseract can be tried on a regular Linux desktop on your favourite distro. IIRC, there is even a MS-Windows version as well, i.e. if you care.
Related
What I have: A trained recurrent neural network in Tensorflow.
What I want: A mobile application that can run this network as fast as possible (inference mode only, no training).
I believe there are multiple ways how I can accomplish my goal, but I would like you feedback/corrections and additions because I have never done this before.
Tensorflow Lite. Pro: Straight forward, available on Android and iOS. Contra: Probably not the fastest method, right?
TensorRT. Pro: Very fast + I can write custom C code to make it faster. Contra: Used for Nvidia devices so no easy way to run on Android and iOS, right?
Custom Code + Libraries like openBLAS. Pro: Probably very fast and possibility to link to it on Android on iOS (if I am not mistaken). Contra: Is there much use for recurrent neural networks? Does it really work well on Android + iOS?
Re-implement Everything. I could also rewrite the whole computation in C/C++ which shouldn't be too hard with recurrent neural networks. Pro: Probably the fastest method because I can optimize everything. Contra: Will take a long time and if the network changes I have to update my code as well (although I am willing to do it this way if it really is the fastest). Also, how fast can I make calls to libraries (C/C++) on Android? Am I limited by the Java interfaces?
Some details about the mobile application. The application will take a sound recording of the user, do some processing (like Speech2Text) and output the text. I do not want to find a solution that is "fast enough", but the fastest option because this will happen over very large sound files. So almost every speed improvement counts. Do you have any advice, how I should approach this problem?
Last question: If I try to hire somebody to help me out, should I look for an Android/iOS-, Embedded- or Tensorflow- type of person?
1. TensorflowLite
Pro: it uses GPU optimizations on Android; fairly easy to incorporate into Swift/Objective-C app, and very easy into Java/Android (just adding one line in gradle.build); You can transform TF model to CoreML
Cons: if you use C++ library - you will have some issues adding TFLite as a library to your Android/Java-JNI (there is no native way to build such library without JNI); No GPU support on iOS (community works on MPS integration tho)
Also here is reference to TFLite speech-to-text demo app, it could be useful.
2. TensorRT
It uses TensorRT uses cuDNN which uses CUDA library. There is CUDA for Android, not sure if it supports the whole functionality.
3. Custom code + Libraries
I would recommend you to use Android NNet library and CoreML; in case you need to go deeper - you can use Eigen library for linear algebra. However, writing your own custom code is not beneficial in the long term, you would need to support/test/improve it - which is a huge deal, more important than performance.
Re-implement Everything
This option is very similar to the previous one, implementing your own RNN(LSTM) should be fine, as soon as you know what you are doing, just use one of the linear algebra libraries (e.g. Eigen).
The overall recommendation would be to:**
try to do it server side: use some lossy compression and serverside
speech2text;
try using Tensorflow Lite; measure performance, find bottlenecks, try to optimize
if some parts of TFLite would be too slow - reimplement them in custom operations; (and make PR to the Tensorflow)
if bottlenecks are on the hardware level - goto 1st suggestion
Maybe you should try this lib, it can run on android and ios devices.
https://github.com/Tencent/TNN
My Situation
I want to build an Application that can recognize an Image to produce a corresponding model.
i.e. I focus the camera to show a printed image on the card that is designed by myself ( apple logo ) , then it will show a 3D model(.md2) on the screen which is also designed by myself.
I have googled many framework that worked on both Android & iOS, but the documentations are very limited and the trial version does not support me to test it.
for example,
http://www.metaio.com/sdk/
But their demo is not comprehensive enough to suite my situation
My Question
1.Would anyone can share their experience of developing with AR framework (not the AR core) on Android & iOS?
2.Is there any framework that support me to add a image as a key then it will map to my model with just a couples line of codes?
3.if Q2 is not possible, is there any approaches of some framework can also archive the some goal but more complex ?
//Logic flow
String key = "APPLE";
sdk.putKeyImage(key,apple.png);
...
if (sdk.identifiedAs(key)){
//Do something
//Example
sdk.showApple3DModel();
play(showSnakeEatApple.mp4);
}
I'm an android app developer. So according to my experince for android, you can go for
Vuforia (https://developer.vuforia.com/resources/sdk/android)
Wikitude (http://www.wikitude.com/developer/documentation/android)
These above 2 are enough good to implement AR app in android.
Vuforia is having well documented, also they have more libraries & mainly those are free. & Wikitude is best for making apps faster. I recommend you to use any of these for your AR app development in android.
I guess you should compare several AR frameworks to find a proper solution for your problem. Many different AR tools come to my mind. For example, Vuforia, LayAR, Kudan AR. To take a deeper look at the variety of possible options read the comprehensive comparison of the most popular frameworks: http://cases.azoft.com/top-5-tools-creating-augmented-reality-apps/
yes i did work on it not much but little bit i did research on augmented reality and after research i found one open source and cross platform well known sdk its name is
VUFORIA SDK
there are various SDK you find on google but trust me VUFORIA is best and flexible who gives lots of option
for that you need to install NDK in eclipse cause it have core cpp code who compile by NDK in eclipse
video tutorial click here
sdk and documentation click here
steps click here
hope it will be helpful for you thank you happy coding
I want to know more about Cross-Platform. I'm currently working on my thesis and decided to make an Android Application but not all target users uses Android device. So, We've come up with an idea to Cross-Platform the android application.
I have read the same topic but it didn't get to me too well. iOS / Android cross platform development. I want to know if there is another way than using the frameworks discussed in the link.
I want know what is the best/shortest way to cross platform a Android Application?
Cross Platform tools are in my opinion not a good idea at all. Android is Java, and iOS is Objective C. Android can use NDK to use a C/C++ library so theoretically if you did most of your code in C/C+ it could be used on both sides. I think two natives is more in order. Study NDK to see what could run on both devices. Lets say you built your app as a C++ Library except for UI interactions. Then in theory the library should work on both sides. C/C++ a lot easier on iOS than android however.
Also it really depends on the dependencies your app will have. Lets say you want to use a library but its only available in Java or C++ how does this effect your decision.
You might also consider designing your app so that it just captures data on the devices and the actual processing of the data takes place in a web service.
Another way is use Mobile Web HTML5 Framework like Phonegapp, Sencha touch and more.
Although web technologies don’t perform as well as native, but it is a useful thing that you can try.
See Choosing Mobile Web HTML5 Framework and http://operationproject.com/2011/adventures-in-html5-part-one/
Android application are apk files that are basically zip file that contains dex/odex files and all other resources that you may wish to add.But I think you can use any coding language to develop as long as you have IDE that converts your code to apk and dex .And most important of all your programming language must support corresponding api or must have something similar to cross compile.The reason Google used Java for Android was they felt it is convenient to do it(read it during case Google v/s Oracle)
Personally I feel cross-platform is good since it increases the developers base and understanding of the System (here Android).And I think the same analogy would go with other Systems as well.
As the topic states - is there any technology that would let me write code for a simple commercial 2D game which could be then compiled to work well on Android, iOS and Web browsers or at least two of them (e.g. Android and iOS)? There is so much buzz about cross-platform nowadays that I was wondering if I'm missing something.
My goal (set by my producer) is to make this game for all of these three platforms, and so far I planned to use:
- Web browsers: Adobe Flex,
- Android phones: Java (since my favourite C++ is recommended only for short snippets),
- iPad/iPhone/iPod: Objective-C (are there any alternatives?).
It pretty much means re-writing the same source code three times. Is there any way to speed it up by using a technology that would work on at least both of them?
However, I don't want to use Adobe Flex for all three (though it's possible), because it requires Adobe AIR (works only with Android 2.2 and higher) and not a lot of phones have it (it's totally different with browsers where 98% of people have Flash).
I don't mind adding some platform-specific code for each version, of course.
Also, if you have any suggestions about choosing a fancier technology for any of the platforms (maybe some nice frameworks?), I'd really appreciate that, too.
Take a look at Corona and Moai both are Lua-based frameworks for game development targeting multiple platforms.
I would like to say that Multimedia Fusion was capable of doing this, but it isn't quite ready yet. There are currently a number of runtime exporters for the product which allow you to export your game to multiple target platforms which include flash, java and iOS at the moment. XNA and Android exporters are currently in the works. I should probably mention that I am only affiliated with these guys as a happy consumer of their products. It is probably worth checking it out even if they can't do everything you want yet. If it's not suitable for your needs it is still a very good choice if you wish to rapidly prototype your application.
See:
http://www.clickteam.com
On a side note, personally I would recommend against going down the js+html5 route, particularly if you are targeting mobiles. I should probably stress that it's simply an opinion I have and that there are plenty of people here who will disagree with me. In all javascript games I have played, I have felt the garbage collector. The issue is that without control over garbage collection it is free to run when it feels like, causing infrequent but very obvious pauses in the middle of your game. You can work around this but I don't see why it should be that difficult. I can definitely align with the ideals of being able to write the thing once and run it on all platforms, but I can't see it working just yet.
To offer some advice assuming you find no suitable tools for the job, your aim is to maximise reusability. This means all of your games resources and levels should be well defined as independent resources. For non-critical logic you can make use of embeddable scripting languages such as Lua, which will maximise your code reuse. C (or C++) code will also be reusable on a lot of platforms, and might be a good choice if you are familiar and confident with the language.
I'm personally using Phonegap and Zepto.js. But you may have more luck with jQuery mobile. Phonegap is nice because its website will compile the apps for you.
1) If you write your game in C++ you can use it for iPhone & Android (NDK) as well. Preferably, you write your game engine in C++, than you can use a OS specific layer for UI stuff..
2) Implement your game in HTML & Javascript. iPhone/Android users will have to use a webbrowser to play your game. For iPhone, you can even create a launcher, that looks like an app, but actually is a webview with HTML & Javascript
Unity3D should be able to do this for you, if you purchase some extra sprite addins to handle 2D (like Ex2D or SM2 or Toolkit 2D). Also right now until Apr 8, the iOS and Android license is free.
Game Maker Studio. It has a package that let's you deploy on all of the platforms you outlined.
http://www.yoyogames.com/gamemaker/studio
I've been working on a game engine runtime environment in C++ for my future games, and have started to consider android as a platform. I noticed that it was tightly bound to Java and uses Java VMs heavily.
But is it possible to sustain a full C++ runtime environment in Android NDK? I have nothing against Java and am prepared to use it if I have to, but performance is one of my prime concerns (I intend for my games to be resource-intensive), especially on phones.
And if a full C++ environment is possible, how exactly would I implement it in Eclipse Indigo CDT? Would I be able to create a compiled game executable for Android ahead of time for maximum performance? And would there be any additional plugins I'd need to install in Eclipse? Could I use MinGW for compiling my games, or would I need to use a different compiler? If I had to use Java in one way or another, would compilation of the C++ code even be required? These are all questions I'd like to answer to get a sturdy development environment going in the Eclipse IDE.
Please know that I'm still fairly new to Android development, and multiplatform programming in general. My goal is to create a game engine that'll take the most advantage of the new hardware out there, especially on phones!
Thanks for any advice you guys can give!
NativeActivity, added in Android 2.3 (API 9) might be what you're looking for in terms of writing Android games using only C++. http://developer.android.com/reference/android/app/NativeActivity.html
However since you say you're new to this you may want to start with one of the currently available Android game engines instead of writing your own right away. Mobile devices have a very different set of constraints than other platforms. A phone isn't very useful if the battery dies mid-afternoon, so starting a project with the stated goal of being "resource-intensive" is already getting off to a bad start. Few people will want to play your games for 20 minutes here and there if it means they won't be able to make a phone call in the evening.
If what you meant is that you are shooting for high-end graphics, keep in mind that devices have a wide range of capability in this area and targeting only the high end limits your audience. Different GPUs have very different strategies and performance characteristics and all have cases where they shine or lag behind. The most successful mobile games aren't the ones with the highest polycounts or the most complex lighting shaders, they're the games that achieve a consistently smooth framerate and have a distinctive style.
Have a look at some of the existing game engines for Android and try them out. Write a couple small games to take them for a test drive and see where they do and don't mesh with what you're trying to do. If you find yourself feeling limited with what's available, take what you've learned and try to write your own engine that fits with the types of games you want to make.
Here are some links to get you started. These engines power some very popular games on Android Market and many developers have found them useful:
http://www.andengine.org/ andengine is written in Java and is open source.
http://unity3d.com/ Unity is based on C# and Mono.
http://www.madewithmarmalade.com/ Marmalade, formerly Airplay SDK is based on C++.
The answer you want to pay close attention to is #adamp's. Ignore his answer at your own peril.
To address a couple of other points in your question, though:
Would I be able to create a compiled game executable for Android ahead of time for maximum performance?
No, insofar as Android does not use "compiled game executables". Your C/C++ code will be compiled into a Linux shared object (.so) file and packaged with other stuff (like your manifest) in an APK file. Compiling the .so will be ahead of time, though.
Could I use MinGW for compiling my games would I need to use a different compiler?
I get the distinct impression that you will claw your eyes out trying to do NDK development in Cygwin on Windows. I heartily encourage you to do your NDK development on OS X or Linux. At least set up a VirtualBox with Ubuntu or something in it to do your NDK compiles.
Regardless, the NDK comes with its own GCC compiler. AFAIK, MinGW is not a cross-compiler, and most Android devices are not running on x86 CPUs.
If I had to use Java in one way or another, would compilation of the C++ code even be required?
Yes. Now, for Honeycomb, you have the option of using Renderscript for OpenGL work -- that amounts to a C99 environment whose code you will not need to compile with the NDK.
I've tried a bunch of them, and one of the best I found that cross compiles instantly to a bunch of other devices and let's you program your goodies in HTML5/C++ combo (sweeeeet) is MoSync (http://www.mosync.com/) and another little known one that is a little more setup friendly is MotoDev by Motorola (http://developer.motorola.com/tools/motodevstudio/?utm_campaign=mhp01092012&utm_source=mhp&utm_medium=mws)
I've noticed there is some virtual machine images and tutorials you can use to prevent the setting up hassle.