Questions are first then some description which helps explain the questions:
Questions:
Will Unity be able to easily import downloaded assets from our API if we save them in a specified directory with our core Android app?
Can we pre-serialize the 3d model assets before uploading them to our API so that they do not need to be serialized by Unity on device? This is so that they load faster
Description
I'm not a Unity/3D developer or an Android developer so apologies if this question doesn't meet community guidelines (I built the API in this scenario). This is also probably very long for what should be simple questions.
We've built an API which serves 3D model files in Collada 1.4 (.dae) format to iOS and Android client applications. File sizes can be quite large (upwards of 80-90mb in some instances) although the majority are in the 4-15mb region.
On iOS we've been having the application download the models from the API and then render them onto the screen via SceneKit which has been relatively simple to achieve. We put the Collada files through a preprocessing step before serving them which converts them into a proprietary format used by SceneKit.
In Android we've encountered any number of problems. We initially experimented with converting models to OBJ instead of Collada and using Rajawali, jPCT and a few other loaders for Open GL ES2.0 but due to our file sizes the time to read and load the models was far too long. We've now decided Unity is probably where we need to go on Android as a rendering engine.
The Android app will consist of two parts, a core app which has an interface for viewing images of the models and downloading the model files and a Unity app which is loaded via an activity from the core Android app to actually render the models.
Related
I've made an AR app (proof of concept), which is bundled with a specific 3d-model at the moment.
Next step is to be able to select between a selection of 3d-models, stored on the users local storage on their phone/tablet, so user can select which 3d-model he wants to see.
I'm fairly new to Unity, and my limitations comes quickly, so i'm wondering if this is even possible, and if you guys have any resources or guide on how to do this ?
You can use this kind of plugin UnityNativeFilePicker to easily access user's local files in iOS and Android.
Then to import your 3D models into Unity only FBX, OBJ, DAE and DXF are supported (Unity 3D formats) by default, but you can find really good runtime importers such as glTFast to import glTF files. It only depends on which file format you want to support in your app.
I am using Tensorflow for poet guide for train own model. I have create retrained_graph.pb and retrained_labels.txt. While I use it in application then I get error that
Caused by: java.lang.UnsupportedOperationException: Op BatchNormWithGlobalNormalization is not available in GraphDef version 21. It has been removed in version 9. Use tf.nn.batch_normalization(). at org.tensorflow.Graph.importGraphDef(Native Method) at org.tensorflow.Graph.importGraphDef(Graph.java:118)
After That further train model for application use Tensorflow for mobile blog and create optimized_graph.pb, rounded_graph.pb, mmapped_graph.pb files.
optimized_graph.pb and rounded_graph.pb file work in android application without any error.
While use mmapped_graph.pb I get error that Failed to initialize: java.io.IOException: Not a valid TensorFlow Graph serialization: Invalid GraphDef
Performance of application is not good while use optimized_graph.pb and rounded_graph.pb file.While application camera screen not contain any flower photos otherwise random flower name show with high confidence rate. Any way to detect only flower and remain blank when not flowers.
Screenshot
Performance of the application is very good, and is realy fast on the gpu of a mobile phone. The problem is how you build your own model. In fact the tensorflow graph of this application is built to recognize images based on the classes you give to it. In other words if, for example, you teach to the model to recognize 4 different classes of images, it try to label everything it see in this 4 classes.
For this reason you have wrong results when camera screen don't contain flowers.
A way to "solve" this problem is add an extra class with random images, that probably will have high confidence with no-flowera photos.
If you want a more strict model, you have to use a completely different algorithm.
However keep in mind that which used in the application is probably the state of the art of computer images recognition
I have been working on the ARToolkit sdk for android since some time.
In the ARToolkit SDK, I have worked on ARBaseLib and ARSimpleNativeCarsProj and implemented successfully. But I am trying to add external 3d Objects(.obj and .mtl) and I am unable to render the new object files.
I have also looked into the source code provided in this link,
https://github.com/kosiara/artoolkit-android-studio-example
but the problem here is the 3D object(Cube) has been created in using the draw(), openGL libraries function, instead i would want to add an external 3D Object.
More Explanation:
Okey, SimpleNativeCarsProj comes along with two 3D Objects(.OBJ and .MTL) in the assets/Data folder. CASE1 I tried replacing the existing 3Dobject with another 3D Objects, App Crashes on the launch CASE2 As I worked around a little, these files are pushed to the cache folder on the app load, I invalidated the caches and restarted android studio, rebuilt and ran the app, still the app crashes on the launch. Technically I am unable to Replace/Delete/Add another 3D Object files to SimpleNativeCarsProject
Any headsup would be appreciated.
Convert your FBX files
The Encoder works wit FBX (.fbx) files. We recommend using FBX wherever possible as tools support for FBX is widely available.
http://www.wikitude.com/products/wikitude-sdk-features/wikitude-3d-encoder/
Give ArToolKitJpctBaseLib a try. It is a wrapper on ArToolKit + jPCT-AE (a 3d Engine on Android) aims to simplify creation of AR apps for Android.
It's possible that we start an android project in which it's necessary after recognising an image by camera to visualise a content generated in unity.
The easy part would be to use webGL to display it, but there is the problem of devices that do not support it directly. My question is if from android (and later iOs). It's possible to download a content of unity, load and visualise it in runtime?.
Is possible that I had to direct all the effort to generate that content in a .jar and then use something like dependency injection to load it?
I have already a unity scene in an activity but of course in project definition not in run time.
Any help or guidance would be welcome.
Unity builds levels into the final runtime executable, so adding a downloaded 'scene' directly is not possible. The best way around this is to create a 'generator' scene which can accept input from a downloaded text file, such as JSON, and use that to render the level.
However, this method does assume that all the possible objects that can be rendered are in your game as prefabs. If you're wanting to pull images from the net to be loaded into textures, the WWW class might get you started down the right path:
https://docs.unity3d.com/ScriptReference/WWW.LoadImageIntoTexture.html
I'm developing an offline routing application for android.
I'm working with osmdroid library. and I downloaded the pbf file for the region where I want to do the offline navigation but I'm not sure how to be able to convert the binary representation of the maps in the pbf files into a normal text.
You don't have to use the PBF format, at the moment the OSM XML format is still widely in use. But if you do, then you will probably have to implement a PBF reader for Android. See osmosis which serves as the reference implementation for PBF and can also convert between those two file formats.
You can take a look at the Android page in the OSM wiki where lots of OSM-related Android application are listed. Maybe there is already an Android application supporting PBF. But most of them will just display tiles and the rather popular OsmAnd uses its own file format instead.
Note that PBF has various advantages compared to XML. It is smaller and it is faster to read and write than compressed XML. It also supports random access which can become very handy for a routing engine.