I am trying to create a small Android app with reasonably simple AR functionality - load a few known markers and render known 2D/3D objects on top of the video stream when those are detected. I would appreciate any pointers to a library for doing this, or at least a decent example of doing it right.
Here are some leads I have looked into:
AndAR - https://code.google.com/p/andar/ - This starts out great, and the AndAR app works well enough to render one cube on a single pattern on a real-time video stream, but it looks like the project is effectively abandoned, and to extend it I'll have to go heavily into OpenGL land - not impossible, but very undesirable. The follow-up AndAR Model Viewer project, which supposedly lets you load custom .obj files, doesn't seem to recognize the marker at all. Once again, this looks very much abandonware, and it could have been so much more.
Processing - Previously mentioned NyARToolkit is great with Processing from a PC - example usage, which works perfectly for the 'here's a pattern, here's an object, just render it there' functionality, but then it all breaks down for Android - GStreamer for Android is at a very very early hacky stage, and in general video functionality seems to be a rather low priority for the Android Processing project - right now import processing.video.*; just fails.
layar, wikitude etc, they all seem to focus more on interactivity, location and whatnot, which I absolutely don't need, and are somehow missing this basic usage.
Where am I going wrong? I would be happy to code some part of the video capture/detection/rendering, I don't need a drag-and-drop library, but the sample code from AndAR just fills me with dread
I suggest to take a look at the Vuforia SDK (formerly QCAR) by Qualcomm plus jPCT-AE as 3D-Engine. They both work very well together, no pure OpenGL needed. However you need some C/C++ knowledge, since Vuforia relies to the NDK to some degree.
It basically boils to to get the marker pose from Vuforia via a simple JNI-function (the SDK contains fully functional and extensive sample code) and use that to place the 3D-objects with jPCT (the easiest way is to set the pose as the rotation matrix of the object, which is a bit hacky, but produces quick results).
jPCT-AE supports 3D-model loading for some common formats. The API docs are good, but you may need to consult the forums for sample code.
Related
I am a complete beginner with ARCore and I would like to ask if it is possible to display 3D objects using 3D coordinates and having the user as the origin? For example, having (x,y,z) values of (0,3,0) will display the 3D object to the right of the user.
First of all: There is no native implementation in ARCore for that!.
But you're not completely lost. There are some very useful libraries in Android like appoly. I've used this fork from Eric Neidhardt.
For iOS I only know this library based on ARKit.
You can also make a Unity3D app in C# and use AR Foundation and Vuforia Plug-in. This way is platform-independent, but imho testing is a pain.
You see, there are a few options out there, but be careful that none is really accurate!
So, for a personal project, I'm trying to make an app that helps newcomers get used to the school quickly. And for one section of it, I'm trying to build a 3D Structure, in a shape of my school campus, with lables incidating where the rooms are, so when the using searches for specific rooms, they can easily find it. So, It would be like from one activity, when I click a button, it opens a new activity with a search bar on the top, and the rest of the screen a 3d image people can navigate through. What's a way I can accomplish this?
I know that the structure part can be accomplished with Unity, which I haven't learned but have an idea of what it is, but I don't know how to build the rest of the app with Unity.. So..
You probably want to start with designing the model (structure) in some-kind of modeling program, Maya, 3DS Max, Cinema 4D, Blender, etc... (Personally I recommend using Blender since it's free and open source). Then you would want to export it to some 3D model format, I recommend using .obj since it is very human-readable and easy to parse on your own, but you can use some other format and parse it using a library.
Now comes the more difficult part and it's to render the loaded model onto the screen. This can be accomplished with OpenGL ES or a more highlevel library which can abstract it for you. If you're not familiar with OpenGL ES or OpenGL in general, I wouldn't recommend using it since it may take some time to learn. There are few libraries out there which can render 3D models pretty easily. Now you should probably create some-kind of a controllable camera as well so you can use it to navigate through the model.
This maybe looks like a lot of work but if you already familiar with computer 3D graphics it can ease it up for you.
Well, you can make a 3d model of your school using Blender (which is a free tool, but will almost certainly be a lot of work). Export the model as a mesh which you can insert into your app as an asset or resource. Read the resource and present it using (probably) opengl. This is a SUPER high level view, in actual fact there's quite a lot of work in each stage. Some third party libraries can make it a little easier, but it will still be a big job. I personally like libgdx for abstracting the pain of opengl, but that's just my personal opinion.
It's a big job, but would be super cool .. good luck!
I recommend starting off with building the structure in a program like Blender, Maja or 3ds Max. You can export your projects from those programs to pretty much any game engine and it makes more sense to build things in one of those programs.
Once you're done with that you can choose which engine you want to use to add the search function, if unity or Unreal or whatever but either way you're going to need to make yourself familiar with java or c to add the search function.
I recommend to use blueprints of building if available, it will save you a loads of nerves. Use it as a material texture and place it on a plane polygon to get a good lining helper.
If you do use Unity for development then try Maya for polygonal modeling. At least they are looking similar to each other whereas 3DSMax is always a mind f*cking experience.
For my final year project at university, I am extending an application called Rviz for Android. This is an application for Android tablets that uses ROS (robot operating system) to display information coming from robots. The main intent of the project is essentially the opposite of traditional augmented reality - instead of projecting something digital onto a view of the real world, I am projecting a view of the real world, coming from the tablet's camera, onto a digital view of the world (an abstract map). The intended purpose of the application is that the view of the real world should move on the map as the tablet moves around.
To move the view of the camera feed on screen, I am using the tablet's accelerometer and calculating distance travelled. This is inherently flawed, and as such, the movement is far from accurate (this itself doesn't matter that much - it's great material for my report). To improve the movement of the camera feed, I wish to use markers placed at predefined positions in the real world, with the intent that, if a marker is detected, the view jumps to the position of the marker. Unfortunately, while there are many SDKs out there that deal with marker detection (such as the Qualcomm SDK), they are all geared towards proper augmented reality (that is to say, overlaying something on top of a marker).
So far, the only two frameworks I have identified that could be somewhat useful are OpenCV (which looks very promising indeed, though I'm not very experienced with C++) and AndAR, which again seems very focused on traditional AR uses, but I might be able to modify. Would either of these frameworks be appropriate here? Is there any other way I could implement a solution?
If it helps at all, this is the source code of the application I am extending, and this is the source code for ROS on Android (the code which uses the camera is in the "android_gingerbread_mr1" folder. I can also provide a link to my extensions to Rviz, if that would also help. Thank you very much!
Edit: the main issue I'm having at the moment is trying to integrate the two separate classes which access the camera (JavaCameraView in OpenCV, and CameraPreviewView in ROS). They both need to be active at the same time, but they do different things. I'm sure I can combine them. As previously mentioned, I'll link to/upload the classes in question if needed.
Have a look at the section about Template Matching in the OpenCV documentation. This thread may also be useful.
So I've managed to find a solution to my problem, and it's completely different from what I thought it would be. All of the image processing is offloaded onto a computer, and is performed by a ROS node. This node uses a library called ArUco to detect markers. The markers are generated by a separate program provided with the library, and each has its own unique ID.
When a marker is detected, a ROS message is published containing the marker's ID. The app on the tablet receives the message, and moves the real-world view according to which marker it receives. It works pretty well, though it's a bit unreliable, because I have to use a low image quality to make rendering and transport of the image quicker. And that's my solution! I may post my source code here once the project is completely finished.
I'm looking for a 'basic' AR SDK that allows me to draw images and 3D shapes around the user (no matter where he is). It would be even better if the SDK includes a simple way to detect interaction with the shapes (something like onClick).
I made a project from scratch on Android but there's still a lot of work to do and I'll need to do the same on iOS after... So that's why I'm looking for an SDK or a similar project (no matter what platform).
I tested Metaio but it's quite expensive and maybe overkill for my purpose because it uses LLA coordinates.
I tested DroidAR on Android but it's only for Android and it looks heavy too (don't need the GPS).
How about Qualcomm's Vuforia? I was able to quickly get a sample project running on it.
EDIT Looks like I was wrong about what it could do. According to this (which is slightly dated, so who knows) Metaio might be your only choice.
i really don't sure what you really want to do ..but if you simply show images or 3d models on camera without any detection you can achieve this very easily i am explaining for Android and you can extend it to ios on same logic.
first approach:
you have to use custom camera of Android in your app,then use any game engine as per your need..i will suggest Jpct-ae or Rajawali
they are very simple to integrate and can be used for 2d images and 3d models.
this tutorial will explains a lot
keep the gl-surafce transparent and you can have model floating in space ...
second approach :
to add some more effect to your AR app you can use sensor values to move model in 3d space as per movement of device..it gives a cool effect.
use first approach and additionally collect sensor values and apply that matrix to gl camera of your game engine..for sensor values follow here
good tutorial here..
i hope this may help you..i done these a long time ago but try to help if you want..
Is there an android Framework that can be used in an app to recognize a 3D image and send the user to a video. This should fall under augmented reality, but so far everything I have viewed uses 2D image and stuff to produce a 3D image on the screen... My situation is backwards from that. I tried using vuforia but I couldn't get the sdk to work, and unity needs an android license. DroidAr doesn't seem to fit the bill either. Or are there any tutorials on this matter? Thanks.
I have not used the feature, but Metaio has a 3D object, "markerless" tracking feature as well as the ability to do video playback within the SDK. I am sure if you would rather simply redirect to a video (YouTube) or something this would not be exceptionally difficult.
http://www.metaio.com/software/mobile-sdk/features/
Metaio's mobile SDK is similar to Vuforia, so if you had trouble with that you might have difficulty getting it up and running. If your programming skills aren't up to that, you might consider looking into Junaio, an AR browser made by Metaio. With Junaio you simply create a content channel rather than having to build the app from scratch. Again, I have not actually tried this feature yet but the documentation seems to indicate that 3D tracking is available in Juniao:
http://www.junaio.com/develop/quickstart/3d-tracking-and-junaio/
Good luck!