I use ffmpeg to play video stream on SurfaceView of Android project. Now I would like to implement following feature.
1) Select one object by drawing a red rectangle on the SurfaceView.
2) Send x, y, width, height of the selected object and the original video frame to opencv.
3) Then, opencv return the new x and y of the object by processing the new video frame.
Anybody did it before? I will be very nice of you to give me some suggestion, or tell me very I can download the demo source code. Thank you so much.
For part (1), try searching Google a little more. It won't be hard to find a tutorial that uses touch input, a tutorial to draw a rectangle, and a tutorial to draw over the SurfaceView. Part (2) is done just by how you set up and define your variables - there isn't a specific mechanism or function that "sends" the data over.
Part (3) is the part that isn't obvious, so that's the part I'll focus on. As with most problems in computer vision, you can solve object tracking in many ways. In no particular order, what comes to mind includes:
Optical Flow - Python openCV examples are here
Lucas-Kanade - the algorithm compares extracted features frame-by-frame. [The features are Shi-Tomasi by default, but can also be BRIEF, ORB, SIFT/SURF, or any others.] This runs quickly enough if the number of features is reasonable [within an order of magnitude of 100].
Dense [Farneback] - the algorithm compares consecutive frames and produces a dense vector field of motion direction and magnitude.
Direct Image Registration - if the motion between frames is small [about 5-15% of the camera's field of view], there are functions that can map the previous image to the current image quickly and efficiently. However, this feature is not in the vanilla OpenCV package - you need to download and compile the contrib modules, and use the Android NDK. If you're a beginner with Java and C++, I don't recommend using it.
Template Matching [example] - useful and cheap if the object and camera orientations do not change much.
Related
My requirement is to scan a fixed object. After recognizing that, I want to highlight the object and to display corresponding pre-feeded parameters accordingly, like height, width, circumference, etc.
This all, I want to do, without internet, using camera only.
Please, let me know if any solution / suggestion for this.
I have seen CraftAR SDK. It seems working as per my requirement, in order to recognize object, but it uses its server for storing images, which I don't want. As I want the static image, to be stored in app itself.
Try using the TensorFlow Object Detection API. Link: TensorFlow Object Detection API
And you can then customize your overall app behaviour accordingly, managing all your requirements (like for eg. showing a pop up with all the details of the object that's being detected after receiving some kind of callback when using the Tensoflow Object Detection API after the object has been detected successfully) as well as I believe that you can customise the TensorFlow object detection scenario part as per your need (Here, I am talking about the UI related part specifically in case of how you want your app to detect the object graphically).
To answer about the details like how it works offline and the resulting overall APK size etc. can be better understood from the links given below:
1] Step by Step TensorFlow Object Detection API Tutorial — Part 1: Selecting a Model
2] How to train your own Object Detector with TensorFlow’s Object Detector API
As an overview, for detecting the objects offline you have to be limited (just to reduce your APK size) with your own set of data/objects (as you have mentioned that you have got a fixed object for detection, that's good) and then you have to train (can be trained locally as well as on cloud) this set of objects using a SSD-Mobilenet model. Then you will have your own trained model (in simpler words) of those set of objects which will give you a retrained_graph.pb file (this goes into your assets folder for your android project) which is the final outcome that includes the trick (in simpler words) to detect and classify the camera frames in real time thereby displaying the results (or object details) as per the info (or the set of data/objects) provided; without the need of any sort of internet connection. For instance, TF Detect can track objects (from 80 categories) in the camera preview in real-time.
For further reference follow these links:
1] Google Inception Model
2] Tensorflow Object Detection API Models
3] Speed/Accuracy Trade-offs for Modern Convolutional Object Detectors
Also you can optimize (or compress) the retrained_graph.pb to optimized_graph.pb as this is the only major thing that would increase your APK file size. Long ago, when I tried detecting 5 different objects (using TF Classify), each object's folder was having about 650 photographs and the overall size of all the 5 folders (together) was about 230 mb and my retrained_graph.pb size was only 5.5 mb (which can further be optimized to optimized_graph.pb reducing its size even more).
For to start learning it from the beginner's level I would suggest you to once go through these codelab links and understand the working of these two projects as I too did so.
1] TensorFlow For Poets
2] TensorFlow For Poets 2: Optimize for Mobile
Wishing you good luck.
The below link to TensorFlow GitHub (Master) includes almost everything:
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android
I am developing an app that captures a business card using custom android camera and then i need to autocrop the unwanted space in android and then store the image . I am using opencv for this. All examples i am seeing are in python . I need it in android native.
You can probably try something like this:
1) Get an edge map of the image (perform edge detection)
2) Find contours on the edge map. The outermost contour should correspond to the boundaries of your business card. (under assumption that the business card image is against a solid background) This will help you extract the business card from the image.
3) Once extracted you can store the image separately without the unwanted space.
OpenCV will help you with points 1,2 and 3. Use something like a cannyedge detection for point 1. The findContours function will come in handy for point 2. Point 3 is basic image manipulation which I guess you don't need help with.
This might not be the most precise answer out there - but neither is the question is very precise - so, i guess it is alright.
I am new to android and ARToolkit.I have to develop the android application which can augment and render the 3D models from CT scan images in DICOM format on the detected marker. I am using ARToolkit SDK for my purpose. But don't how to proceed with the dicom files and render the 3D model on marker. Someone please suggest some approach. Any sort of help will be highly appreciated.
Thanks
I recommend the following process;
Figure out a tool for segmentation. This is the process whereby you will build a 3d model of subset of the data depending on density. For example, you will build a model of the ribs of a chest CT. You should do this outside of Android and then figure out how to move it later. You can use tools like ITK and VTK to learn how to do this stage.
If you want to avoid the ITK/VTK learning curve, use GDCM (grass roots dicom) to learn how to load a DICOM series. With this approach you can have a 3D array of density points in your app in a few hours. At this point you can forget about the DICOM and just work on the numbers. You still have the segmentation problem.
You can look at the NIH app ImageVis3D which has source code and see what there approach is.
Once you have a segmented dataset, conversion to a standard format is not too hard and you will be on your way.
What is the 'detected marker' you refer to? If you have a marker in the image set to aid in segmentation, you can work on detection from the 3d dataset you get back from loading the dicom data.
Once you have the processes worked out, you can then see how to apply it all to Android.
It seems a little old but, recommended for a start: Android OpenGL .OBJ file loader
I was wondering too about building a CustomView to address your needs, since in a CV you can display anything.
I'm creating an Android app that makes use of OpenCV to implement augmented reality. One of the needed features is that it saves the processed video. I can't seem to find any sample code on real-time saving while using OpenCV.
If the above scenario isn't possible, another option is to save the video first and have it post-processed by OpenCV and saved back as a new file. But I can't find any sample code for this either.
Could someone be kind enough to point me to either direction, or give me an alternative? It's ok if the alternative doesn't use OpenCV.
Typical opencv flow is, you receive frames from camera, convert to RGB format, perform matrix operations then return to activity to display in View. You can actually store the modified frames as images somewhere in sdcard and use jcodec to create your mp4 out of your images. See Android make animated video from list of images.
I'm planning to write an app for Android which performs a simple cell counting. The method I'm planning to use is a type of Blob analysis.
The steps of my procedure would be;
Histographing to identify the threshold values to perform the thresholding.
Thresholding to create a binary image where cells are white and the background is black.
Filtering to remove noise and excess particles.
Particle (blob) analysis to count cells.
I got this sequence from this site where functions from the software IMAQ Vision are used to perform those steps.
I'm aware that on Android I can use OpenCV's similar functions to replicate the above procedure. But I would like to know whether I'd be able to implement histographing, thresholding and Blob analysis myself writing the required algorithms without calling API functions. Is that possible? And how hard would it be?
It is possible. From a PNG image (e.g. from disk or camera), you can generate a Bitmap object. The Bitmap gives you direct access to the pixel color values. You can also create new Bitmap objects based on raw data.
Then it is up to you to implement the algorithms. Creating a histogram and thresholding should be easy, filtering and blob analysis more difficult. It depends on your exposure to algorithms and data structures, however a hands-on approach is not bad either.
Just make sure to downscale large images (Bitmap can do that too). This saves memory (which can be critical on Android) and gives better results.