I'm trying to develop a Face Recognition app on Android and since I don't want to use NDK on the project (simply don't have the time to switch), I'm sticking to develop the whole app with Java and therefor I'm having some problems :
It seems the Contrib Module isn't included in OpenCV 2.4.2 . is there anyway to use it in the project ?
I tried using JavaCV to use the Contrib Module's "FaceRecognizer" class. there are two classes available called "FaceRecognizer" & "FaceRecognizerPtr". does anybody know what the difference between these two is ?
The classes mentioned above have a method called "Train" which (In C++) receives two Vectors of types "Mat & Integer" ( model->train(images,labels) & train(Vector<mat> theImages, Vector<int> theLabels) . I tried passing them ArrayList<mat> & ArrayList<integer> and Vectors in Java but it seems that the method explicitly accepts the "CvArr" Data type which I'm not sure how to acquire... Here is the error :
The method train(opencv_core.CvArr, opencv_core.CvArr) in the type
opencv_contrib.FaceRecognizer is not applicable for the arguments
(ArrayList, ArrayList)
Does anyone know how to change my ArrayList to CvArr ?!
This is my first post and I wasn't sure whether to ask all three questions in one post or in three posts so sorry for any inconveniences... If you need any other Information about the project, feel free to ask.
Update
The following article was written by Petter Christian Bjelland, so all credit is his. I am posting it here, because his blog seems to be in Maintenance mode at the moment, but I think it is worth sharing.
Doing face recognition with JavaCV (from http://pcbje.com)
I couldn’t find any tutorial on how to perform face recognition using OpenCV and Java, so I decided to share a viable solution here. The solution is very inefficient in its current form as the training model is built at each run, however it shows what’s needed to make it work.
The class below takes two arguments: The path to the directory containing the training faces and the path to the image you want to classify. Not that all images has to be of the same size and that the faces already has to be cropped out of their original images (Take a look here if you haven’t done the face detection yet).
For the simplicity of this post, the class also requires that the training images have filename format: <label>-rest_of_filename.png. For example:
1-jon_doe_1.png
1-jon_doe_2.png
2-jane_doe_1.png
2-jane_doe_2.png
... and so on.
The code:
import com.googlecode.javacv.cpp.opencv_core;
import static com.googlecode.javacv.cpp.opencv_highgui.*;
import static com.googlecode.javacv.cpp.opencv_core.*;
import static com.googlecode.javacv.cpp.opencv_imgproc.*;
import static com.googlecode.javacv.cpp.opencv_contrib.*;
import java.io.File;
import java.io.FilenameFilter;
public class OpenCVFaceRecognizer {
public static void main(String[] args) {
String trainingDir = args[0];
IplImage testImage = cvLoadImage(args[1]);
File root = new File(trainingDir);
FilenameFilter pngFilter = new FilenameFilter() {
public boolean accept(File dir, String name) {
return name.toLowerCase().endsWith(".png");
}
};
File[] imageFiles = root.listFiles(pngFilter);
MatVector images = new MatVector(imageFiles.length);
int[] labels = new int[imageFiles.length];
int counter = 0;
int label;
IplImage img;
IplImage grayImg;
for (File image : imageFiles) {
// Get image and label:
img = cvLoadImage(image.getAbsolutePath());
label = Integer.parseInt(image.getName().split("\\-")[0]);
// Convert image to grayscale:
grayImg = IplImage.create(img.width(), img.height(), IPL_DEPTH_8U, 1);
cvCvtColor(img, grayImg, CV_BGR2GRAY);
// Append it in the image list:
images.put(counter, grayImg);
// And in the labels list:
labels[counter] = label;
// Increase counter for next image:
counter++;
}
FaceRecognizer faceRecognizer = createFisherFaceRecognizer();
// FaceRecognizer faceRecognizer = createEigenFaceRecognizer();
// FaceRecognizer faceRecognizer = createLBPHFaceRecognizer()
faceRecognizer.train(images, labels);
// Load the test image:
IplImage greyTestImage = IplImage.create(testImage.width(), testImage.height(), IPL_DEPTH_8U, 1);
cvCvtColor(testImage, greyTestImage, CV_BGR2GRAY);
// And get a prediction:
int predictedLabel = faceRecognizer.predict(greyTestImage);
System.out.println("Predicted label: " + predictedLabel);
}
}
The class requires the OpenCV Java interface. If you’re using Maven, you can retrieve the required libraries with the following pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.pcbje</groupId>
<artifactId>opencvfacerecognizer</artifactId>
<version>0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>opencvfacerecognizer</name>
<url>http://pcbje.com</url>
<dependencies>
<dependency>
<groupId>com.googlecode.javacv</groupId>
<artifactId>javacv</artifactId>
<version>0.3</version>
</dependency>
<!-- For Linux x64 environments -->
<dependency>
<groupId>com.googlecode.javacv</groupId>
<artifactId>javacv</artifactId>
<classifier>linux-x86_64</classifier>
<version>0.3</version>
</dependency>
<!-- For OSX environments -->
<dependency>
<groupId>com.googlecode.javacv</groupId>
<artifactId>javacv</artifactId>
<classifier>macosx-x86_64</classifier>
<version>0.3</version>
</dependency>
</dependencies>
<repositories>
<repository>
<id>javacv</id>
<name>JavaCV</name>
<url>http://maven2.javacv.googlecode.com/git/</url>
</repository>
</repositories>
</project>
Original Post
Quoting from my reply on http://answers.opencv.org/question/865/the-contrib-module-problem.
Without ever having used javacv, let's see how far we can get by just looking at the interfaces! The project is on googlecode, which makes it easy to browse the code: http://code.google.com/p/javacv.
First have a look at how cv::FaceRecognizer has been wrapped (opencv_contrib.java, line 845 at time of writing this):
#Namespace("cv") public static class FaceRecognizer extends Algorithm {
static { Loader.load(); }
public FaceRecognizer() { }
public FaceRecognizer(Pointer p) { super(p); }
public /*abstract*/ native void train(#ByRef MatVector src, #Adapter("ArrayAdapter") CvArr labels);
public /*abstract*/ native int predict(#Adapter("ArrayAdapter") CvArr src);
public /*abstract*/ native void predict(#Adapter("ArrayAdapter") CvArr src, #ByRef int[] label, #ByRef double[] dist);
public native void save(String filename);
public native void load(String filename);
public native void save(#Adapter("FileStorageAdapter") CvFileStorage fs);
public native void load(#Adapter("FileStorageAdapter") CvFileStorage fs);
}
Aha, so you need to pass a MatVector for the images! You can pass the labels in a CvArr (one row or one column). The MatVector is defined in opencv_core, line 4629 (at time of writing this) and it looks like this:
public static class MatVector extends Pointer {
static { load(); }
public MatVector() { allocate(); }
public MatVector(long n) { allocate(n); }
public MatVector(Pointer p) { super(p); }
private native void allocate();
private native void allocate(#Cast("size_t") long n);
public native long size();
public native void resize(#Cast("size_t") long n);
#Index #ValueGetter public native #Adapter("MatAdapter") CvMat getCvMat(#Cast("size_t") long i);
#Index #ValueGetter public native #Adapter("MatAdapter") CvMatND getCvMatND(#Cast("size_t") long i);
#Index #ValueGetter public native #Adapter("MatAdapter") IplImage getIplImage(#Cast("size_t") long i);
#Index #ValueSetter public native MatVector put(#Cast("size_t") long i, #Adapter("MatAdapter") CvArr value);
}
Again just by looking at the code, I guess it can be used like this:
int numberOfImages = 10;
// Allocate some memory:
MatVector images = new MatVector(numberOfImages);
// Then fill the MatVector, you probably want to do something useful instead:
for(int idx = 0; idx < numberOfImages; idx++){
// Load an image:
CvArr image = cvLoadImage("/path/to/your/image");
// And put it into the MatVector:
images.put(idx, image);
}
You probably want to write yourself a method that does the conversion from a Java ArrayList to a MatVector (if such a function does not exist in javacv yet).
Now to your second question. FaceRecognizer is the equivalent to cv::FaceRecognizer. The native OpenCV C++ classes return a cv::Ptr<cv::FaceRecognizer>, which is a (Smart) Pointer to a cv::FaceRecognizer. This has to be wrapped as well. See a pattern here?
The interface of FaceRecognizerPtr now looks like this:
#Name("cv::Ptr<cv::FaceRecognizer>")
public static class FaceRecognizerPtr extends Pointer {
static { load(); }
public FaceRecognizerPtr() { allocate(); }
public FaceRecognizerPtr(Pointer p) { super(p); }
private native void allocate();
public native FaceRecognizer get();
public native FaceRecognizerPtr put(FaceRecognizer value);
}
So you can either get a FaceRecognizer from this class or put a FaceRecognizer into. You should only be concerned about the get(), as the Pointer is filled by the method creating the concrete FaceRecognizer algorithm:
#Namespace("cv") public static native #ByVal FaceRecognizerPtr createEigenFaceRecognizer(int num_components/*=0*/, double threshold/*=DBL_MAX*/);
#Namespace("cv") public static native #ByVal FaceRecognizerPtr createFisherFaceRecognizer(int num_components/*=0*/, double threshold/*=DBL_MAX*/);
#Namespace("cv") public static native #ByVal FaceRecognizerPtr createLBPHFaceRecognizer(int radius/*=1*/,
int neighbors/*=8*/, int grid_x/*=8*/, int grid_y/*=8*/, double threshold/*=DBL_MAX*/);
So once you have got the FaceRecognizerPtr, you can do things like:
// Holds your training data and labels:
MatVector images;
CvArr labels;
// Do something with the images and labels... Probably fill them?
// ...
// Then get a Pointer to a FaceRecognizer (FaceRecognizerPtr).
// Java doesn't have default parameters, so you have to add some yourself,
// if you pass 0 as num_components to the EigenFaceRecognizer, the number of
// components is determined by the data, for the threshold use the maximum possible
// value if you don't want one. I don't know the constant in Java:
FaceRecognizerPtr model = createEigenFaceRecognizer(0, 10000);
// Then train it. See how I call get(), to get the FaceRecognizer inside the FaceRecognizerPtr:
model.get().train(images, labels);
This learns you an Eigenfaces model. And that's it!
I made an android app for face recognition using opencv. For good recognition you need a better detection, you can check it out from: https://github.com/yaylas/AndroidFaceRecognizer
I hope it helps.
Related
So in my app i pass a game object, called datacontroller through out my three scenes. The persistent scene is an empty scene, the menuscreen scene and then the game scene. My application works perfectly on my computer and in editor mode but when i download the apk to my android tablet it no longer works! iv'e read this may have to do with my code for my object but i dont think i written anything that only works in the editor.
enter code here
using UnityEngine;
using UnityEngine.SceneManagement;
using System.Collections;
using System.IO; // The System.IO namespace contains functions related to loading and saving
files
public class DataController : MonoBehaviour
{
private RoundData[] allRoundData;
private PlayerProgress playerProgress;
private string gameDataFileName = "data.json";
void Start()
{
DontDestroyOnLoad(gameObject);
LoadGameData();
LoadPlayerProgress();
SceneManager.LoadScene("MenuScreen");
}
public RoundData GetCurrentRoundData()
{
// If we wanted to return different rounds, we could do that here
// We could store an int representing the current round index in PlayerProgress
return allRoundData[0];
}
public void SubmitNewPlayerScore(int newScore)
{
// If newScore is greater than playerProgress.highestScore, update playerProgress with the new value and call SavePlayerProgress()
if (newScore > playerProgress.highestScore)
{
playerProgress.highestScore = newScore;
SavePlayerProgress();
}
}
public int GetHighestPlayerScore()
{
return playerProgress.highestScore;
}
private void LoadGameData()
{
// Path.Combine combines strings into a file path
// Application.StreamingAssets points to Assets/StreamingAssets in the Editor, and the StreamingAssets folder in a build
string filePath = Path.Combine(Application.streamingAssetsPath, gameDataFileName);
if (File.Exists(filePath))
{
// Read the json from the file into a string
string dataAsJson = File.ReadAllText(filePath);
// Pass the json to JsonUtility, and tell it to create a GameData object from it
GameData loadedData = JsonUtility.FromJson<GameData>(dataAsJson);
// Retrieve the allRoundData property of loadedData
allRoundData = loadedData.allRoundData;
}
else
{
Debug.LogError("Cannot load game data!");
}
}
// This function could be extended easily to handle any additional data we wanted to store in our PlayerProgress object
private void LoadPlayerProgress()
{
// Create a new PlayerProgress object
playerProgress = new PlayerProgress();
// If PlayerPrefs contains a key called "highestScore", set the value of playerProgress.highestScore using the value associated with that key
if (PlayerPrefs.HasKey("highestScore"))
{
playerProgress.highestScore = PlayerPrefs.GetInt("highestScore");
}
}
// This function could be extended easily to handle any additional data we wanted to store in our PlayerProgress object
private void SavePlayerProgress()
{
// Save the value playerProgress.highestScore to PlayerPrefs, with a key of "highestScore"
PlayerPrefs.SetInt("highestScore", playerProgress.highestScore);
}
}
I am starting to go through tutorials of unity myself so I am not an expert. :)
But what I would try is the first thing. using System.IO; not sure if this will work to get files on android because android has a different file structure. So I would first remove it and sort of hard code the file path or comment out the code using System.IO classes then recompile apk in unity and check if it works. I also saw this post : http://answers.unity3d.com/questions/1023391/systemio-dont-work-on-android.html
If that did not work I would comment functionality out and compile apk and check if its working if its not comment more code out until you find the line or the code that causes it to error on android. This method takes long to troubleshoot or get your code causing the problem but for me this has worked before.
I am guessing as it is working on you pc its a class or something its referencing that's not available in android.
Please share your findings if you figure out what part of the code does it. As I would also want to know to prevent me from doing it. :)
Avoid the use of System.IO on Android and in general on Unity.
Use "Resources" instead, just following this steps:
Create a folder called "Resources"
Move json file on it and rename in .txt
Use this code to get the string:
var file = Resources.Load("filename_here") as TextAsset;
Debug.Log(file.text)
I'm using muPDF for reading PDFs in my application. I don't like its default animation (Switching horizontally). In other side i found this brilliant library for curl effect on images, and this project for flip-flap effect on layouts.
In curl sample project, in CurlActivity, all of data are images and set in PageProvider like this:
private class PageProvider implements CurlView.PageProvider {
// Bitmap resources.
private int[] mBitmapIds = { R.drawable.image1, R.drawable.image2,
R.drawable.image3, R.drawable.image4};
And use it like this:
private CurlView mCurlView;
mCurlView = (CurlView) findViewById(R.id.curl);
mCurlView.setPageProvider(new PageProvider());
And CurlView extends from GLSurfaceView and implements View.OnTouchListener, CurlRenderer.Observer
But in muPDF if i'm not mistaken, data are in core object. core is instance of MuPDFCore. And using it like this:
MuPDFReaderView mDocView;
MuPDFView pageView = (MuPDFView) mDocView.getDisplayedView();
mDocView.setAdapter(new MuPDFPageAdapter(this, this, core));
MuPDFReaderView extends ReaderView and ReaderView extends AdapterView<Adapter> and implements GestureDetector.OnGestureListener, ScaleGestureDetector.OnScaleGestureListener, Runnable.
My question is where how can I using curl effect in muPDF? Where should I get pages one by one and converting them to bitmaps? and then changing aspects of the Adapter in muPDF to CurlView.
In flip-flap sample project, in FlipHorizontalLayoutActivity (I like this effect too), we have these:
private FlipViewController flipView;
flipView = new FlipViewController(this, FlipViewController.HORIZONTAL);
flipView.setAdapter(new TravelAdapter(this));
setContentView(flipView);
And FlipViewController extends AdapterView<Adapter>, and data set in TravelAdapter that extends BaseAdapter.
No one has done this before? Or can help me to do that?!
EDIT:
I found another good open source PDF reader with curl effect called fbreaderJ. its developer says "An additional module that allows to open PDF files in FBReader. Based on radaee pdf library."
I got confused! cause radaeepdf is closed source and downloadable project is just for demo and inserted username and password is for this package.
People want to change whole fbreader project such as package name.
Another issue for make me confused is where is this additional module source code?!
Anyway, if someone wants to help me, fbreader has done it very well.
EDIT:
I talked to Robin Watts, who developed muPDF (or one of developers), and he said:
Have you read platform/android/ClassStructure.txt ? MuPDF is
primarily a C library. The standard api is therefore a C one. Rather
than exposing that api exactly as is to Java (which would be the
nicest solution, and something that I've done some work on, but have
not completed due to lack of time), we've implemented MuPDFCore to
wrap up just the bits we needed. MuPDFCore handles opening a PDF file,
and getting bitmaps from it to be used in views. or rather, MuPDFCore
returns 'views', not 'bitmaps'. If you need bitmaps, then you're going
to need to make changes in MuPDFCore.
There are too many errors when changing a little part of MuPDFReaderView class. I get confused! These are related to each other.
Please answer more precisely.
EDIT:
And bounty has expired.
If the muPDF does not support rendering to a bitmap, you have no other choice than rendering to a regular view and take a screen dump to a bitmap like this:
View content = findViewById(R.id.yourPdfView);
Bitmap bitmap = content.getDrawingCache();
Then use this bitmap as input to your other library.
Where should i get pages one by one and converting them to bitmaps?
In our application (newspaper app) we use MuPDF to render PDFs.
The workflow goes like this:
Download PDF file (we have one PDF per newspaper page)
Render it with MuPDF
Save the bitmap to the filesystem
Load the Bitmap from filesystem as background image to a view
So, finally, what we use is MuPDFCore.java and its methods drawPage(...) and onDestroy()
Is this what you want to know or do i miss the point?
EDIT
1.) I think it is not necessary to post code how to download a file. But after downloading i add a RenderTask (extends from Runnable) to a Renderqueue and trigger that queue. The RenderTask needs some information for rendering:
/**
* constructs a new RenderTask instance
* #param context: you need Context for MuPdfCore instance
* #param pageNumber
* #param pathToPdf
* #param renderCallback: callback to set bitmap to the view after
* rendering
* #param heightOfRenderedBitmap: this is the target height
* #param widthOfRenderedBitmap: this is the target width
*/
public RenderTask (Context context, Integer pageNumber, String pathToPdf, IRenderCallback,
renderCallback, int heightOfRenderedBitmap,
int widthOfRenderedBitmap) {
//store things in fields
}
2.) + 3.) The Renderqueue wraps the RenderTask in a new Thread and starts it. So the run-method of the RenderTask will be invoked:
#Override
public void run () {
//do not render it if file exists
if (exists () == true) {
finish();
return;
}
Bitmap bitmap = render();
//if something went wrong, we can't store the bitmap
if (bitmap == null) {
finish();
return;
}
//now save the bitmap
// in my case i save the destination path in a String field
imagePath = save(bitmap, new File("path/to/your/destination/folder/" + pageNumber + ".jpg"));
bitmap.recycle();
finish();
}
/**
* let's trigger the callback
*/
private void finish () {
if (renderCallback != null) {
// i send the whole Rendertask to callback
// maybe in your case it is enough to send the pageNumber or path to
// renderend bitmap
renderCallback.finished(this);
}
}
/**
* renders a bitmap
* #return
*/
private Bitmap render() {
MuPDFCore core = null;
try {
core = new MuPDFCore(context, pathToPdf);
} catch (Exception e) {
return null;
}
Bitmap bm = Bitmap.createBitmap(widthOfRenderedBitmap, heightOfRenderedBitmap, Config.ARGB_8888);
// here you render the WHOLE pdf cause patch-x/-y == 0
core.drawPage(bm, 0, widthOfRenderedBitmap, heightOfRenderedBitmap, 0, 0, widthOfRenderedBitmap, heightOfRenderedBitmap, core.new Cookie());
core.onDestroy();
core = null;
return bm;
}
/**
* saves bitmap to filesystem
* #param bitmap
* #param image
* #return
*/
private String save(Bitmap bitmap, File image) {
FileOutputStream out = null;
try {
out = new FileOutputStream(image.getAbsolutePath());
bitmap.compress(Bitmap.CompressFormat.JPEG, 80, out);
return image.getAbsolutePath();
} catch (Exception e) {
return null;
}
finally {
try {
if (out != null) {
out.close();
}
} catch(Throwable ignore) {}
}
}
}
4.) I think it is not necessary to post code how to set a bitmap as background of a view
How to implement a system which takes the best texture based on dpi from a set like in android SDK, because LibGDX is platform independent, and can't use the already existent one?
May be my solution is not the best but I used it in real project and it worked.
In your Game class use Gdx.graphics.getDensity() method to choose appropriate folder, store its name in public field and load your assets:
public class MyGame extends Game {
public static String folder;
private AssetManager assets;
#Override
public void create() {
if (Gdx.graphics.getDensity < 1) {
folder = "lowDpiImages/";
} else {
folder = "highDpiImages/";
}
...
assets = new AssetManager();
assets.load(folder + "image.png", Texture.class, paramsNearest);
...
}
Then in you other classes use folder name to get assets from AssetManager:
assets.get(MyGame.folder + "image.png", Texture.class);
You can write more sophisticated folder choosing algorithm of cause ;-)
I am facing an issue with Retrofit and would like to find a suitable answer as the only way I can think of it is pretty ugly and not practical.
Retrofit PATH annotation requires a "/" in the beginning (as you can read in this code extracted from the library source:
/** Loads {#link #requestUrl}, {#link #requestUrlParamNames}, and {#link #requestQuery}. */
private void parsePath(String path) {
if (path == null || path.length() == 0 || path.charAt(0) != '/') {
throw methodError("URL path \"%s\" must start with '/'.", path);
}
The problem that I am facing is that the PATH part comes from the backend in a response object, meaning that all PATH's strings already come formatted from the backend previously in other response as follows:
Object : {
href: "/resources/login..."
}
As you can see, when including something like this, the URL gets malformed:
#GET("{/loginHref}")
void login(#EncodedPath("loginHref") String loginHref,
Callback<User> callback);
to something like "http://mybaseurl.com//resources/login" *double // in front of resources
This can definitely cause issues in some endpoints and I cannot think a really simple way to solve this issue apart from doing something like:
a) Modify my own version of retrofit to remove that / character check (this is a last resort)
b) Truncate the href before using the method from the interface (which I would like to avoid at all cost as well as would add unnecessary transformation all over the place.
c) Intercept the request and correctly form the URL in case this scenario happens (really ugly solution as well).
Any idea, suggestions?
Thanks!
I think this link will help you Path Replacement
Your new implementation will look like this.
#GET("/")
void login(Callback<User> callback);
You can supply a custom Endpoint implementation on which you can change the relative path.
public final class CustomEndpoint implements Endpoint {
private static final String BASE = "http://192.168.1.64:5050/api/";
private String url;
private String href;
public CustomEndpoint(String href){
this.href = href;
url = BASE + this.href;
}
#Override public String getName() {
return "default";
}
#Override public String getUrl() {
if (url == null) throw new IllegalStateException("relative path not set.");
return url;
}
}
Usage is as follows
RestAdapter restAdapter = new RestAdapter.Builder()
.setEndpoint(new CustomEndPoint(object.href));
then restadapter.create........
Hope this will help you.
I made and compiled a Android Library, containing a simple class and a simple static function:
package moo;
public class MyTestClass {
public static String Foo(){
return "Foo from Moo";
}
}
I placed the .jar in my Assets/Plugins/Android Folder. Then In Unity:
void OnGUI () {
string somestring = "foooooooooooOOooo";
AndroidJavaClass testClass = new AndroidJavaClass("moo.MyTestClass");
somestring = testClass.CallStatic<string>("Foo");
GUI.Label (new Rect (20, 20, 100, 20), somestring);
}
And I get an error:
JNI: Unable to find method id for 'Foo' (static)
UnityEngine.AndroidJavaObject:CallStatic(String, Object[])
Am I missing something to call my static method?
Thanks!
there are 2 problem as far as I can see:
you have to put your jar package to Assets/Plugins/Android/bin;
you will always get this error on your windows/mac editor, you have to run this on your android device;