I have a custom object detection model.
I'm using Interpreter.runForMultipleInputsOutputs to detect an object on bitmap.
The problem is that at some point I started to receive zeroes at output when using GPU and
fine results with CPU.
An interesting fact that I have no problem with GPU on my test devece and I have this problem on remote device, that is my target device.
What could be the problem?
Here is my code, I have changed an exaple a bit:
var gpuDelegate: GpuDelegate? = if (device == Device.GPU) GpuDelegate() else null
val options = Interpreter
.Options()
.apply {
when (device) {
is Device.CPU -> {
setNumThreads(device.numThreads)
}
Device.GPU -> {
addDelegate(gpuDelegate)
}
}
}
val interpreter: Interpreter = Interpreter(
FileUtil.loadMappedFile(context, MODEL_FILENAME),
options
)
...
val inputTensor = processInputImage(bitmap, inputWidth, inputHeight)
val locations = arrayOf(Array(10) { FloatArray(4) })
val labelIndices = arrayOf(FloatArray(10))
val scores = arrayOf(FloatArray(10))
val outputCount = FloatArray(1)
val outputBuffer = mapOf(
0 to locations,
1 to labelIndices,
2 to scores,
3 to outputCount
)
val detectionRegions = mutableListOf<WeightRegion>()
interpreter.runForMultipleInputsOutputs(arrayOf(inputTensor.buffer), outputBuffer)
gpuDelegate?.close()
interpreter.close()
build.gradle:
android {
...
buildFeatures {
mlModelBinding true
}
}
dependencies {
...
implementation 'org.tensorflow:tensorflow-lite:2.7.0'
implementation 'org.tensorflow:tensorflow-lite-gpu:2.7.0'
implementation 'org.tensorflow:tensorflow-lite-support:0.3.0'
implementation 'org.tensorflow:tensorflow-lite-metadata:0.3.0'
}
I have a Plant data class, that has a PlantType sealed class parameter.
I am using a Room local db, but when I try to parse it it fails. It works for other classes that has initializable class parameters.
Thanks for the help in advance.
The error:
java.lang.RuntimeException: Failed to invoke private com.tenyitamas.mylittlegarden.domain.util.PlantType() with no args
at com.tenyitamas.mylittlegarden.data.util.Converters.fromPlantsJson(Converters.kt:99)
// Comment: Converters.kt: 99 is the from json part of the Converters code, I included.
Plant.kt:
data class Plant(
val type: PlantType,
val upgrades: Upgrades
)
PlantType.kt:
sealed class PlantType {
object Carrot : PlantType()
object Tomato : PlantType()
object Cucumber : PlantType()
object Lettuce : PlantType()
object Strawberry : PlantType()
private companion object Constants {
const val INITIAL_TIME_CARROT = 5_000L // 5s
const val INITIAL_TIME_TOMATO = 6_000L
const val INITIAL_TIME_CUCUMBER = 7_000L
const val INITIAL_TIME_LETTUCE = 8_000L
const val INITIAL_TIME_STRAWBERRY = 9_000L
const val INITIAL_COST_CARROT = 10
const val INITIAL_COST_TOMATO = 100
const val INITIAL_COST_CUCUMBER = 1_000
const val INITIAL_COST_LETTUCE = 10_000
const val INITIAL_COST_STRAWBERRY = 100_000
const val INITIAL_INCOME_CARROT = 10
const val INITIAL_INCOME_TOMATO = 100
const val INITIAL_INCOME_CUCUMBER = 1_000
const val INITIAL_INCOME_LETTUCE = 10_000
const val INITIAL_INCOME_STRAWBERRY = 100_000
}
val name: String
get() {
return when (this) {
Carrot -> Resources.getSystem().getString(R.string.carrot)
Tomato -> Resources.getSystem().getString(R.string.tomato)
Cucumber -> Resources.getSystem().getString(R.string.cucumber)
Lettuce -> Resources.getSystem().getString(R.string.lettuce)
Strawberry -> Resources.getSystem().getString(R.string.strawberry)
}
}
val image: Bitmap
get() {
return when (this) {
Carrot -> {
BitmapFactory.decodeResource(Resources.getSystem(), R.drawable.ic_carrot)
}
Tomato -> {
BitmapFactory.decodeResource(Resources.getSystem(), R.drawable.ic_tomato)
}
Cucumber -> {
BitmapFactory.decodeResource(Resources.getSystem(), R.drawable.ic_cucumber)
}
Lettuce -> {
BitmapFactory.decodeResource(Resources.getSystem(), R.drawable.ic_lettuce)
}
Strawberry -> {
BitmapFactory.decodeResource(Resources.getSystem(), R.drawable.ic_strawberry)
}
}
}
val initialTime: Long
get() {
return when (this) {
Carrot -> INITIAL_TIME_CARROT
Tomato -> INITIAL_TIME_TOMATO
Cucumber -> INITIAL_TIME_CUCUMBER
Lettuce -> INITIAL_TIME_LETTUCE
Strawberry -> INITIAL_TIME_STRAWBERRY
}
}
val initialCost: Int
get() {
return when (this) {
Carrot -> INITIAL_COST_CARROT
Tomato -> INITIAL_COST_TOMATO
Cucumber -> INITIAL_COST_CUCUMBER
Lettuce -> INITIAL_COST_LETTUCE
Strawberry -> INITIAL_COST_STRAWBERRY
}
}
val initialIncome: Int
get() {
return when (this) {
Carrot -> INITIAL_INCOME_CARROT
Tomato -> INITIAL_INCOME_TOMATO
Cucumber -> INITIAL_INCOME_CUCUMBER
Lettuce -> INITIAL_INCOME_LETTUCE
Strawberry -> INITIAL_INCOME_STRAWBERRY
}
}
}
Converters.kt
#TypeConverter
fun fromPlantsJson(json: String): List<Plant> {
return jsonParser.fromJson<ArrayList<Plant>>(
json,
object : TypeToken<ArrayList<Plant>>(){}.type
) ?: emptyList()
}
#TypeConverter
fun toPlantsJson(plants: List<Plant>): String {
return jsonParser.toJson(
plants,
object : TypeToken<ArrayList<Plant>>(){}.type
) ?: "[]"
}
Your PlantType sealed class has only objects inside it. You can just use an enum for this.
enum class PlantType {
Carrot, Tomato, Cucumber, Lettuce, Strawberry
}
And most likely your jsonParser will be able to serialize this enum.
Edit: In your edited question, you just have added some new vals inside the PlantType class all of which has custom getters which return value depending upon the PlantType. You can still replace it with the enum and put those vals as extensions on PlantType.
For example,
enum class PlantType {
Carrot, Tomato, Cucumber, Lettuce, Strawberry
}
val PlantType.name: String
get() {
return when (this) {
Carrot -> Resources.getSystem().getString(R.string.carrot)
Tomato -> Resources.getSystem().getString(R.string.tomato)
Cucumber -> Resources.getSystem().getString(R.string.cucumber)
Lettuce -> Resources.getSystem().getString(R.string.lettuce)
Strawberry -> Resources.getSystem().getString(R.string.strawberry)
}
}
var acct = Activity
when {
activityName.contains("MainActivity", true) -> {
acct = (context as MainActivity)
}
activityName.contains("UserMemes", true) -> {
acct = (context as UserMemes)
}
activityName.contains("SearchShow", true) -> {
acct = (context as SearchShow)
}
}
acct.memeIdAndLastPos[meme.id] = scrollY
I get Classifier 'Activity' does not have a companion object, and thus must be initialized here On Activity
When I do Activity() instead then the error disappears but I can't use memeIdAndLastPos any more.
How to fix?
val acct: Activity = when {
activityName.contains("MainActivity", true) -> context as MainActivity
activityName.contains("UserMemes", true) -> context as UserMemes
activityName.contains("SearchShow", true) -> context as SearchShow
}
acct.memeIdAndLastPos[meme.id] = scrollY
I'm trying to set up Safe-room lib in my project but I got a mysterious error.
For this example, I'm using todo-mvp by Google, so they're using Dagger2 and RxJava.
in TodoDatabase.class I have the bunch of code:
fun getInstance(context: Context): ToDoDatabase {
synchronized(lock) {
if (INSTANCE == null) {
val admin = "password123".toCharArray()
val factory = SafeHelperFactory(admin)
INSTANCE = Room.databaseBuilder(context.applicationContext, ToDoDatabase::class.java, "Tasks.db")
.openHelperFactory(factory)
.build()
SQLCipherUtils.encrypt(context, "Tasks.db", admin)
val state = SQLCipherUtils.getDatabaseState(context, "Tasks.db")
if (state.name == SQLCipherUtils.State.ENCRYPTED.name)
Log.e("test", "Database encrypted - $state")
else
Log.e("test", "Database encrypted - $state")
}
return INSTANCE!!
}
}
After executing this line INSTANCE = Room.databaseBuilder(context.applicationContext, ToDoDatabase::class.java, "Tasks.db").openHelperFactory(factory).build()
I caught the error:
E/art: Failed to register native method net.sqlcipher.database.SQLiteDatabase.native_getDbLookaside()I in /data/app/com.example.android.architecture.blueprints.todomvp.mock-1/split_lib_dependencies_apk.apk:classes5.dex
----- class 'Lnet/sqlcipher/database/SQLiteDatabase;' cl=0x12c81280 -----
objectSize=1096 (128 from super)
I am using Transformations.switchMap in my ViewModel so my LiveData collection, observed in my fragment, reacts on changes of code parameter.
This works perfectly :
public class MyViewModel extends AndroidViewModel {
private final LiveData<DayPrices> dayPrices;
private final MutableLiveData<String> code = new MutableLiveData<>();
// private final MutableLiveData<Integer> nbDays = new MutableLiveData<>();
private final DBManager dbManager;
public MyViewModel(Application application) {
super(application);
dbManager = new DBManager(application.getApplicationContext());
dayPrices = Transformations.switchMap(
code,
value -> dbManager.getDayPriceData(value/*, nbDays*/)
);
}
public LiveData<DayPrices> getDayPrices() {
return dayPrices;
}
public void setCode(String code) {
this.code.setValue(code);
}
/*public void setNbDays(int nbDays) {
this.nbDays.setValue(nbDays);
}*/
}
public class MyFragment extends Fragment {
private MyViewModel myViewModel;
myViewModel = ViewModelProviders.of(this).get(MyViewModel.class);
myViewModel.setCode("SO");
//myViewModel.setNbDays(30);
myViewModel.getDayPrices().observe(MyFragment.this, dataList -> {
// update UI with data from dataList
});
}
Problem
I now need another parameter (nbDays commented in the code above), so that my LiveData object reacts on both parameters change (code and nbDays).
How can I chain transformations ?
Some reading pointed me to MediatorLiveData, but it does not solve my problem (still need to call single DB function with 2 parameters, I don't need to merge 2 liveDatas).
So I tried this instead of switchMap but code and nbDays are always null.
dayPrices.addSource(
dbManager.getDayPriceData(code.getValue(), nbDays.getValue),
apiResponse -> dayPrices.setValue(apiResponse)
);
One solution would be to pass an object as single parameter by I'm pretty sure there is a simple solution to this.
Source : https://plus.google.com/+MichielPijnackerHordijk/posts/QGXF9gRomVi
To have multiple triggers for switchMap(), you need to use a custom MediatorLiveData to observe the combination of the LiveData objects -
class CustomLiveData extends MediatorLiveData<Pair<String, Integer>> {
public CustomLiveData(LiveData<String> code, LiveData<Integer> nbDays) {
addSource(code, new Observer<String>() {
public void onChanged(#Nullable String first) {
setValue(Pair.create(first, nbDays.getValue()));
}
});
addSource(nbDays, new Observer<Integer>() {
public void onChanged(#Nullable Integer second) {
setValue(Pair.create(code.getValue(), second));
}
});
}
}
Then you can do this -
CustomLiveData trigger = new CustomLiveData(code, nbDays);
LiveData<DayPrices> dayPrices = Transformations.switchMap(trigger,
value -> dbManager.getDayPriceData(value.first, value.second));
If you use Kotlin and want to work with generics:
class DoubleTrigger<A, B>(a: LiveData<A>, b: LiveData<B>) : MediatorLiveData<Pair<A?, B?>>() {
init {
addSource(a) { value = it to b.value }
addSource(b) { value = a.value to it }
}
}
Then:
val dayPrices = Transformations.switchMap(DoubleTrigger(code, nbDays)) {
dbManager.getDayPriceData(it.first, it.second)
}
Custom MediatorLiveData as proposed by #jL4 works great and is probably the solution.
I just wanted to share the simplest solution that I think is to use an inner class to represent the composed filter values :
public class MyViewModel extends AndroidViewModel {
private final LiveData<DayPrices> dayPrices;
private final DBManager dbManager;
private final MutableLiveData<DayPriceFilter> dayPriceFilter;
public MyViewModel(Application application) {
super(application);
dbManager = new DBManager(application.getApplicationContext());
dayPriceFilter = new MutableLiveData<>();
dayPrices = Transformations.switchMap(dayPriceFilter, input -> dbManager.getDayPriceData(input.code, input.nbDays));
}
public LiveData<DayPrices> getDayPrices() {
return dayPrices;
}
public void setDayPriceFilter(String code, int nbDays) {
DayPriceFilter update = new DayPriceFilter(code, nbDays);
if (Objects.equals(dayPriceFilter.getValue(), update)) {
return;
}
dayPriceFilter.setValue(update);
}
static class DayPriceFilter {
final String code;
final int nbDays;
DayPriceFilter(String code, int nbDays) {
this.code = code == null ? null : code.trim();
this.nbDays = nbDays;
}
}
}
Then in the activity/fragment :
public class MyFragment extends Fragment {
private MyViewModel myViewModel;
myViewModel = ViewModelProviders.of(this).get(MyViewModel.class);
myViewModel.setDayPriceFilter("SO", 365);
myViewModel.getDayPrices().observe(MyFragment.this, dataList -> {
// update UI with data from dataList
});
}
A simplification of jL4's answer, (and also in Kotlin in case it helps anybody)... no need to create a custom class for this:
class YourViewModel: ViewModel() {
val firstLiveData: LiveData<String> // or whatever type
val secondLiveData: LiveData<Int> // or whatever
// the Pair values are nullable as getting "liveData.value" can be null
val combinedValues = MediatorLiveData<Pair<String?, Int?>>().apply {
addSource(firstLiveData) {
value = Pair(it, secondLiveData.value)
}
addSource(secondLiveData) {
value = Pair(firstLiveData.value, it)
}
}
val results = Transformations.switchMap(combinedValues) { pair ->
val firstValue = pair.first
val secondValue = pair.second
if (firstValue != null && secondValue != null) {
yourDataSource.yourLiveDataCall(firstValue, secondValue)
} else null
}
}
Explanation
Any update in firstLiveData or secondLiveData will update the value of combinedValues, and emit the two values as a pair (thanks to jL4 for this).
Calling liveData.value can be null, so this solution makes the values in Pair nullable to avoid Null Pointer Exception.
So for the actual results/datasource call, the switch map is on the combinedValues live data, and the 2 values are extracted from the Pair and null checks are performed, so you can be sure of passing non-null values to your data source.
I use following classes to transform many live data with different types
class MultiMapLiveData<T>(
private val liveDataSources: Array<LiveData<*>>,
private val waitFirstValues: Boolean = true,
private val transform: (signalledLiveData: LiveData<*>) -> T
): LiveData<T>() {
private val mObservers = ArrayList<Observer<Any>>()
private var mInitializedSources = mutableSetOf<LiveData<*>>()
override fun onActive() {
super.onActive()
if (mObservers.isNotEmpty()) throw InternalError(REACTIVATION_ERROR_MESSAGE)
if (mInitializedSources.isNotEmpty()) throw InternalError(REACTIVATION_ERROR_MESSAGE)
for (t in liveDataSources.indices) {
val liveDataSource = liveDataSources[t]
val observer = Observer<Any> {
if (waitFirstValues) {
if (mInitializedSources.size < liveDataSources.size) {
mInitializedSources.add(liveDataSource)
}
if (mInitializedSources.size == liveDataSources.size) {
value = transform(liveDataSource)
}
} else {
value = transform(liveDataSource)
}
}
liveDataSource.observeForever(observer)
mObservers.add(observer)
}
}
override fun onInactive() {
super.onInactive()
for (t in liveDataSources.indices) {
val liveDataSource = liveDataSources[t]
val observer = mObservers[t]
liveDataSource.removeObserver(observer)
}
mObservers.clear()
mInitializedSources.clear()
}
companion object {
private const val REACTIVATION_ERROR_MESSAGE = "Reactivation of active LiveData"
}
}
class MyTransformations {
companion object {
fun <T> multiMap(
liveDataSources: Array<LiveData<*>>,
waitFirstValues: Boolean = true,
transform: (signalledLiveData: LiveData<*>) -> T
): LiveData<T> {
return MultiMapLiveData(liveDataSources, waitFirstValues, transform)
}
fun <T> multiSwitch(
liveDataSources: Array<LiveData<*>>,
waitFirstValues: Boolean = true,
transform: (signalledLiveData: LiveData<*>) -> LiveData<T>
): LiveData<T> {
return Transformations.switchMap(
multiMap(liveDataSources, waitFirstValues) {
transform(it)
}) {
it
}
}
}
}
Usage:
Note that the logic of the work is slightly different. The LiveData that caused the update (signalledLiveData) is passed to the Tranformation Listener as parameter, NOT the values of all LiveData. You get the current LiveData values yourself in the usual way via value property.
examples:
class SequenceLiveData(
scope: CoroutineScope,
start: Int,
step: Int,
times: Int
): LiveData<Int>(start) {
private var current = start
init {
scope.launch {
repeat (times) {
value = current
current += step
delay(1000)
}
}
}
}
suspend fun testMultiMap(lifecycleOwner: LifecycleOwner, scope: CoroutineScope) {
val liveS = MutableLiveData<String>("aaa")
val liveI = MutableLiveData<Int>()
val liveB = MutableLiveData<Boolean>()
val multiLiveWait: LiveData<String> = MyTransformations.multiMap(arrayOf(liveS, liveI, liveB)) {
when (it) {
liveS -> log("liveS changed")
liveI -> log("liveI changed")
liveB -> log("liveB changed")
}
"multiLiveWait: S = ${liveS.value}, I = ${liveI.value}, B = ${liveB.value}"
}
val multiLiveNoWait: LiveData<String> = MyTransformations.multiMap(arrayOf(liveS, liveI, liveB), false) {
when (it) {
liveS -> log("liveS changed")
liveI -> log("liveI changed")
liveB -> log("liveB changed")
}
"multiLiveNoWait: S = ${liveS.value}, I = ${liveI.value}, B = ${liveB.value}"
}
multiLiveWait.observe(lifecycleOwner) {
log(it)
}
multiLiveNoWait.observe(lifecycleOwner) {
log(it)
}
scope.launch {
delay(1000)
liveS.value = "bbb"
delay(1000)
liveI.value = 2222
delay(1000)
liveB.value = true // ***
delay(1000)
liveI.value = 3333
// multiLiveWait generates:
//
// <-- waits until all sources get first values (***)
//
// liveB changed: S = bbb, I = 2222, B = true
// liveI changed: S = bbb, I = 3333, B = true
// multiLiveNoWait generates:
// liveS changed: S = aaa, I = null, B = null
// liveS changed: S = bbb, I = null, B = null
// liveI changed: S = bbb, I = 2222, B = null
// liveB changed: S = bbb, I = 2222, B = true <-- ***
// liveI changed: S = bbb, I = 3333, B = true
}
}
suspend fun testMultiMapSwitch(lifecycleOwner: LifecycleOwner, scope: CoroutineScope) {
scope.launch {
val start1 = MutableLiveData(0)
val step1 = MutableLiveData(1)
val multiLiveData = MyTransformations.multiSwitch(arrayOf(start1, step1)) {
SequenceLiveData(scope, start1.value!!, step1.value!!, 5)
}
multiLiveData.observe(lifecycleOwner) {
log("$it")
}
delay(7000)
start1.value = 100
step1.value = 2
delay(7000)
start1.value = 200
step1.value = 3
delay(7000)
// generates:
// 0
// 1
// 2
// 3
// 4
// 100 <-- start.value = 100
// 100 <-- step.value = 2
// 102
// 104
// 106
// 108
// 200 <-- start.value = 200
// 200 <-- step.value = 3
// 203
// 206
// 209
// 212
}
}
I faced a similar problem. There are 2 ways to solve this:
Either use MediatorLiveData
Use RxJava as it has various operators to do such kind of complex stuff
If you don't know RxJava, then I'd recommend writing your custom MediatorLiveData class.
To learn how write custom MediatorLiveData class check out this example:
https://gist.github.com/AkshayChordiya/a79bfcc422fd27d52b15cdafc55eac6b