Develop Reference

Generating a Material3 theme based on a single arbitrary input color and applying it to Activities

This bounty has ended. Answers to this question are eligible for a +100 reputation bounty. Bounty grace period ends in 2 hours.
Steve M wants to draw more attention to this question.
I've set up my app to use Material3 colors roles and theme, and added an option to enable Material You support using DynamicColors.
However, I would also like the user to be able to select a single (preferably arbitrary) input color within the app, and have the app generate all the color roles and apply a theme from that. I feel like this should be possible somehow given that Material You is doing this based on the wallpaper colors, but don't really a way to do this for an arbitrary color. Is it possible?

Have you taken a look at these two links?
https://m3.material.io/styles/color/the-color-system/tokens
https://m3.material.io/styles/color/the-color-system/custom-colors
They describe what you're talking about, I am unsure of what implementation looks like on the code side. But to answer the question "Is it possible?", it looks like it.
This link describes how the dynamic color palletes are generated:
A dynamic tonal palette must be generated from a single source color that should be derived from wallpaper using com.android.systemui.monet.ColorScheme#getSeedColors, which provides multiple valid source colors. If none of the provided colors meet the source color requirement, the single source color should use the value 0xFF1B6EF3.
https://source.android.com/docs/core/display/dynamic-color#how_is_a_dynamic_tonal_pallet_generated
Here is the actual file from the AOSP if you wanted to look at how the actual dynamic color code works:
/*
* Copyright (C) 2021 The Android Open Source Project
* * Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* * http://www.apache.org/licenses/LICENSE-2.0
* * Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.android.systemui.monet
import android.annotation.ColorInt
import android.app.WallpaperColors
import android.graphics.Color
import com.android.internal.graphics.ColorUtils
import com.android.internal.graphics.cam.Cam
import com.android.internal.graphics.cam.CamUtils.lstarFromInt
import kotlin.math.absoluteValue
import kotlin.math.roundToInt
const val TAG = "ColorScheme"
const val ACCENT1_CHROMA = 48.0f
const val ACCENT2_CHROMA = 16.0f
const val ACCENT3_CHROMA = 32.0f
const val ACCENT3_HUE_SHIFT = 60.0f
const val NEUTRAL1_CHROMA = 4.0f
const val NEUTRAL2_CHROMA = 8.0f
const val GOOGLE_BLUE = 0xFF1b6ef3.toInt()
const val MIN_CHROMA = 5
public class ColorScheme(#ColorInt seed: Int, val darkTheme: Boolean) {
val accent1: List<Int>
val accent2: List<Int>
val accent3: List<Int>
val neutral1: List<Int>
val neutral2: List<Int>
constructor(wallpaperColors: WallpaperColors, darkTheme: Boolean):
this(getSeedColor(wallpaperColors), darkTheme)
val allAccentColors: List<Int>
get() {
val allColors = mutableListOf<Int>()
allColors.addAll(accent1)
allColors.addAll(accent2)
allColors.addAll(accent3)
return allColors
}
val allNeutralColors: List<Int>
get() {
val allColors = mutableListOf<Int>()
allColors.addAll(neutral1)
allColors.addAll(neutral2)
return allColors
}
val backgroundColor
get() = ColorUtils.setAlphaComponent(if (darkTheme) neutral1[8] else neutral1[0], 0xFF)
val accentColor
get() = ColorUtils.setAlphaComponent(if (darkTheme) accent1[2] else accent1[6], 0xFF)
init {
val proposedSeedCam = Cam.fromInt(seed)
val seedArgb = if (seed == Color.TRANSPARENT) {
GOOGLE_BLUE
} else if (proposedSeedCam.chroma < 5) {
GOOGLE_BLUE
} else {
seed
}
val camSeed = Cam.fromInt(seedArgb)
val hue = camSeed.hue
val chroma = camSeed.chroma.coerceAtLeast(ACCENT1_CHROMA)
val tertiaryHue = wrapDegrees((hue + ACCENT3_HUE_SHIFT).toInt())
accent1 = Shades.of(hue, chroma).toList()
accent2 = Shades.of(hue, ACCENT2_CHROMA).toList()
accent3 = Shades.of(tertiaryHue.toFloat(), ACCENT3_CHROMA).toList()
neutral1 = Shades.of(hue, NEUTRAL1_CHROMA).toList()
neutral2 = Shades.of(hue, NEUTRAL2_CHROMA).toList()
}
override fun toString(): String {
return "ColorScheme {\n" +
" neutral1: ${humanReadable(neutral1)}\n" +
" neutral2: ${humanReadable(neutral2)}\n" +
" accent1: ${humanReadable(accent1)}\n" +
" accent2: ${humanReadable(accent2)}\n" +
" accent3: ${humanReadable(accent3)}\n" +
"}"
}
companion object {
/**
* Identifies a color to create a color scheme from.
*
* #param wallpaperColors Colors extracted from an image via quantization.
* #return ARGB int representing the color
*/
#JvmStatic
#ColorInt
fun getSeedColor(wallpaperColors: WallpaperColors): Int {
return getSeedColors(wallpaperColors).first()
}
/**
* Filters and ranks colors from WallpaperColors.
*
* #param wallpaperColors Colors extracted from an image via quantization.
* #return List of ARGB ints, ordered from highest scoring to lowest.
*/
#JvmStatic
fun getSeedColors(wallpaperColors: WallpaperColors): List<Int> {
val totalPopulation = wallpaperColors.allColors.values.reduce { a, b -> a + b }
.toDouble()
val totalPopulationMeaningless = (totalPopulation == 0.0)
if (totalPopulationMeaningless) {
// WallpaperColors with a population of 0 indicate the colors didn't come from
// quantization. Instead of scoring, trust the ordering of the provided primary
// secondary/tertiary colors.
//
// In this case, the colors are usually from a Live Wallpaper.
val distinctColors = wallpaperColors.mainColors.map {
it.toArgb()
}.distinct().filter {
Cam.fromInt(it).chroma >= MIN_CHROMA
}.toList()
if (distinctColors.isEmpty()) {
return listOf(GOOGLE_BLUE)
}
return distinctColors
}
val intToProportion = wallpaperColors.allColors.mapValues {
it.value.toDouble() / totalPopulation
}
val intToCam = wallpaperColors.allColors.mapValues { Cam.fromInt(it.key) }
// Get an array with 360 slots. A slot contains the percentage of colors with that hue.
val hueProportions = huePopulations(intToCam, intToProportion)
// Map each color to the percentage of the image with its hue.
val intToHueProportion = wallpaperColors.allColors.mapValues {
val cam = intToCam[it.key]!!
val hue = cam.hue.roundToInt()
var proportion = 0.0
for (i in hue - 15..hue + 15) {
proportion += hueProportions[wrapDegrees(i)]
}
proportion
}
// Remove any inappropriate seed colors. For example, low chroma colors look grayscale
// raising their chroma will turn them to a much louder color that may not have been
// in the image.
val filteredIntToCam = intToCam.filter {
val cam = it.value
val lstar = lstarFromInt(it.key)
val proportion = intToHueProportion[it.key]!!
cam.chroma >= MIN_CHROMA &&
(totalPopulationMeaningless || proportion > 0.01)
}
// Sort the colors by score, from high to low.
val intToScoreIntermediate = filteredIntToCam.mapValues {
score(it.value, intToHueProportion[it.key]!!)
}
val intToScore = intToScoreIntermediate.entries.toMutableList()
intToScore.sortByDescending { it.value }
// Go through the colors, from high score to low score.
// If the color is distinct in hue from colors picked so far, pick the color.
// Iteratively decrease the amount of hue distinctness required, thus ensuring we
// maximize difference between colors.
val minimumHueDistance = 15
val seeds = mutableListOf<Int>()
maximizeHueDistance# for (i in 90 downTo minimumHueDistance step 1) {
seeds.clear()
for (entry in intToScore) {
val int = entry.key
val existingSeedNearby = seeds.find {
val hueA = intToCam[int]!!.hue
val hueB = intToCam[it]!!.hue
hueDiff(hueA, hueB) < i } != null
if (existingSeedNearby) {
continue
}
seeds.add(int)
if (seeds.size >= 4) {
break#maximizeHueDistance
}
}
}
if (seeds.isEmpty()) {
// Use gBlue 500 if there are 0 colors
seeds.add(GOOGLE_BLUE)
}
return seeds
}
private fun wrapDegrees(degrees: Int): Int {
return when {
degrees < 0 -> {
(degrees % 360) + 360
}
degrees >= 360 -> {
degrees % 360
}
else -> {
degrees
}
}
}
private fun hueDiff(a: Float, b: Float): Float {
return 180f - ((a - b).absoluteValue - 180f).absoluteValue
}
private fun humanReadable(colors: List<Int>): String {
return colors.joinToString { "#" + Integer.toHexString(it) }
}
private fun score(cam: Cam, proportion: Double): Double {
val proportionScore = 0.7 * 100.0 * proportion
val chromaScore = if (cam.chroma < ACCENT1_CHROMA) 0.1 * (cam.chroma - ACCENT1_CHROMA)
else 0.3 * (cam.chroma - ACCENT1_CHROMA)
return chromaScore + proportionScore
}
private fun huePopulations(
camByColor: Map<Int, Cam>,
populationByColor: Map<Int, Double>
): List<Double> {
val huePopulation = List(size = 360, init = { 0.0 }).toMutableList()
for (entry in populationByColor.entries) {
val population = populationByColor[entry.key]!!
val cam = camByColor[entry.key]!!
val hue = cam.hue.roundToInt() % 360
if (cam.chroma <= MIN_CHROMA) {
continue
}
huePopulation[hue] = huePopulation[hue] + population
}
return huePopulation
}
}
}
https://android.googlesource.com/platform/frameworks/base/+/refs/tags/android-12.1.0_r1/packages/SystemUI/monet/src/com/android/systemui/monet/ColorScheme.kt
Going off of this, mind you I haven't tested this, the code would probably look like this:
// Get user color input somehow
val userColor = getUserColor();
// Reference:
// public class ColorScheme(#ColorInt seed: Int, val darkTheme: Boolean)
// Color input needs to be an integer, assuming the color code would probably start in hex
// Using false for darkMode for example purposes
val colorScheme = ColorScheme(userColor.toInt(), false)
val accent1Colors = colorScheme.accent1
val accent2Colors = colorScheme.accent2
val accent3Colors = colorScheme.accent3
val neutral1Colors = colorScheme.neutral1
val neutral2Colors = colorScheme.neutral2
Hopefully you found this helpful 👍

may be this code snippet helpful .
// import the necessary classes
import androidx.compose.material3.*
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.platform.LocalContext
// get the user-selected input color (in this case, we're using a random color)
val inputColor = Color(0xff0099cc)
// get the context of the app
val context = LocalContext.current
// create a DynamicColorPalette instance using the input color and the context
val dynamicColorPalette = DynamicColorPalette.fromSingleColor(inputColor, context)
// create a MaterialTheme using the generated color palette
MaterialTheme(
colorPalette = dynamicColorPalette,
) {
// your app UI here
}

How to run Channel first tflite model in Android

I am able to run my custom tflite model in android but the output is totally wrong. I suspect it is due to my model needs input shape [1, 3, 640, 640] but the code makes channel last ByteBuffer. I have created tensor buffer like this TensorBuffer.createFixedSize(intArrayOf(1, 3, 640, 640), DataType.FLOAT32) but I still suspect inside the for loop, the channel is not properly set in the flat input (ByteBuffer).
I have copied this code from example where the required model shape was [1,32,32,3] (channel last). This is the reason for my doubt.
Below is my code:-
val model = YoloxPlate.newInstance(applicationContext)
val inputFeature0 = TensorBuffer.createFixedSize(intArrayOf(1, 3, 640, 640), DataType.FLOAT32)
val input = ByteBuffer.allocateDirect(640*640*3*4).order(ByteOrder.nativeOrder())
for (y in 0 until 640) {
for (x in 0 until 640) {
val px = bitmap.getPixel(x, y)
// Get channel values from the pixel value.
val r = Color.red(px)
val g = Color.green(px)
val b = Color.blue(px)
// Normalize channel values to [-1.0, 1.0]. This requirement depends on the model.
// For example, some models might require values to be normalized to the range
// [0.0, 1.0] instead.
val rf = r/ 1f
val gf = g/ 1f
val bf = b/ 1f
input.putFloat(bf)
input.putFloat(gf)
input.putFloat(rf)
}
}
inputFeature0.loadBuffer(input)
val outputs = model.process(inputFeature0)
val outputFeature0 = outputs.outputFeature0AsTensorBuffer
val flvals = outputFeature0.getFloatArray();

After using whiteboard and making and setting dim manually of the matrix, I figured it out.
It also used BGR instead of RGB as required by the model.
Working Perfectly now, here is the code (need to optimize multiple loop):-
val model = YoloxPlate.newInstance(applicationContext)
val inputFeature0 = TensorBuffer.createFixedSize(intArrayOf(1, 3, 640, 640), DataType.FLOAT32)
val input = ByteBuffer.allocateDirect(640*640*3*4).order(ByteOrder.nativeOrder())
for (y in 0 until 640) {
for (x in 0 until 640) {
val px = bitmap.getPixel(x, y)
val b = Color.blue(px)
val bf = b/ 1f
input.putFloat(bf)
}
}
for (y in 0 until 640) {
for (x in 0 until 640) {
val px = bitmap.getPixel(x, y)
val g = Color.green(px)
val gf = g/ 1f
input.putFloat(gf)
}
}
for (y in 0 until 640) {
for (x in 0 until 640) {
val px = bitmap.getPixel(x, y)
val r = Color.red(px)
val rf = r/ 1f
input.putFloat(rf)
}
}
inputFeature0.loadBuffer(input)
val outputs = model.process(inputFeature0)
val outputFeature0 = outputs.outputFeature0AsTensorBuffer
val flvals = outputFeature0.getFloatArray();

Retrieving frequency ranges from android visualizer FFT

I am currently attempting to do something very similar (or maybe the same) as the following question:
Getting variable frequency ranges with androids visualizer class
However, the selected answer has a few bugs, and I'm not a DSP/Audio expert at all and I'm learning as I go.
My goal is to break an FFT that I'm getting from Android Visualizer class into frequency bands. Specifically, these bands:
0Hz - 400Hz
400Hz - 900Hz
900Hz - 1500Hz
1500Hz - 2300Hz
2300Hz - 3400Hz
3400Hz - 5000Hz
5000Hz - 7300Hz
7300Hz - 12000Hz
I have the following code, at the top of my class:
private val targetEndpoints = listOf(0f, 400f, 900f, 1500f, 2300f, 3400f, 5000f, 7300f, 12000f)
private const val CAPTURE_SIZE = 1024
and then, in the method where I'm trying to get the frequency bands for the current track in MediaPlayer:
val mp = mediaPlayer!!
val audioSessionId = mp.getAudioSessionId()
val visualizer: Visualizer = Visualizer(audioSessionId)
val captureSizeRange = Visualizer.getCaptureSizeRange().let { it[0]..it[1] }
val captureSize = CAPTURE_SIZE.coerceIn(captureSizeRange)
val captureRate: Int = Visualizer.getMaxCaptureRate()
val isWaveFormRequested: Boolean = false
val isFFTRequested: Boolean = true
visualizer.setCaptureSize(captureSize)
val frequencyOrdinalRanges: List<IntProgression> =
targetEndpoints.zipWithNext { a, b ->
val startOrdinal = 1 + (captureSize * a / samplingRate).toInt()
val endOrdinal = (captureSize * b / samplingRate).toInt()
startOrdinal downTo endOrdinal
}
Now this is the point where things are getting a little murky for me because, like I said, I am no Audio expert.
frequencyOrdinalRanges is a List with IntProgressions that go 1 -> 0
For the audio file that I'm using:
captureSize = 1024
samplingRate = 44100000
With those numbers and my frequency bands, pretty much guarantees that the startOrdinal will always be 1, and endOrdinal will always be 0.
So my frequencyOrdinalRanges looks like this:
[1 downTo 0 step 1, 1 downTo 0 step 1, 1 downTo 0 step 1, 1 downTo 0 step 1, 1 downTo 0 step 1, 1 downTo 0 step 1, 1 downTo 0 step 1]
Then I've got an Listener with a capture rate of 20000 milihertz:
visualizer.setDataCaptureListener(listener, captureRate, isWaveFormRequested, isFFTRequested)
The values for the above call are as follows:
captureRate = 2000 // in milihertz
isWaveFormRequested = false
isFFTRequested = true
The onFftDataCapture of the listener object looks as follows:
override fun onFftDataCapture(visualizer: Visualizer, bytes: ByteArray, samplingRate: Int) {
var output = DoubleArray(frequencyOrdinalRanges.size)
for ((i, ordinalRange) in frequencyOrdinalRanges.withIndex()) {
var logMagnitudeSum = 0.0
for (k in ordinalRange) {
val fftIndex = k * 2
val currentByte = bytes[fftIndex].toDouble()
val nextByte = bytes[fftIndex + 1].toDouble()
val hypot = Math.hypot(currentByte, nextByte)
val logHypot = Math.log10(hypot)
logMagnitudeSum += logHypot
val result = (logMagnitudeSum / (ordinalRange.last - ordinalRange.first + 1)).toDouble()
output[i] = result
}
// do something else with output
}
Now the problem I'm facing with onFftDataCapture is that this line:
val hypot = Math.hypot(currentByte, nextByte)
it often evaluates to 0, thus making the following line evaluate to -Infinity and ultimately giving me an array full of Infinity values which I can't do anything with.
This leads me to believe that I am doing something very wrong, but I am not sure what or how to fix it.
This answer looks more or less what I am trying to do, but then again, I am no expert in audio analysis, so all the finer details totally escape me.
The way to extract 10-band equalization information from mp3 format
Can someone tell me what am I doing wrong? or what am I missing?

The problem with my code was quite silly... I was using samplingRate in milihertz... the formula expects the sampling rate to be in Hertz.
Dividing samplingRate by 1000 fixed the problems.
This changes to:
val samplingRateInHz = samplingRate / 1000
val frequencyOrdinalRanges: List<IntRange> =
targetEndpoints.zipWithNext { a, b ->
val startOrdinal = 1 + (captureSize * a / samplingRateInHz).toInt()
val endOrdinal = (captureSize * b / samplingRateInHz).toInt()
startOrdinal..endOrdinal
}

How to calculate 2D rotation from 3D matrix obtained from sensor TYPE_ROTATION_VECTOR (to lock one axis rotation)

I want to build application which uses phone sensor fusion to rotate 3D object in OpenGL. But I want the Z axis to be locked therefore I want basically to apply 2D rotation to my model.
In order to build 2D rotation from the 3D matrix I get from SensorManager.getRotationMatrixFromVector() I build rotation matrix for each axis as explained on the picture:
I want to apply 2D rotation matrix which would be R=Ry*Rx however this seems not working. But applying R=Rz*Ry works as expected. My guess is that Rx values are not correct.
To build the Rz, Ry, Rx matrices I looked up values used by SensorManager.getOrientation() to calculate angles:
values[0] = (float) Math.atan2(R[1], R[4]);
values[1] = (float) Math.asin(-R[7]);
values[2] = (float) Math.atan2(-R[6], R[8]);
So here is how I build matrices for each axis:
private val degConst = 180/Math.PI
private var mTempRotationMatrix = MatrixCalculations.createUnit(3)
override fun onSensorChanged(event: SensorEvent?) {
val sensor = event?.sensor ?: return
when (sensor.type) {
Sensor.TYPE_ROTATION_VECTOR -> {
SensorManager.getRotationMatrixFromVector(mTempRotationMatrix, event.values)
val zSinAlpha = mTempRotationMatrix[1]
val zCosAlpha = mTempRotationMatrix[4]
val ySinAlpha = -mTempRotationMatrix[6]
val yCosAlpha = mTempRotationMatrix[8]
val xSinAlpha = -mTempRotationMatrix[7]
val xCosAlpha = mTempRotationMatrix[4]
val rx = MatrixCalculations.createUnit(3)
val ry = MatrixCalculations.createUnit(3)
val rz = MatrixCalculations.createUnit(3)
val sina = xSinAlpha
val cosa = xCosAlpha
val sinb = ySinAlpha
val cosb = yCosAlpha
val siny = zSinAlpha
val cosy = zCosAlpha
rx[4] = cosa
rx[5] = -sina
rx[7] = sina
rx[8] = cosa
ry[0] = cosb
ry[2] = sinb
ry[6] = -sinb
ry[8] = cosb
rz[0] = cosy
rz[1] = -siny
rz[3] = siny
rz[4] = cosy
val ryx = MatrixCalculations.multiply(ry, rx)
mTempRotationMatrix = ryx
MatrixCalculations.copy(mTempRotationMatrix, mRenderer.rotationMatrix)
LOG.info("product: [" + mRenderer.rotationMatrix.joinToString(" ") + "]")
val orientation = FloatArray(3)
SensorManager.getOrientation(mTempRotationMatrix, orientation)
LOG.info("yaw: " + orientation[0] * degConst + "\n\tpitch: " + orientation[1] * degConst + "\n\troll: " + orientation[2] * degConst)
}
The question is what I am doing wrong and what values to use for the Rx matrix. Is my math applied to this problem broken? Also interesting would be to know how value of event.values[3] is related to build rotation matrix in SensorManager.getRotationMatrixFromVector().
In theory operation R=Ry*Rx should give me correct rotation but it is not the case.

Inaccurate prediction in MLKit custom model

Trying to use a retrained MobileNet model to predict dog breeds, but when using the model through Firebase MLKit, it is unable to correctly predict the dog breed. The desktop model and the tflite model are both able to correctly predict the breed, but using the same image of a pug, the desktop model and the tflite model (on desktop) are 87.8% confident that it is a pug; whereas on MLKit, the confidence is 1.47x10-2% confident.
I'm suspecting the issue is in my preprocessing of the image in the app code. The docs show how to scale the pixels in the range -1.0, 1.0; which according to the code for the keras image preprocessing function is what is required.
Here is my infer(iStream) function where I think the error may lie. Any help is greatly appreciated, this is driving me crazy.
private fun infer(iStream: InputStream?) {
Log.d("ML_TAG", "infer")
val bmp = Bitmap.createScaledBitmap(BitmapFactory.decodeStream(iStream), 224, 224, true)
i.setImageBitmap(bmp)
val bNum = 0
val input = Array(1) { Array(224) { Array(224) { FloatArray(3) } } }
for (x in 0..223) {
for (y in 0..223) {
val px = bmp.getPixel(x, y)
input[bNum][x][y][0] = (Color.red(px) - 127) / 255.0f
input[bNum][x][y][1] = (Color.green(px) - 127) / 255.0f
input[bNum][x][y][2] = (Color.blue(px) - 127) / 255.0f
}
}
val inputs = FirebaseModelInputs.Builder()
.add(input)
.build()
interpreter.run(inputs, ioOpts).addOnSuccessListener { res ->
val o = res.getOutput<kotlin.Array<FloatArray>>(0)
val prob = o[0]
val r = BufferedReader(InputStreamReader(assets.open("retrained_labels.txt")))
val arrToSort = arrayListOf<Pair<String, Float>>()
val rArr = r.readLines()
for (i in prob.indices) {
val p = Pair(rArr[i], prob[i])
arrToSort.add(p)
}
val sortedList = arrToSort.sortedWith(compareByDescending {it.second})
val topFive = sortedList.slice(0..4)
arrToSort.forEach {
if (it.first == "pug") {
Log.i("ML_TAG", "Pug: ${it.second}")
}
}
sortedList.forEach {
if(it.first == "pug") {
Log.i("ML_TAG", "Pug: ${it.second}")
}
}
topFive.forEach {
Log.i("ML_TAG", "${it.first}: ${it.second}")
}
}
.addOnFailureListener { res ->
Log.e("ML_TAG", res.message)
}
}

I think (Color.red(px) - 127) / 255.0f scales to [-0.5, 0.5]. Does (Color.red(px) - 127) / 128.0f produce better results?