I want to make a waveform drawing for an audio recorder in Android. The usual one with lines/bars, like this one:
More importantly, I want it live, while the song is being recorded. My app already computes the RMS through AudioRecord. But I am not sure which is the best approach for the actual drawing in terms of processing, resources, battery, etc.
The Visualizer does not show anything meaningful, IMO (are those graphs more or less random stuff??).
I've seen the canvas approach and the layout approach (there are probably more?). In the layout approach you add thin vertical layouts in a horizontal layout. The advantage is that you don't need to redraw the whole thing each 1/n secs, you just add one layout each 1/n secs... but you need hundreds of layouts (depending on n). In the canvas layout, you need to redraw the whole thing (right??) n times per second. Some even create bitmaps for each drawing...
So, which is cheaper, and why? Is there anything better nowadays? How much frequency update (i.e., n) is too much for generic low end devices?
EDIT1
Thanks to the beautiful trick #cactustictacs taught me in his answer, I was able to implement this with ease. Yet, the image is strangely rendered kind of "blurry by movement":
The waveform runs from right to left. You can easily see the blur movement, and the left-most and right-most pixels get "contaminated" by the other end. I guess I can just cut both extremes...
This renders better if I make my Bitmap bigger (i.e., making widthBitmap bigger), but then the onDraw will be heavier...
This is my full code:
package com.floritfoto.apps.ave;
import android.content.Context;
import android.graphics.Bitmap;
import android.graphics.Canvas;
import android.graphics.Paint;
import android.graphics.Rect;
import android.util.AttributeSet;
import java.util.Arrays;
public class Waveform extends androidx.appcompat.widget.AppCompatImageView {
//private float lastPosition = 0.5f; // 0.5 for drawLine method, 0 for the others
private int lastPosition = 0;
private final int widthBitmap = 50;
private final int heightBitmap = 80;
private final int[] transpixels = new int[heightBitmap];
private final int[] whitepixels = new int[heightBitmap];
//private float top, bot; // float for drawLine method, int for the others
private int aux, top;
//private float lpf;
private int width = widthBitmap;
private float proportionW = (float) (width/widthBitmap);
Boolean firstLoopIsFinished = false;
Bitmap MyBitmap = Bitmap.createBitmap(widthBitmap, heightBitmap, Bitmap.Config.ARGB_8888);
//Canvas canvasB = new Canvas(MyBitmap);
Paint MyPaint = new Paint();
Paint MyPaintTrans = new Paint();
Rect rectLbit, rectRbit, rectLdest, rectRdest;
public Waveform(Context context, AttributeSet attrs) {
super(context, attrs);
MyPaint.setColor(0xffFFFFFF);
MyPaint.setStrokeWidth(1);
MyPaintTrans.setColor(0xFF202020);
MyPaintTrans.setStrokeWidth(1);
Arrays.fill(transpixels, 0xFF202020);
Arrays.fill(whitepixels, 0xFFFFFFFF);
}
public void drawNewBar() {
// For drawRect or drawLine
/*
top = ((1.0f - Register.tone) * heightBitmap / 2.0f);
bot = ((1.0f + Register.tone) * heightBitmap / 2.0f);
// Using drawRect
//if (firstLoopIsFinished) canvasB.drawRect(lastPosition, 0, lastPosition+1, heightBitmap, MyPaintTrans); // Delete last stuff
//canvasB.drawRect(lastPosition, top, lastPosition+1, bot, MyPaint);
// Using drawLine
if (firstLoopIsFinished) canvasB.drawLine(lastPosition, 0, lastPosition, heightBitmap, MyPaintTrans); // Delete previous stuff
canvasB.drawLine(lastPosition ,top, lastPosition, bot, MyPaint);
*/
// Using setPixel (no tiene sentido, mucho mejor setPixels.
/*
int top = (int) ((1.0f - Register.tone) * heightBitmap / 2.0f);
int bot = (int) ((1.0f + Register.tone) * heightBitmap / 2.0f);
if (firstLoopIsFinished) {
for (int i = 0; i < top; ++i) {
MyBitmap.setPixel(lastPosition, i, 0xFF202020);
MyBitmap.setPixel(lastPosition, heightBitmap - i-1, 0xFF202020);
}
}
for (int i = top ; i < bot ; ++i) {
MyBitmap.setPixel(lastPosition,i,0xffFFFFFF);
}
//System.out.println("############## "+top+" "+bot);
*/
// Using setPixels. Works!!
top = (int) ((1.0f - Register.tone) * heightBitmap / 2.0f);
if (firstLoopIsFinished)
MyBitmap.setPixels(transpixels,0,1,lastPosition,0,1,heightBitmap);
MyBitmap.setPixels(whitepixels, top,1, lastPosition, top,1,heightBitmap-2*top);
lastPosition++;
aux = (int) (width - proportionW * (lastPosition));
rectLbit.right = lastPosition;
rectRbit.left = lastPosition;
rectLdest.right = aux;
rectRdest.left = aux;
if (lastPosition >= widthBitmap) { firstLoopIsFinished = true; lastPosition = 0; }
}
#Override
protected void onSizeChanged(int w, int h, int oldw, int oldh) {
super.onSizeChanged(w, h, oldw, oldh);
width = w;
proportionW = (float) width/widthBitmap;
rectLbit = new Rect(0, 0, widthBitmap, heightBitmap);
rectRbit = new Rect(0, 0, widthBitmap, heightBitmap);
rectLdest = new Rect(0, 0, width, h);
rectRdest = new Rect(0, 0, width, h);
}
#Override
protected void onDraw(Canvas canvas) {
super.onDraw(canvas);
drawNewBar();
canvas.drawBitmap(MyBitmap, rectLbit, rectRdest, MyPaint);
canvas.drawBitmap(MyBitmap, rectRbit, rectLdest, MyPaint);
}
}
EDIT2
I was able to prevent the blurring just using null as Paint in the canvas.drawBitmap:
canvas.drawBitmap(MyBitmap, rectLbit, rectRdest, null);
canvas.drawBitmap(MyBitmap, rectRbit, rectLdest, null);
No Paints needed.
Your basic custom view approach would be to implement onDraw and redraw your current data each frame. You'd probably keep some kind of circular Buffer holding your most recent n amplitude values, so each frame you'd iterate over those, and use drawRect to draw the bars (you'd calculate things like width, height scaling, start and end positions etc in onSizeChanged, and use those values when defining the coordinates for the Rects).
That in itself might be fine! The only way you can really tell how expensive draw calls are is to benchmark them, so you could try this approach out and see how it goes. Profile it to see how much time it takes, how much the CPU spikes etc.
There are a few things you can do to make onDraw as efficient as possible, mostly things like avoiding object allocations - so watch out for loop functions that create Iterators, and in the same way you're supposed to create a Paint once instead of creating them over and over in onDraw, you could reuse a single Rect object by setting its coordinates for each bar you need to draw.
Another approach you could try is creating a working Bitmap in your custom view, which you control, and calling drawBitmap inside onDraw to paint it onto the Canvas. That should be a pretty inexpensive call, and it can easily be stretched as required to fit the view.
The idea there, is that very time you get new data, you paint it onto the bitmap. Because of how your waveform looks (like blocks), and the fact you can scale it up, really all you need is a single vertical line of pixels for each value, right? So as the data comes in, you paint an extra line onto your already-existing bitmap, adding to the image. Instead of painting the entire waveform block by block every frame, you're just adding the new blocks.
The complication there is when you "fill" the bitmap - now you have to "shift" all the pixels to the left, dropping the oldest ones on the left side, so you can draw the new ones on the right. So you'll need a way to do that!
Another approach would be something similar to the circular buffer idea. If you don't know what that is, the idea is you take a normal buffer with a start and an end, but you treat one of the indices as your data's start point, wrap around to 0 when you hit the last index of the buffer, and stop when you hit the index you're calling your end point:
Partially filled buffer:
|start
123400
|end
Data: 1234
Full buffer:
|start
123456
|end
Data: 123456
After adding one more item:
|start
723456
|end
Data: 234567
See how once it's full, you shift the start and end one step "right", wrapping around if necessary? So you always have the most recent 6 values added. You just have to handle reading from the correct index ranges, from start -> lastIndex and then firstIndex -> end
You could do the same thing with a bitmap - start "filling" it from the left, increasing end so you can draw the next vertical line. Once it's full, start filling from the left by moving end there. When you actually draw the bitmap, instead of drawing the whole thing as-is (723456) you draw it in two parts (23456 then 7). Make sense? When you draw a bitmap to the canvas, there's a call that takes a source Rect and a destination one, so you can draw it in two chunks.
You could always redraw the bitmap from scratch each frame (clear it and draw the vertical lines), so you're basically redrawing your whole data buffer each time. Probably still faster than the drawRect approach for each value, but honestly not much easier than the "treat the bitmap as another circular buffer" method. If you're already managing one circular buffer, it's not much more work - since the buffer and the bitmap will have the same number of values (horizontal pixels in the bitmap's case) you can use the same start and end values for both
You would never do this with layouts. Layouts are for premade components. They're high level combinations of components and you don't want to dynamically add or remove views from it frequently. For this, you use a custom view with a canvas. Layouts aren't even an option for something like this.
i am new to unity and need some help regarding creating a background that will look something like this (A bit jittery because its a gif), i want it to be like fill every screen size and have size 1/8th of the screen (the black box):
You can use the following setup:
First the image should have borders like this one and set its Wrap mode to reapeat in the import settings
Your background should be a ScreenSpace Overlay Canvas (depends on your setup ofcourse)
Within that Canvas have a RawImage object, use your image as Texture and add this component to it
[RequireComponent(typeof(RawImage))]
public class BackgroundController : MonoBehaviour
{
[Header("References")]
[SerializeField] private RectTransform _rectTransform;
[SerializeField] private RectTransform _parentRectTransform;
[SerializeField] private RawImage _image;
[Header("Settings")]
[SerializeField] private Vector2 repeatCount;
[SerializeField] private Vector2 scroll;
[SerializeField] private Vector2 offset;
private void Awake()
{
if (!_image) _image = GetComponent<RawImage>();
_image.uvRect = new Rect(offset, repeatCount);
}
// Start is called before the first frame update
private void Start()
{
if (!_rectTransform) _rectTransform = GetComponent<RectTransform>();
if (!_parentRectTransform) _parentRectTransform = GetComponentInParent<RectTransform>();
SetScale();
}
// Update is called once per frame
private void Update()
{
#if UNITY_EDITOR
// Only done in the Unity editor since later it is unlikely that your screensize changes
SetScale();
#endif
offset += scroll * Time.deltaTime;
_image.uvRect = new Rect(offset, repeatCount);
}
private void SetScale()
{
// get the diagonal size of the screen since the parent is the Canvas with
// ScreenSpace overlay it is always fiting the screensize
var parentCorners = new Vector3[4];
_parentRectTransform.GetLocalCorners(parentCorners);
var diagonal = Vector3.Distance(parentCorners[0], parentCorners[2]);
// set width and height to at least the diagonal
_rectTransform.sizeDelta = new Vector2(diagonal, diagonal);
}
}
This first scales the RawImage to fit the diagonal size of the parent. Since it is already fitting the screen this gets us the screen sizes => always fills the entire screen, no matter what the scales or rotation are (as long as your RawImage is on the center of the screen ofcourse).
Using the repeatCount you define how often the texture should be on the background.
Then using the scroll you can define how fast and in which direction the background should scroll. The script basically simply updates the RawImage.uvRect every frame.
Finally you simply rotate the RawImage so the scroll goes in the final direction you want
So you want some kind of infinity scrolling background?
Here is some simple way to create it.
1) You will need tilable image (that can be connected one side to other seamlessly). You can use one frame from your gif. Add it to your assets (just drag and drop it there).
2) In your Unity scene create new Quad object (GameObject->3d Object->Quad)
3) Drag and drop your image from your assets window right onto your Quad. That will apply texture to it.
4) Create simple script on your Quad object. I called mine RollerScript
using System.Collections;
using UnityEngine;
public class RollerScript : MonoBehaviour
{
public float speed = 2f;
public MeshRenderer renderer;
void Update()
{
Vector2 offset = new Vector2(Time.time * speed, Time.time * speed);
renderer.material.mainTextureOffset = offset;
}
}
5) Go back to Editor and assign renderer field (drag your Quad object from Hierarchy to that field)
6) Hit Play and adjust speed parameter in your script editor window. Your texture will scroll diagonally to right-top (as on your gif). If you want another direction you can change this line:
Vector2 offset = new Vector2(Time.time * speed, Time.time * speed);
Set x or y value of Vector2 to zero if you want NO scrolling horizontaly/vertically. Change x or y value to -x or -y if you want to scroll in opposite direction.
I'm new in Android Game Development and I started a simple Game in that a Droid can jump over incoming boxes.
I want to call my droid.jump() method in the surface view with an onTouchEvent (just by a simple tap on the screen)
I created a class called Droid:
public class Droid {
// Log Tag for Debugging
public static final String LOG_TAG = "_1Projekt";
private Bitmap bitmap; // the actual bitmap
private int x; // the X coordinate
private int y; // the Y coordinate
private boolean touched; // if droid is touched/picked up
private Speed speed; // the speed with its directions
private long mLastTime;
public Droid(Bitmap bitmap, int x, int y) {
this.bitmap = bitmap;
this.x = x;
this.y = y;
this.speed = new Speed();
}
.......
...
And the jump() method is my problem. I would like to have a smooth jump but I don’t know how to calculate this with the current System Time.
My idea was that the droid should update it Y position every –TimePeriod- and should start with a fast velocity and then decrease it to 0 to get a smooth jump.
But I don’t know how to calculate this in my while loop.
My current jump():
public void jump() {
Log.d(LOG_TAG, "Jumping");
long now = System.currentTimeMillis();
int elapsedTime = 100;
int jump = y-30;
while(y > jump)
{
if(System.currentTimeMillis() > now + elapsedTime)
{
now = now + elapsedTime;
elapsedTime -=3;
y = y-1;
}
}
}
Up to know I only implemented the "up" part of Jump.
Thank you for your answers! Greetings DroidDude
You may want to look here (the third post):
Before the mainloop, have
//Get the current time
timeStep = System.currentTimeMillis();
And then do your stuff. Then before the loop goes back to start, have
// Hold to lock at FPS
while(System.currentTimeMillis()-timeStep < 1000/60);
Where 60 is the frames per second to run.
This method also allows you to get the difference in times after the
while loop in order to find out how much time it took to render one
frame. Using this, you could have a variable that you can multiply all
your increments by to get frame-independent speed.
For example, if you divide the number of milliseconds it took to
handle the frame by 16.666, the number will equal 1 when the program
runs at about 60 FPS. When the frame rate is lower, it takes more
milliseconds to render the frame and that factor gets bigger.
Be sure to put it BEFORE repaint however. This is the way video games
are timed also. An example is shown in my video game base applet code
snippet under JAVA.
So I have a bitmap that I have loaded from a resource file (an PNG image):
Bitmap map = BitmapFactory.decodeResource(getResources(), R.drawable.wave);
If I draw this bitmap only once using canvas.drawBitmap(...); then there is no problem. However, If I draw that very same bitmap multiple times, then the picture keeps flashing back and forth, not steady like before.
I suspected that I cannot use the same bitmap more than once so I tried to load the image into a new bitmap every time when I want to draw the same picture, but it does not help, the behavior still persists.
The program is complicated, but basically, I want to draw a ocean wave. I have a image of a small wave. To make the effect of the wave moving from the left edge of the screen to the right edge. I keep track of the position of the left edge of the bitmap.
// The ocean.
private ArrayList<Wave> waves;
// Draw the waves and update their positions.
for (int i = 0; i < this.waves.size(); i++)
{
Wave wave = this.waves.get(i);
// Go through each of the sub-waves of this current wave.
for (int j = 0; j < wave.getSubWaveEdges().size(); j++)
{
// Get the sub wave.
final float subWaveEdge = wave.getSubWaveEdges().get(j);
canvas.drawBitmap( wave.getSubWave(j), subWaveEdge, 40, brush);
wave.setSubWaveEdge(j, subWaveEdge + (float) 0.5);
}
// Update this current wave.
wave.update();
// If the wave has passed the left edge of the screen then add a new sub-wave.
if (wave.getFarthestEdge() >= 0)
wave.addSubWaveEdges(wave.getFarthestEdge() - this.getWidth());
}
If the left edge of a bitmap is inside the screen then I create a new bitmap from the same image file and draw. Here is the class Wave:
private class Wave
{
private Bitmap wave;
private float farthestEdge;
private ArrayList<Float> subWaveEdges;
private ArrayList<Bitmap> subWaves;
public Wave(Bitmap wave)
{
this.wave = wave;
this.farthestEdge = 0;
this.subWaveEdges = new ArrayList<Float>();
this.subWaves = new ArrayList<Bitmap>();
}
public Bitmap getWave ()
{ return this.wave; }
public void setWave (Bitmap wave)
{ this.wave = wave; }
public float getFarthestEdge ()
{ return this.farthestEdge; }
public void setFarthestEdge (final float furthestEdge)
{ this.farthestEdge = furthestEdge; }
public ArrayList<Float> getSubWaveEdges ()
{ return subWaveEdges; }
public void setSubWaveEdge (final int index, final float value)
{
this.subWaveEdges.remove(index);
this.subWaveEdges.add(value);
}
public void addSubWaveEdges (final float edge)
{
this.subWaveEdges.add(edge);
Bitmap newSubWave = BitmapFactory.decodeResource(getResources(), R.drawable.wave);
newSubWave = Bitmap.createScaledBitmap(newSubWave, MasterView.this.getWidth(), newSubWave.getHeight(), true);
this.subWaves.add(newSubWave);
}
public Bitmap getSubWave(final int index)
{ return this.subWaves.get(index); }
public void update ()
{
// Check to see if there is any sub-wave going outside of the screen.
// If there is then remove that wave.
for (int index = 0; index < this.subWaveEdges.size(); index++)
if (this.subWaveEdges.get(index) > MasterView.this.getWidth())
{
this.subWaveEdges.remove(index);
this.subWaves.remove(index);
}
// Set the farthest edge to the other side of the screen.
this.farthestEdge = MasterView.this.getWidth();
// Get the farthest edge of the wave.
for (int index = 0; index < this.subWaveEdges.size(); index++)
if (this.subWaveEdges.get(index) < this.farthestEdge)
this.farthestEdge = this.subWaveEdges.get(index);
}
}
Another suspicion that I have is that may be when I create two bitmaps from the same resource file, the pixels of the image are divided among two bitmaps, meaning that each bitmap only gets part of the pixels, not all. I am suspecting this because when the bitmaps are drawn, the parts where they overlaps are drawn steadily, no flashing.
Anyone has stumbled upon this problem and know how to fix?
Thanks,
Viktor Lannér, Thank you for helping, but I don't think that's the problem. I understand it is hard to read my codes since it is only a small piece of the big program.
However, I found the problem: This is not mentioned in my original question, but in order to simulate the two waves moving after one another, I have to draw the next wave as soon as the first wave enters the screen. However, each wave is longer than the width of the screen. Therefore, I have to draw the next wave from "outside" the screen if you know what I mean. It means that the next wave is drawn from a negative x-coordinate from outside the screen:
// If the wave has passed the left edge of the screen then add a new sub-wave.
if (wave.getFarthestEdge() >= 0)
wave.addSubWaveEdges(wave.getFarthestEdge() - this.getWidth());
And I found out that it does not like this. This is what causes the flashing back and forth.
In order to fix this, instead of drawing the next wave from outside the screen, I use this method:
canvas.drawBitmap (Bitmap bitmap, Rect source, Rect destination, Paint paint)
This method allows you to specify a rectangular region on the bitmap to be drawn to the screen and a rectangular region on the screen where that part of the bitmap will be drawn over. I use this method to draw the next wave. As the next wave moves into the screen, I change the "source" and "destination" appropriately to draw parts of the bitmap.
I just wanted to say that I had an issue where the images on my canvas were flashing back and forth, or, flashing between black and my first frame until I made a movement, almost as if the canvas was rapidly switching between its current and last image.
This might have had something to do with your situation, and to fix it I found out that it was because I was locking the canvas every frame, even when I had nothing to draw. For whatever reason, that lock, I think, created this situation.
I got around it by doing something like this:
if (needToRedraw == true) {
canvas = mSurfaceHolder.lockCanvas(null);
... logic to eventually draw on that canvas ...
}
Before canvas.drawBitmap(...) call; try to use canvas.drawColor(Color.BLACK) to clear the Canvas from previous drawings.
Sample code:
// Stuff.
canvas.drawColor(Color.BLACK);
canvas.drawBitmap(wave.getSubWave(j), subWaveEdge, 40, brush);
// Stuff.
Problem seemed very simple to me at first but now I am stuck.
Scenario
I want to move a image on the screen, on a certain path I create. Moving this image is being made on a thread, something like:
#Override
public void run() {
Canvas c;
while (run) {
c = null;
try {
c = panel.getHolder().lockCanvas(null);
synchronized (panel.getHolder()) {
panel.updateImageCoordinates();
panel.onDraw(c);
}
} finally {
if (c != null) {
panel.getHolder().unlockCanvasAndPost(c);
}
}
}
for the Image I want to move I have a List with main points where it should go. Each coordinate has:
public class Coordinates {
private int x = 0;
private int y = 0;
private int speedX=0;
private int speedY=0;
}
For example, my first point is -5;-30 and I need to get to second point 50.50. The calculation of next coordinates to draw the image is made on updateImageCoordinates(). My problem is that I don't know how to calculate speedX and speedY so that I get from point A to point B on a straight line. Basically for each execution of updateImageCoorindates() I need to do:
image.currentX= image.currentX+speedX;
image.currentY= image.currentY+speedY
//Check if I reached the B point. if so, move to next point.
I don't know based on knowing the coordinates, how I can calculate the speed on x and Y directions.
I attach a image for exemplification. Any help is appreciated.
I'm not shure if I've understood your question clearly...
If you are looking for function which will translate PointA into point on line A-B.
Line containing both points will have equation:
-30 = -5*a + b
and
50 = 50*a + b
so
b = -250/11
a = 16/11
so to find next point you have to:
check if x of next point is at left (-1) or right (+1) of the destination point
and calculate next point by:
image.currentX= image.currentX+((-1 or +1)*movement_speed);
image.currentY= image.currentY+16/11*(-1 or +1)*movement_speed + (-250/11)
You'll find the API Demos for Animation useful I think. In particular, check out Custom Evaluator.