I wanna develop a pictionary style app. I've figured out the drawing part (using canvas, paint and related libraries) on the device, and now I need to update the drawings in real time on all devices that are connected.
The approach I have in mind is to take screenshots at very close intervals and upload them to the server (Firebase). The app will constantly check for server side updates. I know this is a horrible way to keep things in relative synchronization, is there any other way I can do this?
Maybe like a video stream or something.
If you are drawing using paths, then you could send a list of paths to the other devices and redraw them there.
I do not think there is a fast way to convert a series of bitmaps into a video(by bitmaps I mean images that are generated using the Android canvas).
If you do your drawing using OpenGL, then you could convert the output of an OpenGL surface into a video using a video encoder
Related
I'm working on a robot that is controller via the VR headset and sends a real-time video feed to the headset.
I've chosen to go the native way on Android and now have everything I need to receive the video stream and encode it (using GStreamer) and also to send the control data to the robot via UDP.
The last thing to do (and the one I most struggle with as I nave no prior experience with computer graphics) is to draw the image (encoded camera feed) to the screen. In the last few days, I've been reading stuff about how Vulkan and OpenGL works, I've also went through the examples provided in Oculus Mobile SDK (mainly VRCubeWorld_SurfaceView) but that's way to complex for what I need, I've tried to simplify it so I could just draw two images, but then I thought.
Do I even need any of that? And this question might sound stupid, but I really don't have any prior experience doing this.
I mean, the example is using OpenGL to basically compute all the layers of the 3D scene, apply colors and then fuse them together to get a final frame that is passed to VR_API via the function:
vrapi_SubmitFrame2(appState.Ovr, &frameDesc);
Can I just take those images, and somehow force them into the frameDesc structure to skip the whole OpenGL pipeline? If so, can anyone knowledgeable enough point me to a working solution?
I don't need any kind of panning over the images, just to render them. Later I'll be using head sensor data, but it won't actually do anything with the "scene".
I'm working on an application for sharing SurfaceView's draws with other android devices in real time (as a streaming). So all the connected devices on the net should be able to get the same canvas, draw in real time and see the modifications of each others.
I didn't found a way to do that. I thought about capturing screen shots and share them, but it will not allow multi-users modification.
How can I manage that? Is there an API or something like that?
I couldn't find from where to start.
After a lot of searching and days of experiments I haven't found a straight-forward solution.
I'm developing an app that user will interact with a pet on the screen and i want to let him save it as video.
Is there any "simple" way to capture the screen of the app itself?
I found a workaround (to save some bitmaps every second and then pass them to an encoder) but it seems too heavy. I will happy even with a framerate of 15fps
It seems to be possible, i.e. there is a similar app that does this, its called "Talking Tom"
It really depends on the way you implement your "pet view". Are you drawing on a Canvas? OpenGl ES? Normal Android view hierarchy?
Anyway, there is no magical "recordScreenToVideo()" like one of the comments said.
You need to:
Obtain bitmaps representing your "frames".
This depends on how you implement your view. If you draw yourself (Canvas or OpenGL), then save your raw pixel buffers as frames.
If you use the normal view hierarchy, subclass Android's onDraw and save the "frames" that you get on the canvas. The frequency of the system's call to onDraw will be no less than the actual "framerate" of the screen. If needed, duplicate frames afterwards to supply a 15fps video.
Encode your frames. Seems like you already have a mechanism to do that, you just need it to be more efficient.
Ways you can optimize encoding:
Cache your bitmaps (frames) and encode afterwards. This will work
only if your expected video will be relatively short, otherwise
you'll get out of storage.
Record only at the framerate that your app actually generates (depending on the way you draw) and use an encoder parameter to generate a 15fps video (without actually supplying 15 frames per second).
Adjust quality settings to current device. Can be done by performing a hidden CPU cycle test on app startup and defining some thresholds.
Encode only the most relevant portion of the screen.
Again, really depending on the way you implement - if you can save some "history data", and then convert that to frames without having to do it in real time, that would be best.
For example, "move", "smile", "change color" - or whatever your business logic is, since you didn't elaborate on that. Your "generate movie" function will animate this history data as a frame sequence (without drawing to the screen) and then encode.
Hope that helps
(I tried to stuff the question with keywords in case someone else has this issue - I couldn't find much help.)
I have a custom View in Android that contains an LED bargraph that displays levels received via socket communication. It's basically just a clipped image. The higher the level, the less clipped the image is.
When I update the level and then invalidate the View, some devices seem to "collect" multiple updates and render them in chunks. The screen visibly hesitates for say 1/10th of a second, then rapidly paints multiple frames, and then hesitates again. It looks like it's overwhelmed and dropping frames.
However, when changing another UI control on the screen, the LED bargraph paints much more frequently and smoothly. I'm thinking Android is trying to help me by "collecting" multiple invalidations and then doing them all at once. Perhaps by manipulating controls, I'm "increasing" my frame rate simply by giving it "more to do" so it delays less between actual paints.
Unlike animation (with smooth transitions) I want to show the absolute latest value as quickly as possible. My data samples aren't faster than 10-20fps anyway.
Is there an easy way to "force" a paint at certain points, or is this a limit of how Views work? Should I be implementing this in a SurfaceView instead? (I have not played with that yet... want advice first.) Thanks in advance for suggestions.
(Later that same day...)
Update: I found a page in the Docs that does suggest implementing my widget as a SurfaceView is the way to go:
http://developer.android.com/guide/topics/graphics/2d-graphics.html
(An hour after that...)
SurfaceView seems overkill for what I want to do. The best-practice method is to "own" the whole canvas, but I have already developed the rest of my controls and layouts and they work well. It must be possible to get some better performance with what I have, especially since interacting with the UI makes the redraw speed satisfactory.
It turns out SurfaceView was the way to go. I was benchmarking on an older phone which didn't help. (The frame rate using a standard View was fine on an ASUS eeePad). I had to throw away some code, but the end result is smoother and faster with SurfaceView. Further, I was able to re-use more code than I expected and actually dramatically simplified my multitouch handling code (since everything I want to touch is in the same SurfaceView.
FYI: I'm still only getting about 15fps on Droid X, but half of the CPU load appears to be data packet processing. The eeePad is doing almost 40fps now -- and my data rate is only 20 samples/sec.
So... a win I guess. I want the Droid X to run better, but it flies on a real tablet.
In my view I have a simple ARGB drawable that takes about 2ms to draw but I can draw the same file as a bitmap in under 0.5ms (just some quick code, I can't really consider it an option). What are the best ways to optimize the drawing speed of a drawable?
It will depend on the number of drawables and how many times each gets drawn. For a small number, use canvas (an exact number will also depend on the device) I would suggest using Canvas as it's a nice higher level approach to drawing.
If you want to crank out a lot of images (think hundreds), I would suggest creating a GLSurfaceView and using openGL to render your images using VBOs tailored to your app. I would also recommend using a texture sheet if you go down this route since you'll get a huge increase in performance at the cost of code complexity.
But this will also depend on that type of app. My background is in game development so I use openGL exclusively for better performance. for a simple app (something along the lines of androidify) Canvas should be fine. If you want a simple tutorial for openGL, I suggest visiting Bergman's series of posts on the topic (google should give you a link for that). It is a nice intro to openGL.