I have two smartphones that I no longer use and I'd like to use their cameras for a basic video surveillance system, rather than buying expensive cameras.
My idea is to record overnight, and save the videos on my laptop.
It would be a good opportunity for me to learn a bit more about Android programming as well.
So I guess the approach is:
a TCP/IP server, gathering the information coming from the two (or N)
phones;
a TCP/IP client, to run on each phone, recording and sending the
information to the server;
I am not sure what that "information" should be, though. Should that be the single frames captured by the cameras, or is there a way to stream a video?
In case I wanted to implement a basic motion detection, would it be better to do it on the clients or on the server?
Is my approach above correct?
Have a look at the open-source libstreaming project, which will allow you stream video from your phones. But if you want a time-lapse recording, e.g. 1 frame per second or less, then sending single frames may be preferable.
Note that your smartphones will need all-time power supply, because camera and communications will drain any battery in very short time. Also keep in mind that phone cameras do not perform well in low-illumination conditions, and this may justify investment in expensive dedicated cameras.
You can use OpenCV for motion detection either on device or on server, or even both. The choice depends on your needs and resources. E.g. you may significantly reduce overall data volumes if the device only send the video when it detects motion.
Related
I want to develop an Android app that captures a video with the phone's camera, then uploads the video (as a single file) to cloud. And then I want to make it more efficient by segmenting the recorded video into small chunks (say 5 seconds) and upload the chunks to the cloud. And then compare the two approach and show that the second approach is more efficient and provides faster upload according to this blog.
Is the chunking approach more efficient? In what ways? And how can I implement this? Shall I wait until the video is finished, then chunk them up, or can we do it in real-time as capturing progresses? Any tips or experience doing this would certainly help.
Breaking a video into chunks and performing processing, such are encoding or packaging, in parallel is a very common technique for VOD video at this time.
It allows for faster processing so long as you have the computing resources to tackle tasks in parallel, which many multi core computers and the cloud do have.
It can also allow you to to schedule the processing when your resources are most available, in a period of low load from other jobs for example, or when they are cheapest which can help with cloud computing costs.
Its hard to say if that is more efficient as it depends on what you are measuring - if you added up the total compute instructions or cycles it is quite likely it actually takes more this way, but for the reasons above it is often the preferred approach anyway.
For video transfer or transport, if you were able to send the different chunks on different paths, each of which had a limit that you could fill with that segment, it might indeed provide time savings or efficiency.
However, If your device has a single transmission path, for example it's WiFi IP connection, then the its not clear that segmenting the video would have benefit beyond the 'breaking up' of the video that already happens when you send it packet by packet over an IP network anyway.
If you goal is fast video transmission then it would likely be worth taking a look at some of the specialist protocols which exist for fast and efficient live video transmission. Some of these may be UDP based rather than TCP and if so you may need to check that your target network firewalls and routing rules will support them. SRT would be a good example to look, see like below, bit others exists such as ZiXi which is proprietary:
https://en.wikipedia.org/wiki/Secure_Reliable_Transport
Focusing on transmission, internet Video streams are actually broken into chunks anyway, because they are delivered over packet networks. The video stream will also be 'chunked' at a higher level also frequently, in addition to encoding and container (e.g. mp4) groupings, to support ABR protocols which segment the video to allow different bitrate renditions be switched between (see: https://stackoverflow.com/a/42365034/334402).
Breaking a video into chunks can also help with playback if the chunks can start playing before the entire video is downloaded - again this is typical of most streamed videos on the internet.
If we ignore any effect of different protocols retry, packet loss etc strategies, and if you have a single transmission 'pipe' of a set capacity, breaking the video into chunks does not make it faster to transfer.
If, however, you have multiple transmission 'pipes' then breaking the video up and sending different parts in parallel across the different pipes may indeed speed it up. It important to remember that, even then, you are limited if the video is a live stream, by the actual live video rate itself - i.e. you can't transfer the video faster than the source is producing it.
How much lag is expected when an android device's camera is streamed live?
I checked Miracast and it has a lag of around 150-300 ms(usually its around 160-190 ms).
I have a bluetooth and a wifi direct app and both lag by around 400-550 ms. I was wondering if it would be possible to reproduce or come closer to Miracast's performance.
I am encoding the camera frames in H.264 and using my own custom protocol to transmit the encoded frames over a TCP connection(in case of WiFi Direct).
Video encoding is compute intensive so any help you can get from the hardware usually speeds things up. It is worth making sure you are using codecs which leverage the hardware - i.e avoid having your H.264 encoding all in software.
There is a video here which discusses how to access HW accelerated video codecs, although it is a little old, but will apply to older devices:
https://www.youtube.com/watch?v=tLH-oGfiGaA
I believe that MediaCodec now provides a Java API to the Hardware Codecs (have not tried or seen speed comparison tests myself), which makes things easier:
http://developer.android.com/about/versions/android-4.1.html#Multimedia
See the note here also about using the camera's Surface preview as a speed aid:
https://stackoverflow.com/a/17243244/334402
As an aside, 500ms does not seem that bad, and you are probably into diminishing returns so the effort to reduce may be high if you can live with your current lag.
Also, I assume you are measuring the lag (or latency) on the server side? If so you need to look at how the server is decoding and presenting also, especially if you are comparing your own player with a third party one.
It is worth looking at your jitter buffer on the receiving side, irrespective of the stream etc latency - in simple terms waiting for a larger number of packets before you start playback may cause more startup delay but it also may provide a better overall user experience. This is because the large buffer will be more tolerant of delayed packets, before going into a 'buffering' mode that users tend not to like.
Its a balance, and your needs may dictate a bias one way or the other. If you were planning a video chat like application for example, then the delay is very important as it starts to become annoying to users above 200-300 ms. If you are providing a feed from a sports event, then delay may be less important and avoiding buffering pauses may give better user perceived quality.
I'm currently working on an app with the end goal of being roughly analogous to an Android version of Air Play for the iDevices.
Streaming media and all that is easy enough, but I'd like to be able to include games as well. The problem with that is that to do so I'd have to stream the screen.
I've looked around at various things about taking screenshots (this question and the derivatives from it in particular), but I'm concerned about the frequency/latency. When gaming, anything less than 15-20 fps simply isn't going to cut it, and I'm not certain such is possible with the methods I've seen so far.
Does anyone know if such a thing is plausible, and if so what it would take?
Edit: To make it more clear, I'm basically trying to create a more limited form of "remote desktop" for Android. Essentially, capture what the device is currently doing (movie, game, whatever) and replicate it on another device.
My initial thoughts are to simply grab the audio buffer and the frame buffer and pass them through a socket to the other device, but I'm concerned that the methods I've seen for capturing the frame buffer are too slow for the intended use. I've seen people throwing around comments of 3 FPS limits and whatnot on some of the more common ways of accessing the frame buffer.
What I'm looking for is a way to get at the buffer without those limitations.
I am not sure what you are trying to accomplish when you refer to "Stream" a video game.
But if you are trying to mimic AirPlay, all you need to do is connect via a Bluetooth/ internet connection to a device and allow sound. Then save the results or handle it accordingly.
But video games do not "Stream" a screen because the mobile device will not handle much of a work load. There are other problems like, how to will you handle the game if the person looses internet connection while playing? On top of that, this would require a lot of servers to support the game workload on the backend and bandwidth.
But if you are trying to create an online game. Essentially all you need to do is send and receive messages from a server. That is simple. If you want to "Stream" to another device, simply connect the mobile device to speakers or a TV. Just about all mobile video games or applications just send simple messages via JSON or something similar. This reduces overhead, is simple syntax, and may be used across multiple platforms.
It sounds like you should take a look at this (repost):
https://stackoverflow.com/questions/2885533/where-to-start-game-programming-for-android
If not, this is more of an open question about how to implement a video game.
I'm working on an Augmented Reality (AR) demo in which high quality openGL renders (in C++) will be generated from a PC and then streamed to an Android display device (running minimum Android 2.2). What is the easiest way to achieve this in real-time (30 FPS on Android Device) ?
I've looked into existing Anrdroid applications, and have not found anything to be suitable so far. The best available were remote desktop applications (such as TeamViewer) however the frame rates were far too low and unreliable.
Possible solution A:
1) Encode openGL window as H.264 Video (natively supported by Android)
2) Stream the H.264 Video via RTSP using a server
3) View the content from an Android Browser (android and pc connected via WiFi)
Possible solution B:
1) Encode openGL window as IP Camera in c++ (is this possible?)
2) Use an IPCamViewer on Android device to view (again connected via WiFi)
I'm not entirely sure if either or both of these approaches are viable and would like some reassurance before moving forward.
What is the resolution of the image (is it equal to the current screen resolution, larger or smaller)? It is possible and efficient to transport a H.264 stream, but it also depends on the machine used to do the encoding. Hardware encoders or GPU-accelerated encoders are your best bet.
Remember - if you choose to go with encoding, you will have latency due to buffering (on the encode and the decode side). It will be a constant time offset so if that's not a problem you should be ok.
The total system delay as proposed by this paper is composed of
Note that none of these delays can be fully measured directly in frame-time. Some of these depend on the frame data and/or the encoder/processing performed. But as a rough estimate with fast GPU-encoding and hardware decoding I'd say a lag of around 5-20 frames. You'll have to measure the final latency per-scenario. I did this by sending frames containing text (ticks) and once the frames were steady, comparing them side-by-side. In fact, I even allowed the user to enter "test mode" at anytime to compensate for network traffic peak times or have him change the "quality" settings to tweak this delay.
If someone wants to write a android application that interacts with a physical device, specifically a reader using mobiles audio jack
(e.g. Like how Square Inc is doing ) how is this done?
Is there a api's to interact with the reader and get the cards data?
When a company creates a reader (physical device) does it provide relevant apis?
Are the physical details abstracted from the application programmer?
I have found the AudioRecord class which can record magnetic stripe data from audio jack
But I can't fiqure out how to capture the actual card swipe event and
to extract the meaningful data from RAW DATA
Can any one help me with this
Any input is highly welcome!
The way this usually works is by encoding the data signal sent out by the device, like the card reader, in such a way that is can be decoded on the other end. Sound is a wave, and different amplitudes correspond to different loudness, and different frequencies correspond to different pitches. Imagine you have a sine wave, that varies between a high and a low frequency that are sufficiently different from each other so as to be easily distinguishable. The device sending out binary data (0's and 1's) can translate this data into an audio signal that varies by frequency (an alternative is varying amplitude). The receiver, in this case the mobile device, decodes the signal back into 0's and 1's. This is called "Frequency-shift-keying" (check out more here: http://en.wikipedia.org/wiki/Frequency-shift_keying).
The simplest way to implement this is to try and find an open library that already does it. The device sending the data will also need to contain some kind of microcontroller that can perform the initial modulation. If you come across any good libraries, let me know, because I'm currently
looking.
To answer your question, companies do not generally provide APIs etc to perform this.
This may seem like a lot of extra work to convert a digital signal, into an audio signal, and back, and you're right. However, every mobile device has essentially the same headphone jack, whereas the USB port on an Android is drastically different from an iPhone's lighting connector, or the connector in previous iPhones. Sending wirelessly through a network or Bluetooth is also an option, but they have their disadvantages as well.
Now the mobile device must be using a special headphone jack that supports microphones, otherwise it cannot receive input, it can only output sound. Most smartphones can do this.
Radios work on this principle (FM = Frequency modulation, AM = amplitude modulation).
Old dial up modems used FSK, which is why you heard those weird noises each time it connected.
Hope that helps!