Xamarin multiplatform - Video from jpeg sequence - android

I want to know how I can display video from jpegs in Xamarin (all platforms).
My jpegs are being streamed from a http client stream sent by a popular video surveillance management software.
My jpegs are in the form of byte[] and I get about 10 jpegs/second. This format is imposed.
I tried rapidly changing the Source on a Image but it results in severe fliquering on Android. This seems to work on Windows phone but not so good performance.
How can I create a videoplayer for each one? Unless I am wrond, the existing components cannot do this.
Best,

Thank you Jason! Works great, very fluid rendering!!
Simply add the SkiaSharp.Views.Forms with NuGet to the project and voila!
Here is what that would look like in code (shared project):
// Content page initialization
private void InitUI() {
Title = "Xamavideo";
var button = new Button
{
Text = "Connect!"
};
Label label = new Label
{
Text = ""
};
var scroll = new ScrollView();
scroll.BackgroundColor = Color.Black;
Content = scroll;
var stack = new StackLayout
{
Padding = 40,
Spacing = 10
};
//Add a SKCanvasView item to the stack
var videoCanvas = new SKCanvasView
{
HeightRequest = 400,
WidthRequest = 600,
};
videoCanvas.PaintSurface += OnCanvasViewPaintSurface;
stack.Children.Add(videoCanvas);
}
//Create the event handler
void OnCanvasViewPaintSurface(object sender, SKPaintSurfaceEventArgs args)
{
SKImageInfo info = args.Info;
SKSurface surface = args.Surface;
// using (var stream = new SKManagedStream(fileStream))
if (lastFrame == null) return;
using (var canvas = surface.Canvas)
// use KBitmap.Decode to decode the byte[] in jpeg format
using (var bitmap = SKBitmap.Decode(lastFrame))
using (var paint = new SKPaint())
{
// clear the canvas / fill with black
canvas.DrawColor(SKColors.Black);
canvas.DrawBitmap(bitmap, SKRect.Create(640, 480), paint);
}
}
void UpdateFrame(VideoClient client){
//Use this to update the canvas:
byte[] lastFrame = client.imageBytes;
videoCanvas.InvalidateSurface();
}

Related

ZXing BarcodeReader Don't Decode Some Barcodes

I am developing a barcode reader with Xamarin.Forms. And I'm trying to scan the image on Android device.
First I select the image from the gallery with Xamarin.Essentials MediaPicker and from the path of this image I get an RGBLuminance with the Dependency class.
Then I am trying to decode this RGBLuminance with the Decode() method of the ZXing BarcodeReaderGeneric class.
The application successfully decodes the barcodes in some images. However, sometimes it returns null when decoding. I might have made a mistake while converting the image to Bitmap or creating the RGBLuminanceSource.
I would like to find out how a class that can decode both color, black and white and grayscale images should be.
public RGBLuminanceSource GetRGBLuminanceSource(string imagePath)
{
if (File.Exists(imagePath))
{
Android.Graphics.Bitmap bitmap = BitmapFactory.DecodeFile(imagePath);
List<byte> rgbBytesList = new List<byte>();
for (int y = 0; y < bitmap.Height; y++)
{
for (int x = 0; x < bitmap.Width; x++)
{
var c = new Android.Graphics.Color(bitmap.GetPixel(x, y));
rgbBytesList.AddRange(new[] { c.A, c.R, c.G, c.B });
}
}
byte[] rgbBytes = rgbBytesList.ToArray();
return new RGBLuminanceSource(rgbBytes, bitmap.Width, bitmap.Height, RGBLuminanceSource.BitmapFormat.RGB32);
}
return null;
}
Command in the ViewModel class:
public ICommand PickCommand => new Command(PickImage);
private async void PickImage()
{
var pickResult = await MediaPicker.PickPhotoAsync(new MediaPickerOptions
{
Title = "Select a barcode."
});
var path = pickResult.FullPath;
var RGBLuminance = DependencyService.Get<ILuminance>().GetRGBLuminanceSource(path);
var reader = new BarcodeReaderGeneric();
var result = reader.Decode(RGBLuminance);
}
I am using this code in xamarin.android and i never had issues with it:
var scanner = new MobileBarcodeScanner();
var result = await scanner.Scan(_context, MobileBarcodeScanningOptions.Default);
It opens the camera, user takes a pic of barcode and result.Text contains the scanned barcode.

Xamarin.Forms and Android - Getting the result of a toggle button from a different view

I have been following the custom pin tutorial using Xamarin.Forms which I have linked below. I have finished implementing it and I have also moved onto adding pins to the map as well through tapping.
https://learn.microsoft.com/en-us/xamarin/xamarin-forms/app-fundamentals/custom-renderer/map/customized-pin
(side note: I am working almost exclusively with Xamarin.Forms and Android)
Currently, I am able to tap on the map and a new custom pin will be added at that location, which is great. What is not great is that I was unable to figure out how to get a tap and long hold gesture to work instead of just the normal tap. To try to combat this, and because I will eventually have to add these anyways, I am planning on adding a button that the user can press to initiate that they want to add a button to the map, and I will later add an undo button, and many others, etc.
The problem is, I have no idea how to get the result of what my toggle button state is from the custom render that I am using for the map. Where can I get the result of my toggle button and use it as a boolean towards whether to add a button or not?
Here is the toggle button code, which I took line by line from their example on this page:
https://learn.microsoft.com/en-us/xamarin/xamarin-forms/user-interface/button
Here is the code where I add a custom pin just by a single tap:
private void GoogleMap_MapClick(object sender, GoogleMap.MapClickEventArgs e)
{
((CustomMap)Element).OnTap(new Position(e.Point.Latitude, e.Point.Longitude));
var addingPin = new CustomPin
{
Type = PinType.Place,
Position = new Position(e.Point.Latitude, e.Point.Longitude),
Address = " - need to possibly implement - ",
Id = "shelter",
Label = "Pin from tap",
Url = "http://www.redcross.org"
};
Map.Pins.Add(addingPin);
this.customPins.Add(addingPin);
}
I thought about making a custom button render but by my knowledge I can only apply one Android render to a content page at a time, and when I tried to add a custom button render to the map render then this method was not happy, as it was taking in some sort of Android Map View and not a button view:
protected override void OnElementChanged(Xamarin.Forms.Platform.Android.ElementChangedEventArgs<Map> e)
{
base.OnElementChanged(e);
if (e.OldElement != null)
{
NativeMap.InfoWindowClick -= OnInfoWindowClick;
}
if (e.NewElement != null)
{
var formsMap = (CustomMap)e.NewElement;
customPins = formsMap.CustomPins;
Control.GetMapAsync(this);
}
}
Any help is greatly appreciated. Below I have included a pic of what my application looks like so far, along with the custom page that I am using as well.
using Hermes.Models;
using System;
using System.Collections.Generic;
using Xamarin.Forms;
using Xamarin.Forms.Maps;
namespace Hermes
{
public class MapPage : ContentPage
{
public MapPage()
{
CustomMap customMap = new CustomMap()
{
HeightRequest = 100,
WidthRequest = 960,
VerticalOptions = LayoutOptions.FillAndExpand,
MapType = MapType.Street,
};
var examplePinSupplies = new CustomPin
{
Type = PinType.Place,
Position = new Position(42.02525, -93.65087),
Address = " - need to possibly implement - ",
Id = "supplies",
Label = "supplies",
Url = "https://www.redcross.org/store"
};
var examplePinMedical = new CustomPin
{
Type = PinType.Place,
Position = new Position(42.02290, -93.63912),
Address = " - need to possibly implement - ",
Id = "medical",
Label = "medical",
Url = "http://www.redcross.org"
};
var examplePinShelter = new CustomPin
{
Type = PinType.Place,
Position = new Position(42.02045, -93.60968),
Address = " - need to possibly implement - ",
Id = "shelter",
Label = "shelter",
Url = "http://www.redcross.org"
};
customMap.CustomPins = new List<CustomPin> { examplePinSupplies, examplePinMedical, examplePinShelter };
customMap.Pins.Add(examplePinSupplies);
customMap.Pins.Add(examplePinMedical);
customMap.Pins.Add(examplePinShelter);
customMap.MoveToRegion(MapSpan.FromCenterAndRadius(new Position(42.025250, -93.650870), Distance.FromMiles(1.0)));
var addPin = new ToggleButton { Text = "Add pin" };
var buttons = new StackLayout
{
Orientation = StackOrientation.Horizontal,
Children = {
addPin
}
};
Content = new StackLayout
{
Spacing = 0,
Children = {
customMap,
buttons
}
};
}
}
}
What I am trying to do: I am trying to tap the 'add pin' toggle button, and then the map will allow me to only add one pin on the map by tapping, and then any other consecutive taps that are not on another pin will not add another pin to the map.

Xamarin.Android - Dialog is not displayed

I have a problem with my custom Dialog.
I want to show it while my application do something (in particular, creates a bitmap and does a HTTP request).
This is the portion of my code:
LoadingDialog myLoading = new LoadingDialog();
myLoading.Show()
TakePicture();
while (myPhotoFace == null) { }
Context context = Android.App.Application.Context;
if (Connectivity.isConnected(context))
{
byte[] bytePhoto;
using (var stream = new MemoryStream())
{
Constants.myGallery.Compress(Bitmap.CompressFormat.Jpeg, 40, stream);
bytePhoto = stream.ToArray();
}
byte[] byteFace;
using (var stream = new MemoryStream())
{
myPhotoFace.Compress(Bitmap.CompressFormat.Jpeg, 40, stream);
byteFace = stream.ToArray();
}
MediaEmotion myMedia = new MediaEmotion();
string token = Utils.utils.GetParametro("token");
myMedia.mediaID = 0;
myMedia.face = byteFace;
myMedia.photo = bytePhoto;
var client = new RestClient("https://myServer.net/");
var request = new RestRequest("/Controller/Method?token=" + token, Method.POST);
string json = JsonConvert.SerializeObject(myMedia);
request.AddParameter("stringaMediaEmotion", json);
IRestResponse response = client.Execute(request);
Object myObject = JsonConvert.DeserializeObject<Object>(response.Content.ToString());
[...]
After, I do something before start new activity.
As you can see, as a first operation I show my dialog (which, I verified, works correctly) but only appears as the last thing before moving on to the new activity. Also by simply trying to change the text of a textview the problem is the same.
Why? What is the problem? How can resolve this?
Try to put all the network related code inside an async task: you'll free the main thread from network operation and the popup will appear until you'll move to another activity.
You can find some good examples and documentation here https://blog.xamarin.com/getting-started-with-async-await/

Inline CSS for hr not applied while converting html to pdf using iText library

I am using Itext library for android for converting html to pdf which is working fine but at certain things it is not parsing properly. I want to create a dotted line separator of red color but it is always gives me a solid line separator with dark gray color.
My html tag is
<hr noshade style="border: 0; width:100%;border-bottom-width: 1px; border-bottom-style: dotted; border-bottom-color: red">
My conversion code
Document document = new Document(PageSize.A4);
//this sets the margin to the created pdf
document.setMargins(35, 35, 150, 100);
PdfWriter writer = PdfWriter.getInstance(document,
new FileOutputStream(fileWithinMyDir));
if (isPrescription) {
HeaderFooterPageEvent event = new HeaderFooterPageEvent();
writer.setPageEvent(event);
} else {
CertificateFooterPageEvent event = new CertificateFooterPageEvent();
writer.setPageEvent(event);
}
document.open();
HtmlPipelineContext htmlContext = new HtmlPipelineContext(null);
htmlContext.setTagFactory(Tags.getHtmlTagProcessorFactory());
htmlContext.setImageProvider(new AbstractImageProvider() {
public String getImageRootPath() {
Uri uri = Uri.parse("file:///android_asset/");
return uri.toString();
}
});
CSSResolver cssResolver =
XMLWorkerHelper.getInstance().getDefaultCssResolver(false);
// Pipelines
PdfWriterPipeline pdf = new PdfWriterPipeline(document, writer);
HtmlPipeline html = new HtmlPipeline(htmlContext, pdf);
CssResolverPipeline css = new CssResolverPipeline(cssResolver, html);
XMLWorker worker = new XMLWorker(css, true);
XMLParser p = new XMLParser(worker);
InputStream is = new ByteArrayInputStream(htmlString.getBytes());
XMLWorkerHelper.getInstance().parseXHtml(writer, document, is);
p.parse(is);
document.close();
I'm a .NET developer, so the code is in C#. But you should be able to easily translate the following.
iText is a PDF-first library, and [X]HTML parsing is quite complex so it's not full featured in that regard. Whenever parsing [X]HTML and things aren't going the way you expect for specific tags, the basic steps you should follow are:
Verify XML Worker supports the tag: Tags class.
If the tag is supported, which in this case is true, take a look at the default implementation. Here it's handled by the the HorizontalRule class. However, we see there's no support for your use case, so one way to go is use that code as a blueprint. (follows below) You can also inherit from the specific tag class and override the End() method as done here. Either way, all you're doing is implementing a custom tag processor.
If the tag is not supported, you need to roll your own custom tag processor by inheriting from AbstractTagProcessor.
Anyway, here's a simple example to get you started. First, the custom tag processor:
public class CustomHorizontalRule : AbstractTagProcessor
{
public override IList<IElement> Start(IWorkerContext ctx, Tag tag)
{
IList<IElement> result;
LineSeparator lineSeparator;
var cssUtil = CssUtils.GetInstance();
try
{
IList<IElement> list = new List<IElement>();
HtmlPipelineContext htmlPipelineContext = this.GetHtmlPipelineContext(ctx);
Paragraph paragraph = new Paragraph();
IDictionary<string, string> css = tag.CSS;
float baseValue = 12f;
if (css.ContainsKey("font-size"))
{
baseValue = cssUtil.ParsePxInCmMmPcToPt(css["font-size"]);
}
string text;
css.TryGetValue("margin-top", out text);
if (text == null) text = "0.5em";
string text2;
css.TryGetValue("margin-bottom", out text2);
if (text2 == null) text2 = "0.5em";
string border;
css.TryGetValue(CSS.Property.BORDER_BOTTOM_STYLE, out border);
lineSeparator = border != null && border == "dotted"
? new DottedLineSeparator()
: new LineSeparator();
var element = (LineSeparator)this.GetCssAppliers().Apply(
lineSeparator, tag, htmlPipelineContext
);
string color;
css.TryGetValue(CSS.Property.BORDER_BOTTOM_COLOR, out color);
if (color != null)
{
// WebColors deprecated, but docs don't state replacement
element.LineColor = WebColors.GetRGBColor(color);
}
paragraph.SpacingBefore += cssUtil.ParseValueToPt(text, baseValue);
paragraph.SpacingAfter += cssUtil.ParseValueToPt(text2, baseValue);
paragraph.Leading = 0f;
paragraph.Add(element);
list.Add(paragraph);
result = list;
}
catch (NoCustomContextException cause)
{
throw new RuntimeWorkerException(
LocaleMessages.GetInstance().GetMessage("customcontext.404"),
cause
);
}
return result;
}
}
Most of the code is taken directly from the existing source, with the exception of the checks for CSS.Property.BORDER_BOTTOM_STYLE and CSS.Property.BORDER_BOTTOM_COLOR to set border style and color if they're inlined in the <hr> style attribute.
Then you add the custom tag processor above to the XML Worker TagProcessorFactory:
using (var stream = new FileStream(OUTPUT_FILE, FileMode.Create))
{
using (var document = new Document())
{
var writer = PdfWriter.GetInstance(document, stream);
document.Open();
var tagProcessorFactory = Tags.GetHtmlTagProcessorFactory();
// custom tag processor above
tagProcessorFactory.AddProcessor(
new CustomHorizontalRule(),
new string[] { HTML.Tag.HR }
);
var htmlPipelineContext = new HtmlPipelineContext(null);
htmlPipelineContext.SetTagFactory(tagProcessorFactory);
var pdfWriterPipeline = new PdfWriterPipeline(document, writer);
var htmlPipeline = new HtmlPipeline(htmlPipelineContext, pdfWriterPipeline);
var cssResolver = XMLWorkerHelper.GetInstance().GetDefaultCssResolver(true);
var cssResolverPipeline = new CssResolverPipeline(
cssResolver, htmlPipeline
);
var worker = new XMLWorker(cssResolverPipeline, true);
var parser = new XMLParser(worker);
var xHtml = "<hr style='border:1px dotted red' />";
using (var stringReader = new StringReader(xHtml))
{
parser.Parse(stringReader);
}
}
}
One thing to note is that even though we're using the shorthand border inline style, iText's CSS parser appears to set all the styles internally. I.e., you can use any of the four longhand styles to check - I just happened to use CSS.Property.BORDER_BOTTOM_STYLE and CSS.Property.BORDER_BOTTOM_COLOR.
The resulting PDF:
You could use a div without any or with any content you want instead of an hr and give border style to that div, I am sure it will work in your case.

Output file using FFmpeg in Xamarin Android

I'm building an android app using Xamarin. The requirement of the app is to capture video from the camera and encode the video to send it across to a server.
Initially, I was using an encoder library on the server-side to encode recorded video but it was proving to be extremely unreliable and inefficient especially for large-sized video files. I have posted my issues on another thread here
I then decided to encode the video on the client-side and then send it to the server. I've found encoding to be a bit complicated and there isn't much information available on how this can be done. So, I searched for the only way I knew how to encode a video that is by using FFmpeg codec. I've found some solutions. There's a project on GitHub that demonstrates how FFmpeg is used inside a Xamarin android project. However, running the solution doesn't give any output. The project has a binary FFmpeg file which is installed to the phone directory using the code below:
_ffmpegBin = InstallBinary(XamarinAndroidFFmpeg.Resource.Raw.ffmpeg, "ffmpeg", false);
Below is the example code for encoding video into a different set of outputs:
_workingDirectory = Android.OS.Environment.ExternalStorageDirectory.AbsolutePath;
var sourceMp4 = "cat1.mp4";
var destinationPathAndFilename = System.IO.Path.Combine (_workingDirectory, "cat1_out.mp4");
var destinationPathAndFilename2 = System.IO.Path.Combine (_workingDirectory, "cat1_out2.mp4");
var destinationPathAndFilename4 = System.IO.Path.Combine (_workingDirectory, "cat1_out4.wav");
if (File.Exists (destinationPathAndFilename))
File.Delete (destinationPathAndFilename);
CreateSampleFile(Resource.Raw.cat1, _workingDirectory, sourceMp4);
var ffmpeg = new FFMpeg (this, _workingDirectory);
var sourceClip = new Clip (System.IO.Path.Combine(_workingDirectory, sourceMp4));
var result = ffmpeg.GetInfo (sourceClip);
var br = System.Environment.NewLine;
// There are callbacks based on Standard Output and Standard Error when ffmpeg binary is running as a process:
var onComplete = new MyCommand ((_) => {
RunOnUiThread(() =>_logView.Append("DONE!" + br + br));
});
var onMessage = new MyCommand ((message) => {
RunOnUiThread(() =>_logView.Append(message + br + br));
});
var callbacks = new FFMpegCallbacks (onComplete, onMessage);
// 1. The idea of this first test is to show that video editing is possible via FFmpeg:
// It results in a 150x150 movie that eventually zooms on a cat ear. This is desaturated, and there's a fade-in.
var filters = new List<VideoFilter> ();
filters.Add (new FadeVideoFilter ("in", 0, 100));
filters.Add(new CropVideoFilter("150","150","0","0"));
filters.Add(new ColorVideoFilter(1.0m, 1.0m, 0.0m, 0.5m, 1.0m, 1.0m, 1.0m, 1.0m));
var outputClip = new Clip (destinationPathAndFilename) { videoFilter = VideoFilter.Build (filters) };
outputClip.H264_CRF = "18"; // It's the quality coefficient for H264 - Default is 28. I think 18 is pretty good.
ffmpeg.ProcessVideo(sourceClip, outputClip, true, new FFMpegCallbacks(onComplete, onMessage));
//2. This is a similar version in command line only:
string[] cmds = new string[] {
"-y",
"-i",
sourceClip.path,
"-strict",
"-2",
"-vf",
"mp=eq2=1:1.68:0.3:1.25:1:0.96:1",
destinationPathAndFilename2,
"-acodec",
"copy",
};
ffmpeg.Execute (cmds, callbacks);
// 3. This lists codecs:
string[] cmds3 = new string[] {
"-codecs",
};
ffmpeg.Execute (cmds, callbacks);
// 4. This convers to WAV
// Note that the cat movie just has some silent house noise.
ffmpeg.ConvertToWaveAudio(sourceClip, destinationPathAndFilename4, 44100, 2, callbacks, true);
I have tried different commands but no output file is generated. I have tried to use another project found here but this one has the same issue. I don't get any errors but no output file is generated. I'm really hoping someone can help me find a way I can manage to use FFmpeg in my project or some way to compress video to transport it to the server.
I will really appreciate if someone can point me in the right direction.
Just figure how to get the output by adding the permission in AndroidManifest file.
android.permission.WRITE_EXTERNAL_STORAG
Please read the update on the repository, it says that there is a second package, Xamarin.Android.MP4Transcoder for Android 6.0 onwards.
Install NuGet https://www.nuget.org/packages/Xamarin.Android.MP4Transcoder/
await Xamarin.MP4Transcoder.Transcoder
.For720pFormat()
.ConvertAsync(inputFile, ouputFile, f => {
onProgress?.Invoke((int)(f * (double)100), 100);
});
return ouputFile;
For Previous Android versions
Soruce Code https://github.com/neurospeech/xamarin-android-ffmpeg
Install-Package Xamarin.Android.FFmpeg
Use this as template, this lets you log output as well as calculates progress.
You can take a look at source, this one downloads ffmpeg and verifies sha1 hash on first use.
public class VideoConverter
{
public VideoConverter()
{
}
public File ConvertFile(Context contex,
File inputFile,
Action<string> logger = null,
Action<int,int> onProgress = null)
{
File ouputFile = new File(inputFile.CanonicalPath + ".mpg");
ouputFile.DeleteOnExit();
List<string> cmd = new List<string>();
cmd.Add("-y");
cmd.Add("-i");
cmd.Add(inputFile.CanonicalPath);
MediaMetadataRetriever m = new MediaMetadataRetriever();
m.SetDataSource(inputFile.CanonicalPath);
string rotate = m.ExtractMetadata(Android.Media.MetadataKey.VideoRotation);
int r = 0;
if (!string.IsNullOrWhiteSpace(rotate)) {
r = int.Parse(rotate);
}
cmd.Add("-b:v");
cmd.Add("1M");
cmd.Add("-b:a");
cmd.Add("128k");
switch (r)
{
case 270:
cmd.Add("-vf scale=-1:480,transpose=cclock");
break;
case 180:
cmd.Add("-vf scale=-1:480,transpose=cclock,transpose=cclock");
break;
case 90:
cmd.Add("-vf scale=480:-1,transpose=clock");
break;
case 0:
cmd.Add("-vf scale=-1:480");
break;
default:
break;
}
cmd.Add("-f");
cmd.Add("mpeg");
cmd.Add(ouputFile.CanonicalPath);
string cmdParams = string.Join(" ", cmd);
int total = 0;
int current = 0;
await FFMpeg.Xamarin.FFMpegLibrary.Run(
context,
cmdParams
, (s) => {
logger?.Invoke(s);
int n = Extract(s, "Duration:", ",");
if (n != -1) {
total = n;
}
n = Extract(s, "time=", " bitrate=");
if (n != -1) {
current = n;
onProgress?.Invoke(current, total);
}
});
return ouputFile;
}
int Extract(String text, String start, String end)
{
int i = text.IndexOf(start);
if (i != -1)
{
text = text.Substring(i + start.Length);
i = text.IndexOf(end);
if (i != -1)
{
text = text.Substring(0, i);
return parseTime(text);
}
}
return -1;
}
public static int parseTime(String time)
{
time = time.Trim();
String[] tokens = time.Split(':');
int hours = int.Parse(tokens[0]);
int minutes = int.Parse(tokens[1]);
float seconds = float.Parse(tokens[2]);
int s = (int)seconds * 100;
return hours * 360000 + minutes * 60100 + s;
}
}

Categories

Resources