Develop Reference

What is the use to send spitting/chunk files to server

I have to upload big video files to a server, but it's taking too long to upload, so I decided to split/chunk the files and then send them to the server
After splitting my files, I get a response like the following:
[ /storage/emulated/0/1493357699.mp4.001, /storage/emulated/0/1493357699.mp4.002, /storage/emulated/0/1493357699.mp4.003, /storage/emulated/0/1493357699.mp4.004, /storage/emulated/0/1493357699.mp4.005, /storage/emulated/0/1493357699.mp4.006, /storage/emulated/0/1493357699.mp4.007, /storage/emulated/0/1493357699.mp4.008 ]
My thought is what is the use to upload spitting/chunk file to server?
My code for splitting files:
public static List<File> splitFile(File f) {
try {
int partCounter = 1;
List<File> result = new ArrayList<>();
int sizeOfFiles = 1024 * 1024;// 1MB
byte[] buffer = new byte[sizeOfFiles];
// create a buffer of bytes sized as the one chunk size
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(f));
String name = f.getName();
int tmp = 0;
while ((tmp = bis.read(buffer)) > 0) {
File newFile = new File(f.getParent(), name + "." + String.format("%03d", partCounter++));
// naming files as <inputFileName>.001, <inputFileName>.002, ...
FileOutputStream out = new FileOutputStream(newFile);
out.write(buffer, 0, tmp);//tmp is chunk size. Need it for the last chunk,
// which could be less then 1 mb.
result.add(newFile);
}
return result;
} catch (Throwable throwable) {
throwable.printStackTrace();
}
return null;
}

I have implemented in one of my projects. I see two primary reasons:
To achieve multi-threaded / multiple connection for uploading chunks. You can upload multiple chunks at the same time.
Stop/Resume uploading of rest of the chunks if either of the chunk fails to upload (depending on server response)

Retrofit2 Streaming GET method OutOfMemoryError

My interface:
public interface Downloader {
#Streaming #GET Call<ResponseBody> get(#Url final String url);
}
My download method:
private void download(final String url, final File zip) throws IOException {
AsyncHelper.assertNotMainThread();
final Call<ResponseBody> call = downloaderInstance.get(url);
final Response<ResponseBody> response = call.execute();
if (!response.isSuccessful()) {
downloadFailed(response.code());
return;
}
final ResponseBody body = response.body();
downloadedBytes = 0;
totalBytes = body.contentLength();
final InputStream in;
final OutputStream fout;
in = body().byteStream();
fout = new FileOutputStream(zip);
int read;
while ((read = in.read(BUFFER)) != -1) {
fout.write(BUFFER, 0, read);
downloadedBytes += read;
LOG.d("wrote %d bytes (total: %d)", downloadedBytes, totalBytes);
}
body.close();
fout.close();
}
This whole thing runs on a background thread and I don't get any log lines with "wrote x bytes (total: y)" in the log, which tells me that it's not streaming.
In a previous implementation I was running this stream directly into a ZipInputStream to unzip on the fly, I split the process into download and decompress steps thinking that the ZipInputStream might be the problem. The process fails during download, which means once I fix this I can revert back to unzip on the fly.
What am I doing wrong?
I've also tried final InputStream in = response.body().source().inputStream(); with the same results. I've also tried enqueue instead of execute with the same results.

So, this is embarrassing, but I'll leave it here in case others come across a similar scenario.
The answer to my question is: "Nothing". I'm doing everything right. At least according to the above code.
What I were doing wrong though was that for debugging purposes I had logging level set to BODY, which means it has to put the whole thing in a buffer. That's also why no incremental progress was being reported.

Android: Read audio data from file uri

I want to analyse an audio file (mp3 in particular) which the user can select and determine what notes are played, when they're player and with what frequency.
I already have some working code for my computer, but I want to be able to use this on my phone as well.
In order to do this however, I need access to the bytes of the audio file. On my PC I could just open a stream and use AudioFormat to decode it and then read() the bytes frame by frame.
Looking at the Android Developer Forums I can only find classes and examples for playing a file (without access to the bytes) or recording to a file (I want to read from a file).
I'm pretty confident that I can set up a file chooser, but once I have the Uri from that, I don't know how to get a stream or the bytes.
Any help would be much appreciated :)
Edit: Is a similar solution to this possible? Android - Read a File
I don't know if I could decode the audio file that way or if there would be any problems with the Android API...

So I solved it in the following way:
Get an InputStream with
final InputStream inputStream = getContentResolver().openInputStream(selectedUri);
Then pass it in this function and decode it using classes from JLayer:
private synchronized void decode(InputStream in)
throws BitstreamException, DecoderException {
ArrayList<Short> output = new ArrayList<>(1024);
Bitstream bitstream = new Bitstream(in);
Decoder decoder = new Decoder();
float total_ms = 0f;
float nextNotify = -1f;
boolean done = false;
while (! done) {
Header frameHeader = bitstream.readFrame();
if (total_ms > nextNotify) {
mListener.OnDecodeUpdate((int) total_ms);
nextNotify += 500f;
}
if (frameHeader == null) {
done = true;
} else {
total_ms += frameHeader.ms_per_frame();
SampleBuffer buffer = (SampleBuffer) decoder.decodeFrame(frameHeader, bitstream); // CPU intense
if (buffer.getSampleFrequency() != 44100 || buffer.getChannelCount() != 2) {
throw new DecoderException("mono or non-44100 MP3 not supported", null);
}
short[] pcm = buffer.getBuffer();
for (int i = 0; i < pcm.length-1; i += 2) {
short l = pcm[i];
short r = pcm[i+1];
short mono = (short) ((l + r) / 2f);
output.add(mono); // RAM intense
}
}
bitstream.closeFrame();
}
bitstream.close();
mListener.OnDecodeComplete(output);
}
The full project (in case you want to look up the particulars) can be found here:
https://github.com/S7uXN37/MusicInterpreterStudio/

How to post large video on a server, in Android?

I am trying to post a large video (nearly 1 GB).
I am using FTP to send video to a server, but the upload stops after a while. On the server the video crashes, but I am able to upload a smaller sized video.
I've also used HTTP to send video to the server, sent as a Base64 enoded string, but there is an out of memory exception while encoding.
I've tried to upload the video as a file, but without success. What is the best way to upload a large video to a server?

Use HTTP POST, and post content as Form-based File Upload (mime type: multipart/form-data). This system is standard on web for sending forms and/or uploading files.
Use HTTP chunked post mode, so that size doesn't need to be known beforehand, and you can stream any file in small parts. You still have to make some code on server (e.g. in PHP) to accept the file and do what is needed.
Use HttpURLConnection to initiate connection. Then use my attached class to send the data. It will create proper headers, etc, and you'll use it as OutputStream to write your raw data to it, then call close, and you're done. You can overrite its onHandleResult to handle resulting error code.
public class FormDataWriter extends FilterOutputStream{
private final HttpURLConnection con;
/**
* #param formName name of form in which data are sent
* #param fileName
* #param fileSize size of file, or -1 to use chunked encoding
*/
FormDataWriter(HttpURLConnection con, String formName, String fileName, long fileSize) throws IOException{
super(null);
this.con = con;
con.setDoOutput(true);
String boundary = generateBoundary();
con.setRequestProperty(HTTP.CONTENT_TYPE, "multipart/form-data; charset=UTF-8; boundary="+boundary);
{
StringBuilder sb = new StringBuilder();
writePartHeader(boundary, formName, fileName==null ? null : "filename=\""+fileName+"\"",
"application/octet-stream", sb);
headerBytes = sb.toString().getBytes("UTF-8");
sb = new StringBuilder();
sb.append("\r\n");
sb.append("--"+boundary+"--\r\n");
footerBytes = sb.toString().getBytes();
}
if(fileSize!=-1) {
fileSize += headerBytes.length + footerBytes.length;
con.setFixedLengthStreamingMode((int)fileSize);
}else
con.setChunkedStreamingMode(0x4000);
out = con.getOutputStream();
}
private byte[] headerBytes, footerBytes;
private String generateBoundary() {
StringBuilder sb = new StringBuilder();
Random rand = new Random();
int count = rand.nextInt(11) + 30;
int N = 10+26+26;
for(int i=0; i<count; i++) {
int r = rand.nextInt(N);
sb.append((char)(r<10 ? '0'+r : r<36 ? 'a'+r-10 : 'A'+r-36));
}
return sb.toString();
}
private void writePartHeader(String boundary, String name, String extraContentDispositions, String contentType, StringBuilder sb) {
sb.append("--"+boundary+"\r\n");
sb.append("Content-Disposition: form-data; charset=UTF-8; name=\""+name+"\"");
if(extraContentDispositions!=null)
sb.append("; ").append(extraContentDispositions);
sb.append("\r\n");
if(contentType!=null)
sb.append("Content-Type: "+contentType+"\r\n");
sb.append("\r\n");
}
#Override
public void write(byte[] buffer, int offset, int length) throws IOException{
if(headerBytes!=null) {
out.write(headerBytes);
headerBytes = null;
}
out.write(buffer, offset, length);
}
#Override
public void close() throws IOException{
flush();
if(footerBytes!=null) {
out.write(footerBytes);
footerBytes = null;
}
super.close();
int code = con.getResponseCode();
onHandleResult(code);
}
protected void onHandleResult(int code) throws IOException{
if(code!=200 && code!=201)
throw new IOException("Upload error code: "+code);
}
}

I guess it failed because of a timeout by the big size.
Since
Small size video uploaded successfully
, My suggestion is
Split one big file to several small files.
Upload one by one or several together based on the condition of network.
Join all of parts (after all of those uploaded successfully) at server.
Because of small size, Re-upload failed part will be easy.
Just a theroy.
This site may help .
Added 08.01.2013
It has been a while, don't know if you still need this. Anyway, I wrote some simple codes implement the theory above, because of interest mainly.
Split one big file to several small files. Read the big file into several small parts.
ByteBuffer bb = ByteBuffer.allocate(partSize);
int bytesRead = fc.read(bb);
if (bytesRead == -1) {
break;
}
byte[] bytes = bb.array();
parts.put(new Part(createFileName(fileName, i), bytes));
Upload one by one or several together based on the condition of network.
Part part = parts.take();
if (part == Part.NULL) {
parts.add(Part.NULL);// notify others to stop.
break;
} else {
uploader.upload(part);
}
Join all of parts (after all of those uploaded successfully) at server.
Because it is via HTTP, so it can be in any language, such as Java, PHP, Python, etc. Here is a java example.
...
try (FileOutputStream dest = new FileOutputStream(destFile, true)) {
FileChannel dc = dest.getChannel();// the final big file.
for (long i = 0; i < count; i++) {
File partFile = new File(destFileName + "." + i);// every small parts.
if (!partFile.exists()) {
break;
}
try (FileInputStream part = new FileInputStream(partFile)) {
FileChannel pc = part.getChannel();
pc.transferTo(0, pc.size(), dc);// combine.
}
partFile.delete();
}
statusCode = OK;// set ok at last.
} catch (Exception e) {
log.error("combine failed.", e);
}
I put all codes on GitHub. And made a Android example too.
Please have a look if you still need.

private HttpsURLConnection conn = null;
conn.setDoInput(true);
conn.setDoOutput(true);
conn.setUseCaches(false);
conn.setChunkedStreamingMode(1024);

Loading and executing a binary file on Android

Is there any way to execute Binary file in an android application. without JNI wrapper approach.
plz give me sample codes.

try this
public void pump(InputStream in, OutputStream out, int size) {
byte[] buffer = new byte[4096]; // Or whatever constant you feel like using
int done = 0;
while (done < size) {
int read = in.read(buffer);
if (read == -1) {
throw new IOException("Something went horribly wrong");
}
out.write(buffer, 0, read);
done += read;
}
// Maybe put cleanup code in here if you like, e.g. in.close, out.flush, out.close
}
from this link Reading and writing binary file in Java (seeing half of the file being corrupted)