Develop Reference

FTPS over SSL/TLS using Apache Commons to upload a file fails

I am trying to implement a FTP file upload in android, using Apache commons library. The communication must be done through explicit TLS authentication. I can login, connect to server and list files successfully, but when i am trying to store a file to the server, it creates a file in the destination folder of the server with size 0 and throwing SSLException: Write error, broken pipe.
How to overcome this?
Below is my code:
FTPSClient ftpClient = new FTPSClient("TLS", false);
ftpClient.addProtocolCommandListener(new PrintCommandListener(new
KeyManagerFactory kmf = getInstance(KeyManagerFactory.getDefaultAlgorithm());
kmf.init(null, null);
KeyManager km = kmf.getKeyManagers()[0];
ftpClient.setKeyManager(km);
ftpClient.setBufferSize(1000);
ftpClient.setConnectTimeout(5000);
ftpClient.connect(InetAddress.getByName("server ip address"), 990);
// Set protection buffer size
ftpClient.execPBSZ(0);
// // Set data channel protection to private
ftpClient.execPROT("P");
ftpClient.login("username", "password");
ftpClient.changeWorkingDirectory("/");
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
ftpClient.enterLocalPassiveMode();
buffInp = new BufferedInputStream(new FileInputStream(file.getAbsolutePath()));
//throwing exception here
boolean status = ftpClient.storeFile( picture.getName(), buffInp );

Register pjsip account using Tls setting in pjsua

I am using Pjusa for Voip, I have done communication through UDP and TCP but could not communitcate through TLS setting`
TlsConfig tlsConfig = new TlsConfig();
File cacheDir = new File(ctx.getCacheDir(), "ca");
InputStream input = ctx.getResources().openRawResource(R.raw.ca);
tlsConfig.setCaListFile(cacheDir.getPath());
input = ctx.getResources().openRawResource(R.raw.secure);
cacheDir = new File(ctx.getCacheDir(), "secure");
tlsConfig.setCertFile(cacheDir.getPath());
tlsConfig.setMethod(pjsip_ssl_method.PJSIP_SSLV23_METHOD);`
I have attached CaListFile and setCertFile files like that method and that setting working but When I establish a call then I found error like that
tlsc0x9e7db014 TLS connect() error: Connection refused [code=120111]
tsx0x9d945864 Failed to send Request msg INVITE/cseq=25416 (tdta0x9d991000)! err=120111 (Connection refused)
In IOS I found some more setting for TLS , but could not implement in PJSUA
any one can help me out how to use that piece of code in android in PJSUA library
pj_sockaddr_in remote;
pjsip_transport *transport;
remote.sin_family = pj_AF_INET();
remote.sin_zero_len = 0;
remote.sin_addr.s_addr = pj_inet_addr(#"xxx.xx.xx.xx").s_addr;
remote.sin_addr.s_addr = INADDR_ANY;
bzero(&(remote.sin_zero),sizeof(remote.sin_zero));
app_config->cfg.use_srtp = PJMEDIA_SRTP_MANDATORY;
app_config->cfg.srtp_secure_signaling = 1;
thanks in advance

TlsConfig tlsConfig = new TlsConfig();
tlsConfig.setCertFile(certFile);
sipTpConfig.setTlsConfig(tlsConfig);
tlsConfig.setCaListFile(caListFile);
I did R&D from web searching and reading document of SIP and PJSIP, then I find the sluotion for accountConfig for PJMEDIA_SRTP_MANDATORY and setSrtpSecureSignaling(1) by that way provided below
accCfg.getMediaConfig().setSrtpUse(pjmedia_srtp_use.PJMEDIA_SRTP_MANDATORY);
accCfg.getMediaConfig().setSrtpSecureSignaling(1);
Hence TLS working with CAListFile and CertificateFile

Android Orbot malformed reply from SOCKS server

I am trying to enable TOR support on my own XMPP app in android. I am using orbot as TOR proxy and I have the following code to connect app socket:
socket = new Socket(new Proxy(Proxy.Type.SOCKS, new InetSocketAddress("127.0.0.1", 9050)));
socket.connect(addr, Config.SOCKET_TIMEOUT * 1000);
but I am getting Malformed reply from SOCKS server even that my Orbot is up and running. I believe that this error is thrown when app cant access proxy server or mentioned server is not SOCKS proxy.
I have also tried to use jsocks:
Socks5Proxy sProxy = new Socks5Proxy("127.0.0.1", 9050);
sProxy.resolveAddrLocally(false);
String host = account.getServer().toString();
int port = 5222;
System.out.println(host + ":" + port);
try {
socket = new SocksSocket(sProxy, host, port);
}catch(SocksException sock_ex){
System.err.println("SocksException:"+sock_ex);
}
System.out.println("here we are");
where host is "jabbim.com"
But I never get to "here we are" println so it looks like my app hangs somewhere on creating SocksSocket but I am not getting any errors either. When I debug it it hangs on impl.getInputStream somewhere in jsocks (PlainSocketImpl I believe)
Any idea how to fix this?
Thanks in forward

I solved this using
compile "org.igniterealtime.smack:smack-android-extensions:4.1.4"
compile "org.igniterealtime.smack:smack-tcp:4.1.4"
and then connecting using this code:
final Random rndForTorCircuits = new Random();
String user = rndForTorCircuits.nextInt(100000) + "";
String pass = rndForTorCircuits.nextInt(100000) + "";
ProxyInfo proxyInfo = new ProxyInfo(ProxyInfo.ProxyType.SOCKS5, "127.0.0.1", 9050, user, pass);
socket = proxyInfo.getSocketFactory().createSocket(addr.getHostName(), addr.getPort());
by using this you get socket that is already connected to proxy.

IHE and HL7. PCD-01 ACK

I'm trying to get data from a monitor to an Android application and I've took the IHE - PCD-01 transaction as a model.
The scheme is simple, is based on achieve the interconnection between the monitor and the tablet, where the monitor sends constantly information and the application is listening.
But what I don't understand is if I need an ACK or not after every message. Does anyone can help me with this?

TL;DR yes, nothing special here, support the usual HL7 ACK/NACK driven by MSH-15, MSH-16 fields. ACK-ing everything by default is "better safe then sorry"
The document "IHE Patient Care Device (PCD), Technical Framework, Volume 2 (PCD TF-2) Transactions, Revision 1.0 - Final Text, August 12, 2011" available at http://www.ihe.net/technical_framework/upload/ihe_pcd_tf_vol2_ft_2011-08-12.pdf says
..The common static definition of the HL7 acknowledgement (ACK) message is described in Appendix G, "HL7 Implementation Notes"..
which says
G.1 Network Guidelines
The HL7 2.6 standard does not define a network communications protocol. Beginning with HL7 2.2, the definitions of lower layer protocols were moved to the Implementation Guide, but are not HL7 requirements. The IHE Framework makes these recommendations:
Applications shall use the Minimal Lower Layer Protocol defined in Appendix C of the HL7 Implementation Guide.
An application that wants to send a message (initiate a transaction) will initiate a network connection to start the transaction. The receiver application will respond with an acknowledgement or response to query but will not initiate new transactions on this network connection
G.1.1 Acknowledgment Modes
ACKNOWLEDGMENT MESSAGES
Acknowledgment messages may be defined on an application basis. However the simple general acknowledgment message (ACK) may be used where the application does not define a special message (application level acknowledgment) and in other cases as described in Section 2.9, "Message Processing Rules".
The IHE PCD transaction PCD-03 supports „enhanced mode‟ acknowledgements. See discussion under PCD-03 Transactions as well as in B.1 MSH – Message Header Segment and B.2 MSA – Message Acknowledgement Segment
and document "Health Level Seven, Version 2.6 © 2007, Chapter 2: Control" coming from the "HL7 Messaging Standard Version 2.6" package which can be downloaded from http://www.hl7.org/implement/standards/product_brief.cfm?product_id=185 describes the accept and validate behavior in
2.9.2 Message response using the original processing rules
..too long to quote..
2.9.3 Response using enhanced acknowledgement
..too long to quote..
depending on the values of MSH-15 Accept Acknowledgement Type and MSH-16 Application Acknowledgment Type fields in the HL7 message
The above chapters from the HL7 standard contain what you want to read and implement/support.
EDIT:
Simply put, in HL7 protocol in every message sent the sender may request an ACK receipt by flagging appropriate fields in the message header segment. IHE does not remove this rule and does not enforce any other but enables any other convention to be defined on an application basis. Correct expected behavior is defined by the HL7 specification and in order to get it right and create a conforming implementation (without hidden surprises for your 3rd parties) you may need to read it several times (see also Stack Overflow: How can I make my system HL7 certified?)
For example this is how HAPI library handles the ACKing, snippet comes from http://sourceforge.net/p/hl7api/code/764/tree/tags/Root_REL_1_2/hapi-mvn/hapi-base/src/main/java/ca/uhn/hl7v2/protocol/impl/ProcessorImpl.java
/**
* #see ca.uhn.hl7v2.protocol.Processor#cycle(boolean)
*/
public void cycle(boolean expectingAck) throws HL7Exception {
log.debug("In cycle({})", expectingAck);
cleanReservations();
cleanAcceptAcks();
cleanReservedMessages();
Transportable in = null;
try {
if (expectingAck) {
in = tryReceive(myContext.getLocallyDrivenTransportLayer());
} else {
in = tryReceive(myContext.getRemotelyDrivenTransportLayer());
}
} catch (TransportException e) {
try {
Thread.sleep(1000);
} catch (InterruptedException e1) {}
throw e;
}
// log
if (in != null) {
log.debug("Received message: {}", in.getMessage());
} else {
log.debug("Received no message");
}
// If we have a message, handle it
if (in != null) {
String acceptAckNeeded = null;
// String appAckNeeded = null;
String ackCode = null;
String ackId = null;
try {
String[] fieldPaths = {"MSH-15", "MSH-16", "MSA-1", "MSA-2"};
String[] fields = PreParser.getFields(in.getMessage(), fieldPaths);
acceptAckNeeded = fields[0];
// appAckNeeded = fields[1];
ackCode = fields[2];
ackId = fields[3];
} catch (HL7Exception e) {
log.warn("Failed to parse accept ack fields in incoming message", e);
}
if (ackId != null && ackCode != null && ackCode.startsWith("C")) {
long expiryTime = System.currentTimeMillis() + 1000 * 60;
myAcceptAcks.put(ackId, new ExpiringTransportable(in, expiryTime));
} else {
AcceptAcknowledger.AcceptACK ack = AcceptAcknowledger.validate(getContext(), in);
if ((acceptAckNeeded != null && acceptAckNeeded.equals(AL))
|| (acceptAckNeeded != null && acceptAckNeeded.equals(ER) && !ack.isAcceptable())
|| (acceptAckNeeded != null && acceptAckNeeded.equals(SU) && ack.isAcceptable())) {
trySend(myContext.getRemotelyDrivenTransportLayer(), ack.getMessage());
}
if (ack.isAcceptable()) {
if (isReserved(ackId)) {
log.debug("Received expected ACK message with ACK ID: {}", ackId);
removeReservation(ackId);
long expiryTime = System.currentTimeMillis() + 1000 * 60 * 5;
myAvailableMessages.put(ackId, new ExpiringTransportable(in, expiryTime));
} else {
log.debug("Sending message to router");
Transportable out = myContext.getRouter().processMessage(in);
sendAppResponse(out);
}
} else {
// TODO: should we do something more here? Might be nice to
// allow a configurable handler for this situation
log.warn("Incoming message was not acceptable");
}
}
} else {
String transport = expectingAck ? " Locally driven " : "Remotely driven";
log.debug("{} TransportLayer.receive() returned null.", transport);
}
sleepIfNeeded();
log.debug("Exiting cycle()");
}

Thanks for your answer :)
of course that it is better to use an ACK to make sure if the receiver is getting the message but what I wanted to know if it was mandatory or not using the PCD-01 transaction.
I've read your documents and what I've understood is that the use of ACK depends on the MSH-15 and MSH-16 fields content, but with the following information:
An application that wants to send a message (initiate a transaction) will initiate a network connection to start the transaction. The receiver application will respond with an acknowledgement or response to query but will not initiate new transactions on this network connection
I understand that the ACK is only at the beginning of the connection not after every message, is it right?

Android's SSLServerSocket causes increasing native memory in the App, OOM

Background
I am developing an Android App which provides a simple HTTP/HTTPS server. If the HTTPS serving is configured then on every connection an increasing native memory usage is observed which eventually leads to an app crash (oom), while using the HTTP configuration keeps the native memory usage relative constant. The app's Java VM keeps relative constant in both configurations.
The app serves an HTML page which contains a javascript with periodic polling (one json poll every second), so calling the app page using the HTTPS configuration and keeping the page open for several hours will lead to the mentioned out-of-memory because of increasing native memory usage. I have tested many SSLServerSocket and SSLContext configurations found on internet with no luck.
I observe the same problem on various Android devices and various Android versions beginning with 2.2 up to 4.3.
The code for handling client requests is the same for both configurations HTTP/HTTPS. The only difference between the two configurations is the setup of the server socket. While in the case of HTTP server socket one single line similar to this "ServerSocket serversocket = new ServerSocket(myport);" does the job, in the case of HTTPS server setup the usual steps for setting up the SSLContext are taken -- i.e. setting up the keymanager and initializing the SSLContext. For now, I use the default TrustManager.
Need For Your Advice
Does somebody know about any memory leak problems in Android's default TLS Provider using OpenSSL? Is there something special I should consider to avoid the leak in the native memory? Any hint is highly appreciated.
Update: I have also tried both TLS providers: OpenSSL and JSSE by explicitly giving the provider name in SSLContext.getInstance( "TLS", providerName ). But that did not change anything.
Here is a code block which demonstrates the problem. Just create a sample app put it into the bottom of the main activity's onCreate and build & run the app. Make sure that your Wifi is on and call the HTML page by following address:
https://android device IP:9090
Then watch the adb logs, after a while you will see the native memory beginning to increase.
new Thread(new Runnable() {
public void run() {
final int PORT = 9090;
SSLContext sslContext = SSLContext.getInstance( "TLS" ); // JSSE and OpenSSL providers behave the same way
KeyManagerFactory kmf = KeyManagerFactory.getInstance( KeyManagerFactory.getDefaultAlgorithm() );
KeyStore ks = KeyStore.getInstance( KeyStore.getDefaultType() );
char[] password = KEYSTORE_PW.toCharArray();
// we assume the keystore is in the app assets
InputStream sslKeyStore = getApplicationContext().getResources().openRawResource( R.raw.keystore );
ks.load( sslKeyStore, null );
sslKeyStore.close();
kmf.init( ks, password );
sslContext.init( kmf.getKeyManagers(), null, new SecureRandom() );
ServerSocketFactory ssf = sslContext.getServerSocketFactory();
sslContext.getServerSessionContext().setSessionTimeout(5);
try {
SSLServerSocket serversocket = ( SSLServerSocket )ssf.createServerSocket(PORT);
// alternatively, the plain server socket can be created here
//ServerSocket serversocket = new ServerSocket(9090);
serversocket.setReceiveBufferSize( 8192 );
int num = 0;
long lastnatmem = 0, natmemtotalincrease = 0;
while (true) {
try {
Socket soc = (Socket) serversocket.accept();
Log.i(TAG, "client connected (" + num++ + ")");
soc.setSoTimeout(2000);
try {
SSLSession session = ((SSLSocket)soc).getSession();
boolean valid = session.isValid();
Log.d(TAG, "session valid: " + valid);
OutputStream os = null;
InputStream is = null;
try {
os = soc.getOutputStream();
// just read the complete request from client
is = soc.getInputStream();
int c = 0;
String itext = "";
while ( (c = is.read() ) > 0 ) {
itext += (char)c;
if (itext.contains("\r\n\r\n")) // end of request detection
break;
}
//Log.e(TAG, " req: " + itext);
} catch (SocketTimeoutException e) {
// this can occasionally happen (handshake timeout)
Log.d(TAG, "socket timeout: " + e.getMessage());
if (os != null)
os.close();
if (is != null)
is.close();
soc.close();
continue;
}
long natmem = Debug.getNativeHeapSize();
long diff = 0;
if (lastnatmem != 0) {
diff = natmem - lastnatmem;
natmemtotalincrease += diff;
}
lastnatmem = natmem;
Log.i(TAG, " answer the request, native memory in use: " + natmem / 1024 + ", diff: " + diff / 1024 + ", total increase: " + natmemtotalincrease / 1024);
String html = "<!DOCTYPE html><html><head>";
html += "<script type='text/javascript'>";
html += "function poll() { request(); window.setTimeout(poll, 1000);}\n";
html += "function request() { var xmlHttp = new XMLHttpRequest(); xmlHttp.open( \"GET\", \"/\", false ); xmlHttp.send( null ); return xmlHttp.responseText; }";
html += "</script>";
html += "</head><body onload=\"poll()\"><p>Refresh the site to see the inreasing native memory when using HTTPS: " + natmem + " </p></body></html> ";
byte[] buffer = html.getBytes("UTF-8");
PrintWriter pw = new PrintWriter( os );
pw.print("HTTP/1.0 200 OK \r\n");
pw.print("Content-Type: text/html\r\n");
pw.print("Content-Length: " + buffer.length + "\r\n");
pw.print("\r\n");
pw.flush();
os.write(buffer);
os.flush();
os.close();
} catch (IOException e) {
e.printStackTrace();
}
soc.close();
}
catch (IOException e) {
e.printStackTrace();
}
}
} catch (SocketException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}).start();
-- EDIT --
I have uploaded a sample app project called SSLTest for eClipse which demonstrates the problem:
http://code.google.com/p/android/issues/detail?id=59536
-- UPDATE --
Good news: today the reported Android issue above was identified and proper submissions were made to fix the memory leak. For more details see the link above.

I imagine this would be a substantial time investment, but I see that Valgrind has been ported to Android. You could try getting that up and running. Of course, if you find there's an internal memory leak, there isn't a lot you can do about it except attempt to get the bug fixed in future Android releases.
As a workaround, you could make your application multi-process and put the https service in a separate process. That way you could restart it periodically, avoiding OOM. You might also have to have a third process just accepting port 443 connections and passing them on to the https worker - in order to avoid tiny outages when the https worker is restarted.
This also sounds like a substantial time investment :) But it would presumably successfully avoid the problem.
--- EDIT: More detail ---
Yes, if you have a main application with its own UI, a worker process for handling SSL and a worker process for accepting the SSL requests (which as you say probably can't be 443), then on top of your normal Activity classes, you would have two Service classes, and the manifest would place them in separate processes.
Handling SSL process: Rather than waiting for an OOM to crash the service, the service could monitor its own Debug.getNativeHeapSize(), and explicitly restart the service when it increased too much. Either that, or restart automatically after every 100 requests or so.
Handling listening socket process: This service would just listen on the TCP port you choose and pass on the raw data to the SSL process. This bit needs some thought, but the most obvious solution is to just have the SSL process listen on a different local port X (or switch between a selection of different ports), and the listening socket process would forward data to port X. The reason for having the listening socket process is to gracefully handle the possibility that X is down - as it might be whenever you restart it.
If your requirements allow for there being occasional mini-outages I would just do the handling SSL process, and skip the listening socket process, it's a relatively simple solution then - not that different to what you'd do normally. It's the listening socket process that adds complexity to the solution...

Does it help to explicitly close the input stream? In the sample code the input stream seems to only be closed in the case of a SocketTimeoutException exception.
--EDIT--
You could rename run() to run2() and move the while loop into run() and remove it from run2() and see if that makes a difference? This couldn't be a solution but would tell you if any of the long-lived objects free up the memory when their references are dropped.

There is one detail I would recommend changing in your implementation.
Make a list of all your resource variables, for example Sockets, Streams, Writers, etc. Be sure to have the declaration outside your try statement and be sure to do cleanup / closing in the finally statement. I normally do something like this to be 100% sure:
InputStream in = null;
OutputStream out = null;
try {
//assign a proper value to in and out, and use them as needed.
} catch(IOException e) {
//normal error handling
} finally {
try {
in.close();
} catch(IOException e) {}
try {
out.close();
} catch(IOException e) {}
}
It looks a little bit confusing, but imagine you use your in Stream inside the try block and you get some Exception, then your Streams never get closed and that is a potential reason for memory leaks.
I cannot guarantee that this is the reason, but it should be a good startup point.
About managing your service. I had a lot of bad experiences with Android services because I was running them in the same thread as the GUI. Under some circumstances, Android will see some code that is executing for too long and kill your main process in order to protect from crashes. The solution I found was to follow the suggestion from this tutorial (look at point 4):
http://www.vogella.com/articles/AndroidServices/article.html
After this, my service just worked as expected and didn't interfere with my GUI Process.
Regards