I want to send UTF-8 text that is stored with ElasticSeach to an application via sockets.
I have a ThreadedTCPServer Implemented, here is the class that should handle the reply.
I have implemented basic string based handshaking to share some info like query was sent and that response will be sent.
class ThreadedTCPRequestHandler(SocketServer.BaseRequestHandler):
def handle(self):
es = Elasticsearch()
#receive query from the client
query = self.request.recv(1024)
#Cut off the characters that aren't recognized
query=query[2:]
#for testing
query=query.lower().strip().replace(' ','_')
print query
#Send response that query was received
self.request.send("200...OK\n")
res = es.search(index="painters",body={"query": { "match" : {"title" : query}},"size":1 })
if res['hits']['hits']:
response = res['hits']['hits'][0]['_source']['text']
self.request.send("201...RE\n")
print response
response=response.encode('utf-8')
self.request.sendall(response)
On the android side I have two functions one for reading responses and one for reading bytes.
private String getResponse(InputStream is){
String line="";
BufferedReader rd = new BufferedReader(new InputStreamReader(is),8);
try{
line=rd.readLine();
}
catch (Exception e){
Toast.makeText(MainActivity.this, "Stream Exception", Toast.LENGTH_SHORT).show();
}
return line;
}
private String convertStreamToString(InputStream is) {
BufferedInputStream bi = new BufferedInputStream(is);
byte[] b = new byte[1024];
StringBuilder total = new StringBuilder();
try {
while (bi.read(b,0,1024)!=-1)
{
total.append(decodeUTF8(b));
Log.d("TOTAL",decodeUTF8(b));
}
} catch (Exception e) {
e.printStackTrace();
}
return total.toString();
}
And here is the function that should decode the string:
String decodeUTF8(byte[] bytes) {
return new String(bytes, UTF8_CHARSET);
}
Problem is Sometimes not the whole string is shown on the android Side,
and when the whole thing goes through some UTF-8 Characters end up deformed (totally different character than sent)
AsyncTask post execute that starts new Activty:
protected void onPostExecute(String s) {
//super.onPostExecute(s);
if (s.contains("ECONNREFUSED")){
Toast.makeText(MainActivity.this,"Connection Failed",Toast.LENGTH_LONG).show();
return;
}
Intent intent = new Intent(MainActivity.this,ReplyActivity.class);
intent.putExtra(EXTRA_MESSAGE,s);
startActivity(intent);
}
New Intent getting the string:
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
//get message
Intent intent = getIntent();
String summary = intent.getStringExtra(MainActivity.EXTRA_MESSAGE);
Example ouput:
Early life (1928–1949)
Andy Warhol ("né" Andrej Varhola, Jr.) was born on August 6, 1928 in Pittsburgh, Pennsylvania. He was the fourth child of Andrij Warhola (Americanized as Andrew Warhola, Sr., 1889–1942) and Júlia ("née" Zavacká, 1892–1972), w
As you can see even when sending the query from android to python I get some crap that I need to cut off.
here:
#Cut off the characters that aren't recognized
query=query[2:]
repr(response):
<h2>Early life (1928\xe2\x80\x931949)</h2>\nAndy Warhol ("n\xc3\xa9" Andrej Varhola, Jr.) was born on August 6, 1928 in Pittsburgh, Pennsylvania. He was the fourth child of Andrij Warhola (Americanized as Andrew Warhola, Sr., 1889\xe2\x80\x931942) and J\xc3\xbalia ("n\xc3\xa9e" Zavack\xc3\xa1, 1892\xe2\x80\x931972), whose first child was born in their homeland and died before their move to the U.S.
Terminal print:
<h2>Early life (1928–1949)</h2>
Andy Warhol ("né" Andrej Varhola, Jr.) was born on August 6, 1928 in Pittsburgh, Pennsylvania. He was the fourth child of Andrij Warhola (Americanized as Andrew Warhola, Sr., 1889–1942) and Júlia ("née" Zavacká, 1892–1972), whose first child was born in their homeland and died before their move to the U.S.
Related
I am new in Android, and I´m making an app and I want to focus on the secure of the app, I found this link, in it says "keep sensitive information in RAM for the minimum time possible by setting it to null after use." and later it says "avoid the use of Java’s String class to hold sensitive information. Instead use char arrays or byte arrays. The reason for this is because Strings are immutable"
In my app I have a code similar to this (this code just checks a PIN the users enters and compares it with another one in the internal storage):
public class Class extends Activity implements OnClickListener{
private static final String fileName = "FilePin";
private Button button;
private EditText editText = null;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.layout_example);
editText = (editText) findViewById(R.id.editText);
button = (Button) findViewById(R.id.button);
button.setOnClickListener(this);
}
#Override
public void onClick(View v){
if (readPin()) {
textView.setText(new char[]{' '}, 0, 0);
Intent intent = new Intent(this, OtherClass.class);
startActivity(intent);
}
}
// this method read the file where the PIN the user create is save in the internal storage
public boolean readPin(){
StringBuilder stringBuilder = null;
StringBuilder inputString;
try {
BufferedReader inputReader = new BufferedReader(new InputStreamReader
(openFileInput(fileName)));
stringBuilder = new StringBuilder();
while ((inputString = new StringBuilder(inputReader.readLine())) != null) {
stringBuilder.append(inputString);
}
inputReader.close();
} catch (Exception e) {
e.printStackTrace();
}
inputString = new StringBuilder("");
assert stringBuilder != null;
boolean comparePin = compare(stringBuilder);
stringBuilder = new StringBuilder("");
return comparePin;
}
// this method compare the PIN saved with the PIN the users enters
private boolean compare(StringBuilder pinSaved){
if (!editText.getText().toString().equals(pinSaved.toString())) {
Toast.makeText(getBaseContext(), "the PIN it´s incorrect"
, Toast.LENGTH_SHORT).show();
pinSaved = new StringBuilder("");
return false;
}
else {
pinSaved = new StringBuilder("");
return true;
}
}
}
For what I read in the previews link I didn´t use String instead I use StringBuilder because StringBuilder are mutable and after I use it a change the value to "stringBuilder = new StringBuilder("");", I didn´t use char[] because I don´t know how to save the editText to a char[] variable or how to save the PIN saved in the file in a char[] variable and I didn´t find examples about how to use char[] in those cases.
My questions are: this case is secure for an android app or is it better to change to char[] variables?, Is StringBuffer class insecure for Android? How can I save the editText value in a char[]? How Can I save a file in a char[] variable?
stringBuilder = new StringBuilder("");
The above is weaker than zeroing out a char[]. The original StringBuilder, along with is internal buffer, will stay in memory. At some point, the garbage collector will mark the memory as free, and later still, something might overwrite it.
Zeroing out a char[] gives you more control over when the secret is overwritten.
Also, StringBuilder comes with logic that automatically copies its internal char[] buffer when it needs more room. You need to be careful to make sure it never does this. A plain char[] is inconvenient because you can't resize it, but it's good in this case, because any attempt to copy it is explicit.
See Java: convert a char[] to a CharSequence for how to get your char[] into a form that you can pass to EditText.setText.
I am making an app in which I want to get the current time from internet.
I know how to get the time from the device using System.currentTimeMillis, and even after searching a lot, I did not get any clue about how to get it from internet.
You can get time from internet time servers using the below program
import java.io.IOException;
import org.apache.commons.net.time.TimeTCPClient;
public final class GetTime {
public static final void main(String[] args) {
try {
TimeTCPClient client = new TimeTCPClient();
try {
// Set timeout of 60 seconds
client.setDefaultTimeout(60000);
// Connecting to time server
// Other time servers can be found at : http://tf.nist.gov/tf-cgi/servers.cgi#
// Make sure that your program NEVER queries a server more frequently than once every 4 seconds
client.connect("time.nist.gov");
System.out.println(client.getDate());
} finally {
client.disconnect();
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
1.You would need Apache Commons Net library for this to work. Download the library and add to your project build path.
(Or you can also use the trimmed Apache Commons Net Library here : https://www-us.apache.org/dist//commons/net/binaries/commons-net-3.6-bin.tar.gz This is enough to get time from internet )
2.Run the program. You will get the time printed on your console.
Here is a method that i have created for you
you can use this in your code
public String getTime() {
try{
//Make the Http connection so we can retrieve the time
HttpClient httpclient = new DefaultHttpClient();
// I am using yahoos api to get the time
HttpResponse response = httpclient.execute(new
HttpGet("http://developer.yahooapis.com/TimeService/V1/getTime?appid=YahooDemo"));
StatusLine statusLine = response.getStatusLine();
if(statusLine.getStatusCode() == HttpStatus.SC_OK){
ByteArrayOutputStream out = new ByteArrayOutputStream();
response.getEntity().writeTo(out);
out.close();
// The response is an xml file and i have stored it in a string
String responseString = out.toString();
Log.d("Response", responseString);
//We have to parse the xml file using any parser, but since i have to
//take just one value i have deviced a shortcut to retrieve it
int x = responseString.indexOf("<Timestamp>");
int y = responseString.indexOf("</Timestamp>");
//I am using the x + "<Timestamp>" because x alone gives only the start value
Log.d("Response", responseString.substring(x + "<Timestamp>".length(),y) );
String timestamp = responseString.substring(x + "<Timestamp>".length(),y);
// The time returned is in UNIX format so i need to multiply it by 1000 to use it
Date d = new Date(Long.parseLong(timestamp) * 1000);
Log.d("Response", d.toString() );
return d.toString() ;
} else{
//Closes the connection.
response.getEntity().getContent().close();
throw new IOException(statusLine.getReasonPhrase());
}
}catch (ClientProtocolException e) {
Log.d("Response", e.getMessage());
}catch (IOException e) {
Log.d("Response", e.getMessage());
}
return null;
}
If you don't care for millisecond accuracy, and if you are already using google firebase or don't mind using it (they provide a free tier), check this out: https://firebase.google.com/docs/database/android/offline-capabilities#clock-skew
Basically, firebase database has a field that provides offset value between the device time and the firebase server time. You can use this offset to get the current time.
DatabaseReference offsetRef = FirebaseDatabase.getInstance().getReference(".info/serverTimeOffset");
offsetRef.addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot snapshot) {
double offset = snapshot.getValue(Double.class);
double estimatedServerTimeMs = System.currentTimeMillis() + offset;
}
#Override
public void onCancelled(DatabaseError error) {
System.err.println("Listener was cancelled");
}
});
As I said, it will be inaccurate based on network latency.
I think the best solution is to use SNTP, in particular the SNTP client code from Android itself, e.g.:
http://grepcode.com/file/repository.grepcode.com/java/ext/com.google.android/android/4.1.1_r1/android/net/SntpClient.java/
I believe Android uses SNTP for automatic date/time updates when a cell network is not available (e.g. wifi tablets).
I think it is better then the other solutions because it uses SNTP/NTP rather then the Time protocol (RFC 868) used by the Apache TimeTCPClient. I don't know anything bad about RFC 868, but NTP is newer and seems to have superceeded it and is more widely used. I believe that Android devices that don't have cellular uses NTP.
Also, because it uses sockets. Some of the solutions proposed use HTTP so they will lose something in their accuracy.
You will need to have access to a webservice that provides current time in XML or JSON format.
If you don't find such type of service, you could parse the time from a web page, like http://www.timeanddate.com/worldclock/, or host your own time service on a server using a simple PHP page for example.
Check out JSoup for the parsing of HTML pages.
Nothing from the above worked from me. This is what I ended up with (with Volley);
This example also converts to another timezone.
Long time = null;
RequestQueue queue = Volley.newRequestQueue(this);
String url ="http://www.timeapi.org/utc/now";
StringRequest stringRequest = new StringRequest(Request.Method.GET, url,
new Response.Listener<String>() {
#Override
public void onResponse(String response) {
try {
SimpleDateFormat simpleDateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss");
simpleDateFormat.setTimeZone(TimeZone.getTimeZone("UTC"));
Date date = simpleDateFormat.parse(response);
TimeZone tz = TimeZone.getTimeZone("Israel");
SimpleDateFormat destFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
destFormat.setTimeZone(tz);
String result = destFormat.format(date);
Log.d(TAG, "onResponse: " + result.toString());
} catch (ParseException e) {
e.printStackTrace();
}
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
Log.w(TAG, "onErrorResponse: "+ error.getMessage());
}
});
queue.add(stringRequest);
return time;
Import Volley in gradle:
compile 'com.android.volley:volley:1.0.0'
There is a clear answer available already in Stackoverflow
https://stackoverflow.com/a/71274296/11789675
Call this url or use as GET API
http://worldtimeapi.org/api/timezone/Asia/Kolkata
the response will be like
{
"abbreviation": "IST",
"client_ip": "45.125.117.46",
"datetime": "2022-02-26T10:50:43.406519+05:30",
}
This thing works best for my apps. I use jsoup to search the google time and gets current time and then I compare the phone time with google time. So if these time are different you can stop user using a dialogbox or alertbox to tell them the times have changed. You can implement in MainActivity to check this condition. Here is a snippet so you get the idea more clearly.
public class HomeActivity extends AppCompatActivity {
//phoneDate and phoneTime to get current phone date and time
String phoneDate = new SimpleDateFormat("dd MMM yyyy ").format(clnd.getTime()).trim();
String phoneTime = new SimpleDateFormat("hh:mm a").format(clnd.getTime()).trim();
String googleDate;
String googleTime ;
#Override
protected void onCreate(Bundle _savedInstanceState) {
super.onCreate(_savedInstanceState);
setContentView(R.layout.home);
Thread thread = new Thread(new Runnable() {
#Override
public void run() {
try {
//URL to search time
String url = "https://www.google.co.in/search?q=time";
Document document = Jsoup.connect(url).get();
org.jsoup.select.Elements time = document.getElementsByClass("gsrt vk_bk FzvWSb YwPhnf");
org.jsoup.select.Elements date = document.getElementsByClass("KfQeJ");
Log.d("HTML", "google date" + String.format(date.text()));
Log.d("HTML", "google time" + time.text());
googleDate = date.text().trim();
googleTime = time.text().trim();
//'0'is not present when hour is single digit
char second = googleTime.charAt(1);
if(second == ':'){
googleTime = "0" + googleTime;
}
Log.d("Proper format", "google time" + googleTime);
Log.d("Date", "your current url when webpage loading.." + phoneDate);
Log.d("Time", "your current url when webpage loading.." + phoneTime);
if(googleDate.contains(phoneDate) && googleTime.equals(phoneTime)){
Log.d("Time", "your current url when webpage loading.." + " true");
}else{
Log.d("Time", "your current url when webpage loading.." + " false");
}
} catch (Exception e) {
e.printStackTrace();
}
}
});
thread.start();
}
}
Ok Here's my problem.
I am making an FQL query and I am storing the response data. My code runs fine, except after about 4 minutes of the app running I get a JSONException. The Exception reads
A JSONArray text must start with '[' at character 0 of
Which is strange, because if I type in the URL the query is making into a browser I get the following
[{"uid":SOMENUMBER},{"uid":SOMENUMBER}]
Which clearly begins with the "[" character.
Here's the request
String query = "SELECT uid FROM user WHERE is_app_user=1 and uid IN (SELECT uid2 FROM friend WHERE uid1 =" + fbId + ")";
Bundle params = new Bundle();
params.putString("method", "fql.query");
params.putString("query", query);
mAsyncFacebookRunner.request(null, params, new FQLRequestListener());
Here's the RequestListener
private class FQLRequestListener implements RequestListener
{
#Override
public void onComplete(String response, Object state)
{
try {
JSONArray json = new JSONArray(response);
friendsIds = new ArrayList<String>();
for(int i = 0; i < json.length(); ++i)
{
String uid = json.getJSONObject(i).getString("uid");
friendsIds.add(uid);
Log.d("friends", friendsIds.get(i));
}
InviteFriends.this.runOnUiThread(new Runnable()
{
#Override
public void run()
{
setUpList();
}
});
} catch (JSONException e) {
e.printStackTrace();
}
}
And the Stack..
06-15 16:43:21.744: D/Facebook-Util(866): GET URL: THIS IS THE URL THAT WORKS IN A BROSWER (actual URL removed)
06-15 16:43:21.774: W/System.err(866): org.json.JSONException: A JSONArray text must start with '[' at character 0 of
06-15 16:43:21.774: W/System.err(866): at org.json.JSONTokener.syntaxError(JSONTokener.java:448)
06-15 16:43:21.774: W/System.err(866): at org.json.JSONArray.<init>(JSONArray.java:104)
06-15 16:43:21.774: W/System.err(866): at org.json.JSONArray.<init>(JSONArray.java:150)
06-15 16:43:21.774: W/System.err(866): at rageup.android.InviteFriends$FQLRequestListener.onComplete(InviteFriends.java:129)
06-15 16:43:21.774: W/System.err(866): at com.facebook.android.AsyncFacebookRunner$2.run(AsyncFacebookRunner.java:254)
I should note that..
Query works when I logout and log back in.
I have ensured my Facebook session is valid.
The Facebook provided "Hackbook" example has the same problem when downloading a friend's list.
Flow of Getting the Exception
Home Page Where User Logs In
User Clicks a Button to Start Second Activity
Second Activity is Where Query is Made
Return Home
Repeat after about 4 minutes, minus the login
First try to validate the response.
I would use:
JSONObject json = new JSONObject(response);
It gives you an opportunity to check the content of the json object using it's methods. You can't assume, that facebook api will always respond expected form of data.
You can do:
Log.w("FacebookListener", "Unexpected response: " + response);
And then inspect Logcat output.
OK. I haven't tested it yet, but according to: http://developers.facebook.com/docs/reference/rest/fql.query/ the graph path for fql queries is "fql"
In mAsyncFacebookRunner.request(null, params, new FQLRequestListener()); you pass null, but "fql" is probably expected
EDIT
I have checked it on graphapi explorer - it works, when "query" is replaced by simple "q".
Make sure, you are authenticated within your app (Single sign on, or OAuth dialog)
Bundle params = new Bundle();
params.putString("q", query);
mAsyncFacebookRunner.request("fql", params, new FQLRequestListener());
Another EDIT:
While working on different project, I have noticed, that there may be a problem with Facebook Util class's method "read" - sometimes (even if InputStream had data) BufferedReader returned null, and broke loop. I have changed body of the method into:
private static String read(InputStream in) throws IOException {
StringBuffer sb = new StringBuffer();
final InputStreamReader rd = new InputStreamReader(in);
try {
final char[] buffer = new char[1024];
int read;
while((read = rd.read(buffer)) != -1){
sb.append(buffer, 0, read);
}
} catch (Exception e) {
} finally {
try {
rd.close();
} catch (Exception e2) {
}
}
return sb.toString();
}
and it worked.
I have the exact same problem:
calling the fql (simple fql call, not asyncfqlrunner) I have no problems at all. If I wait a few minutes (2-4) and try again sending an fql request, I get an exception by the facebook android api:
A JSONObject text must begin with '{' at character 0 of
I also have ensured that the fb session is valid und the query is correct. nothing has changed, except the time of excecution.
I tried the approche to rewrite the
private static String read(InputStream in) throws IOException method but the error still occurs.
EDIT
Since even facebook isn't aware of this problem and nobody else has posted anything about this I have figured it out by myself. When using the normal request-method (not the async one), go to the Facebook.java in the FB-API and change the following line under in
the public String request(Bundle parameters):
FROM:
return request(null, parameters, "GET");
TO
return request(null, parameters, "POST");
and everything should work fine!
i am using a single port to receive both files and messages in a messenger app. i wanted to ask how can i differenntiate between the files and the messages? i have researched and found that m going to have to make a protoco, i tried but i cant seem to make one. is there any way to accomplish this?
right now i am receiving the connection like this :
public class ReceiveConnection extends Thread {
Socket clientSocket = null;
public ReceiveConnection(Socket socket)
{
this.clientSocket = socket;
SocketOperator.this.sockets.put(socket.getInetAddress(), socket);
}
#Override
public void run() {
try {
BufferedReader in = new BufferedReader(newInputStreamReader(clientSocket.getInputStream()));
while ((inputLine = in.readLine()) != null) {
if (inputLine.contains("TEXT") == true)
{
Log.i("SocketOP","text");
appManager.messageReceived(inputLine);
}
}
if (inputLine.contains("TEXT") == false)
{
InputStream is=clientSocket.getInputStream();
while(is!= null){
Log.i("SocketOP","filee");
appManager.fileReceived(is);
}
}
i have concatinated the string "TEXT" with my text message so it is coming through. putting an ELSE statement isnt working. how can i add a notifier with the file so that i know when the file is being received?
You should create a message header that contains the following data:
Starts with unique byte sequence (2-4 bytes long) that acts as a message separator. This should be a unique non-text sequence that is not very common ( e.g. not CRLF or other control sequence).
Contains message type byte, so that you know if this is text message or file.
Contains length of the message, which helps while extracting the message.
new to the community. been up all night trying to flesh out the underlying html reading system that's at the core of my app's functionally. I could really use a fresh pair of eyes on this one.
Problem: While trying to return a string to be displayed on my app's home activity, I've run into an issue where I'm almost certain that the data was taken in correctly, cleaned up into XML via "Html Cleaner" (http://htmlcleaner.sourceforge.net/), and pulled through Jaxen (opensource Xpath) the result should display some text. Problem is of course, dispite my efforts I've yet to figure out exactly why it wont. My code follows below.
As a test I'm trying to pull the word "maps" from the http://www.google.com home page which is inside an tag with the hyperlink "http://maps.google.com/maps?hl=en&tab=wl" (which i'm using to uniquely identify the tag):
public class home extends Activity {
TextView text1;
//** Called when the activity is first created. */
#Override
public void onCreate(Bundle savedInstanceState)
{
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
text1 = (TextView)findViewById(R.id.text1);
text1.setText(LoadHTMLFromURL("http://www.google.com"));
}
private String LoadHTMLFromURL(String url)
{
try
{
// Load data from URL
InputStream is = (InputStream) new URL(url).getContent(); //generate
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder stringBuilder = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null)
{
stringBuilder.append(line + "");
}
is.close();
String HTMLout = stringBuilder.toString();
// Clean up HTML input.
//Initialize HTML Cleaner.
HtmlCleaner cleaner = new HtmlCleaner();
// This next line Cleans the html and exports it to a Tagnode named "node"
TagNode node = cleaner.clean(HTMLout);
// This is the xpath parsing info
String SearchTerm = "//a[#href='http://maps.google.com/maps?hl=en&tab=wl']";
Object[] info_nodes = node.evaluateXPath(SearchTerm);
TagNode info_node = (TagNode) info_nodes[0];
String info = info_node.getChildren().iterator().next().toString().trim();
return info;
}
catch (Exception e)
{
System.out.println( "Inside: home.LoadHTMLFromURL()" + "Exc="+e);
return null;
}
}
}
I apologize for the clutter, and lack of neatness in the code, still a mid to low range programer in a "learn as you go" stage of my ability. Any advice is appreciated.
side note: I ran a string containing some hand made simple XML to test if it would read the info, and it worked perfectly but not on xml generated from html webpages.
Ok, I believe the issue was my search term. my xpath term was typed wrong.