I have this file which contains python pickle data stream. I've to read contents of this file in Android.
For example, if I wanted to read this data stream in python, I'd just use the following code
queue = pickle.load(open('filename', 'rb'))
I want to achieve same thing in Android such that I can read this pickle stream data and store it in some kind of collection.
How can I achieve this?
UPDATE: This only works for pickle protocols 2 and 3.
I think the Unpickler class from Pyrolite (MIT license) may be of particular interest to you. It is technically Java, but Android is basically Java. To unpickle you would do something similar to the following:
InputStream stream = new FileInputStream("filename");
Unpickler unpickler = new Unpickler();
Object data = unpickler.load(stream);
// And cast *data* to the appropriate type.
With the imports:
import java.io.FileInputStream;
import java.io.InputStream;
import net.razorvine.pickle.Unpickler;
These are the objects supported by default:
PYTHON ----> JAVA
------ ----
None null
bool boolean
int int
long long or BigInteger (depending on size)
string String
unicode String
complex net.razorvine.pickle.objects.ComplexNumber
datetime.date java.util.Calendar
datetime.datetime java.util.Calendar
datetime.time java.util.Calendar
datetime.timedelta net.razorvine.pickle.objects.TimeDelta
float double (float isn't used)
array.array array of appropriate primitive type (char, int, short, long, float, double)
list java.util.List<Object>
tuple Object[]
set java.util.Set
dict java.util.Map
bytes byte[]
bytearray byte[]
decimal BigDecimal
custom class Map<String, Object> (dict with class attributes including its name in "__class__")
Please also note:
The unpickler simply returns an Object. Because Java is a statically typed
language you will have to cast that to the appropriate type. Refer to this
table to see what you can expect to receive.
UPDATE: I ran tests using the various pickle protocols (0-3) and found that it fails for 0 and 1, but succeeds for 2 and 3.
Here's the python code used to generate the pickled data:
import pickle
class Data(object):
def __init__(self):
self.x = 12
data = Data()
for p in [0, 1, 2]:
with open('data.{}'.format(p), 'wb') as fh:
pickle.dump(data, fh, protocol=p)
# Python 3 only.
with open('data.3', 'wb') as fh:
pickle.dump(data, fh, protocol=3)
And the java code to unpickle it:
import java.io.FileInputStream;
import java.io.InputStream;
import java.io.IOException;
import java.util.Map;
import net.razorvine.pickle.Unpickler;
public class Test {
public static void main(String[] args) throws IOException {
String filename = args[0];
InputStream inputStream = new FileInputStream(filename);
Unpickler unpickler = new Unpickler();
Map<String, Object> data = (Map<String, Object>)unpickler.load(inputStream);
}
}
When run with data.0 and data.1, it fails with:
Exception in thread "main" net.razorvine.pickle.PickleException: expected zero arguments for construction of ClassDict (for copy_reg._reconstructor)
at net.razorvine.pickle.objects.ClassDictConstructor.construct(ClassDictConstructor.java:23)
at net.razorvine.pickle.Unpickler.load_reduce(Unpickler.java:617)
at net.razorvine.pickle.Unpickler.dispatch(Unpickler.java:170)
at net.razorvine.pickle.Unpickler.load(Unpickler.java:84)
at Test.main(Test.java:13)
When run with data.2 and data.3, it succeeds.
I am developing an Android App which has to extract data from a website and the extracted data will be displayed in a text view in the application
After having tried all the possible ways that i found in the googling and Stackoverflow i am still unable to process the data and now can any one share if they have done ..
Details
Website: https://www.amrita.edu/campus/bengaluru
In this website i was looking to extract the data of Latest News block and Upcoming Events
Here's the code : I have used JSOUP to extract
package out.in;
import java.io.IOException;
import org.jsoup.Jsoup;
import org.jsoup.select.Elements;
import org.w3c.dom.Document;
import android.app.Activity;
import android.os.Bundle;
import android.sax.Element;
import android.widget.TextView;
import android.widget.Toast;
public class HtmlExtracterActivity extends Activity {
/** Called when the activity is first created. */
// url
static final String URL = "https://www.amrita.edu/campus/bengaluru";
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
try {
((TextView)findViewById(R.id.tv)).setText(getdata());
}
catch (Exception ex) {
((TextView)findViewById(R.id.tv)).setText("Error");
}
}
protected String getdata() throws Exception {
String result = "";
// get html document structure
Document document = (Document) Jsoup.connect(URL).get();
// selector query
*********Need help
// check results
*********Need help
return result;
}
}
I have given the Internet Permission in the Manifest file
and
Xml file is as follows
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:orientation="vertical"
android:layout_width="fill_parent"
android:layout_height="fill_parent"
>
<TextView android:text=" "
android:id="#+id/tv" android:layout_width="wrap_content"
android:layout_height="wrap_content"></TextView>
</LinearLayout>
I would sincrely Appreciate the needed Help in advance
You've not mentioned the exact problem you are facing. Did you try to see what is being returned at this:
Document document = (Document) Jsoup.connect(URL).get();
I am assuming that this might be because of missing User-Agent in the above mentioned code. Please try this and let us know if you still face the error:
Response response= Jsoup.connect(location)
.ignoreContentType(true)
.userAgent("Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:25.0) Gecko/20100101 Firefox/25.0")
.referrer("http://www.google.com")
.timeout(12000)
.followRedirects(true)
.execute();
Document doc = response.parse(); User Agent
Use the latest User agent. Here's the complete list
http://www.useragentstring.com/pages/Firefox/.
Timeout
Also don't forget to add timout, since sometimes it takes more than
normal timeout to download the page.
Referer
Set the referer as google.
Follow redirects
follow redirects to get to the page.
execute() instead of get()
Use execute() to get the Response object. Which can help you to check
for content type and status codes incase of error.
Source: https://stackoverflow.com/a/20284953/1262177
I'm working on an app that simply has to go out to UrbanDictionary and return the results of a word search in JSON format...then display it in an Android project.
Here is my code in its entirety:
package org.twodee.mitchemc.webapi;
import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.ArrayList;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
import android.app.ListActivity;
import android.os.AsyncTask;
import android.os.Bundle;
import android.util.Log;
import android.view.View;
import android.view.View.OnClickListener;
import android.widget.ArrayAdapter;
import android.widget.Button;
import android.widget.EditText;
public class MainActivity extends ListActivity {
private ArrayList<String> definitionsList;
private ArrayAdapter<String> adapter;
private EditText wordText;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
wordText = (EditText) findViewById(R.id.wordText);
Button defineButton = (Button) findViewById(R.id.defineButton);
definitionsList = new ArrayList<String>();
adapter = new ArrayAdapter<String>(this, android.R.layout.simple_list_item_1, definitionsList);
setListAdapter(adapter);
defineButton.setOnClickListener(new OnClickListener() {
#Override
public void onClick(View v) {
String word = wordText.getText().toString();
new DownloadTask().execute(word);
}
});
}
private ArrayList<String> getDefs(String word) {
ArrayList<String> definitions = new ArrayList<String>();
try {
URL url = new URL("http://api.urbandictionary.com/v0/define?term="
+ word);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
InputStream in = connection.getInputStream();
String json = Utilities.slurp(in);
Log.d("FOO", json);
JSONArray array = new JSONArray(json);
for (int i = 0; i < array.length(); ++i) {
JSONObject object = array.getJSONObject(i);
definitions.add(object.getString("definition"));
}
in.close();
} catch (MalformedURLException e) {
} catch (IOException e) {
e.printStackTrace();
} catch (JSONException e) {
e.printStackTrace();
}
return definitions;
}
class DownloadTask extends AsyncTask<String, Void, ArrayList<String>> {
#Override
protected ArrayList<String> doInBackground(String... params) {
String word = params[0];
return getDefs(word);
}
#Override
protected void onPostExecute(ArrayList<String> defs) {
super.onPostExecute(defs);
definitionsList.clear();
definitionsList.addAll(defs);
adapter.notifyDataSetChanged();
}
}
}
Pay no attention to the Utilities.slurp behind the curtain.
So, if I run this, it doesn't crash, and it actually does give me results, but it throws them in LogCat...not my app. I previously found the following exception:
of type org.json.jsonarray cannot be converted to jsonobject
I looked over this forum and found the notion that I've got objects within an array (or something like that) but I can't figure out how to reflect that in code.
What I'd like to have happen is you type, say, "Hey" into the app, hit "Define" and then there's a list of definitions below it. Only the definitions...so the JSON part of it should be "definition"
Here's some JSON output that I get in LogCat. I hope this is readable:
12-06 19:00:57.647: W/System.err(1400): org.json.JSONException: Value {"total":32,"result_type":"exact","list":[{"thumbs_down":428,"author":"D.B. Echo","definition":"A contraction for \"Hello, I find you attractive and would like to dance with you, share some drinks with you, and then perhaps have sex with you. Nothing serious, of course, and I doubt that this will result in a long term relationship, but I would appeciate you considering my proposal.\"","permalink":"http:\/\/hey.urbanup.com\/1099714","thumbs_up":1530,"word":"hey","current_vote":"","example":"Guy at bar to attractive girl: \"Hey.\"\r\n\r\nAttractive girl to guy: \"Go to hell, creep.\" (Moves to other side of bar.)","defid":1099714},{"thumbs_down":250,"author":"Erica","definition":"hey is a way to say \"hi\" or \"what's up\"\r\nthe type of 'hey' horses eat is hAy you morons!!!","permalink":"http:\/\/hey.urbanup.com\/574637","thumbs_up":541,"word":"Hey","current_vote":"","example":"hey what's up!?!\r\nhorses eat hay!","defid":574637},{"thumbs_down":144,"author":"SoMe RaNdOm PeRsOn","definition":"I word to use to get someone's attention. Even if you don't know the persons name by simply saying \"hey\" you can easily get their attention","permalink":"http:\/\/hey.urbanup.com\/1106278","thumbs_up":294,"word":"hey","current_vote":"","example":"guy 1:Hey Alex what's that guys name over there?\r\nAlex: How the hell should I know?\r\nGuy 1: Hey!!\r\nMystery guy: ::looks around confused::\r\nGuy 1: O hi.","defid":1106278},{"thumbs_down":120,"author":"Jamesrob92","definition":"1. The most informal form of greeting. Usually used in a friendly manner or to sound cool and relaxed.\r\n2. An exclamation used to get someone's attention","permalink":"http:\/\/hey.urbanup.com\/2153666","thumbs_up":249,"word":"hey","current_vote":"","example":"1. Hey, what's up?\r\n2. HEY! Wait for me!\r\n3. HEY YOU! Get away from my wife!","defid":2153666},{"thumbs_down":133,"author":"marshmallow","definition":"a slang term used by many people in place of hello","permalink":"http:\/\/hey.urbanup.com\/1129122","thumbs_up":219,"word":"hey","current_vote":"","example":"Hey Ma...whats crackin?","defid":1129122},{"thumbs_down":79,"author":"hey malasadas","definition":"an expression used to get someone's attention","permalink":"http:\/\/hey.urbanup.com\/198491","thumbs_up":128,"word":"hey","current_vote":"","example":"hey man, what the hell are you doing?!","defid":198491},{"thumbs_down":54,"author":"HappyGirl1993","definition":"Considered to be a lot more flirtatious than 'hello' or 'hi.'","permalink":"http:\/\/hey.urbanup.com\/5302185","thumbs_up":68,"word":"Hey","current_vote":"","example":"Example...\n\nGuy: Hey.\r\nGirl: Hi.\r\nGuy: *Whoa, total FAIL!*","defid":5302185},{"thumbs_down":37,"author":"Eric Klein...","definition":"1. Interjection, an informal greeting. \r\n\r\n2. Verb, to 'hey' someone. The act of driving down public roads in a lane near a sidewalk, coming upon an unsuspecting pedestrian, and hollering \"HEY!\" as loudly as possible in their general direction. Most effective if done with all windows rolled down, and all members of the vehicle participating. (Originating in Sioux Falls, SD)","permalink":"http:\/\/hey.urbanup.com\/2705914","thumbs_up":42,"word":"Hey","current_vote":"","example":"1. \"Hey buddy, how's the syphilis?\"\r\n\r\n2. \"Jimmy and I heyed a kid so bad the other day, he fell off his bike!\"","defid":2705914},{"thumbs_down":80,"author":"J-Lib","definition":"a slang interjection or prompt often used in place of \"eh\" or \"right\" in order to make a sentence into a question; used in parts of Canada","permalink":"http:\/\/hey.urbanup.com\/1262450","thumbs_up":67,"word":"hey","current_vote":"","example":"It's supposed to snow tomorrow, hey?\r\nThat guy was pretty attractive, hey?","defid":1262450},{"thumbs_down":7,"author":"Bam Bam Branson","definition":"Why would you even search this?","permalink":"http:\/\/hey.urbanup.com\/5850424","thumbs_up":8,"word":"Hey","current_vote":"","example":"Hey is Idk","defid":5850424}],"pages":4,"sounds":["http:\/\/media.urbandictiona
Any help would be greatly appreciated! Thanks in advance!
UPDATE: Here's the code that works:
String json = Utilities.slurp(in);
JSONObject jsonObject = new JSONObject(json);
JSONArray array = jsonObject.getJSONArray("list");
for (int i = 0; i < array.length(); ++i) {
JSONObject object = array.getJSONObject(i);
definitions.add(object.getString("definition"));
}
Based on the json you get back from that request, you are setting up your JSONObject wrong. Here is an example of the json that comes back:
{
"has_related_words": true,
"result_type": "exact",
"list": [
{
"defid": 2957653,
"word": "test",
"author": "sm1g",
"permalink": "http://test.urbanup.com/2957653",
"definition": "To check if something coresponds the promised result or what effect does it have at all.",
"example": "By typing in the word \"test\" you prolly tried to search if there was a definition for this word.",
"thumbs_up": 123,
"thumbs_down": 46,
"current_vote": ""
}
],
"sounds": [
"http://media.urbandictionary.com/sound/test-8076.mp3"
],
"total": 20,
"pages": 2
}
In your code, to grab an array like list, you would need to do:
String json = Utilities.slurp(in);
JSONObject jsonObject = new JSONObject(json);
//now get the list array
JSONArray list = new JSONArray(jsonObject.getArray("list"));
The method names may not be exact.
I am new to android, I tried XML parse with SAX previously but I got Exception in Network,I tried lot with that , But not able to get that, many suggest to use AsyncTask, I tried with that , But again failed, So I tried with DOM now, Again same got some exception in android.os.NetworkOnMainThreadException and gralloc_goldfish error (i.e) Emulator without GPU emulation detected .... help me for this problem. i gng this program for 2 days
Program::XMLParsingDOMExample.java
package com.androidpeople.xml.parsing;
import java.net.URL;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.InputSource;
import android.app.Activity;
import android.os.Bundle;
import android.widget.LinearLayout;
import android.widget.TextView;
public class XMLParsingDOMExample extends Activity {
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
/** Create a new layout to display the view */
LinearLayout layout = new LinearLayout(this);
layout.setOrientation(1);
/** Create a new textview array to display the results */
TextView name[];
TextView website[];
TextView category[];
try {
URL url = new URL(
"http://www.androidpeople.com/wp-content/uploads/2010/06/example.xml");
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
DocumentBuilder db = dbf.newDocumentBuilder();
Document doc = db.parse(new InputSource(url.openStream()));
doc.getDocumentElement().normalize();
NodeList nodeList = doc.getElementsByTagName("item");
/** Assign textview array lenght by arraylist size */
name = new TextView[nodeList.getLength()];
website = new TextView[nodeList.getLength()];
category = new TextView[nodeList.getLength()];
for (int i = 0; i < nodeList.getLength(); i++) {
Node node = nodeList.item(i);
name[i] = new TextView(this);
website[i] = new TextView(this);
category[i] = new TextView(this);
Element fstElmnt = (Element) node;
NodeList nameList = fstElmnt.getElementsByTagName("name");
Element nameElement = (Element) nameList.item(0);
nameList = nameElement.getChildNodes();
name[i].setText("Name = "
+ ((Node) nameList.item(0)).getNodeValue());
NodeList websiteList = fstElmnt.getElementsByTagName("website");
Element websiteElement = (Element) websiteList.item(0);
websiteList = websiteElement.getChildNodes();
website[i].setText("Website = "
+ ((Node) websiteList.item(0)).getNodeValue());
category[i].setText("Website Category = "
+ websiteElement.getAttribute("category"));
layout.addView(name[i]);
layout.addView(website[i]);
layout.addView(category[i]);
}
} catch (Exception e) {
System.out.println("XML Pasing Excpetion = " + e);
}
/** Set the layout view to display */
setContentView(layout);
}
}
example.xml
<maintag>
<item>
<name>AndroidPeople</name>
<website category="android">www.androidpeople.com</website>
</item>
<item>
<name>iPhoneAppDeveloper</name>
<website category="iPhone">www.iphone-app-developer.com</website>
</item>
</maintag>
i m not able to get output for the above code.. got some exception in android.os.NetworkOnMainThreadException and gralloc_goldfish error (i.e) Emulator without GPU emulation detected .... help me for this problem. i gng this program for 2 days.. Thank in advance
Execute this on a background thread (maybe using an AsyncTask).
A NetworkOnMainThreadException means exactly this - you are running a network call on the main UI thread. This is to encourage us to not make calls that block UI interaction.
url.openStream() is what triggers this.
im new to android development and im trying to build my first app which looks for a online generated xml file to display information. In the first activity i created a ListView with all the entries from an XML file, as soon as i click on an entry it passes the id and goes to the 2nd activity which should access another XML file with the details. However i keep getting this error when trying to fetch the XML for the details:
java.lang.ClassCastException:
org.apache.harmony.xml.dom.ElementImpl
Any ideas whats wrong? Here is the source for the "details" activity:
package en.android.itleaked.com;
import java.io.InputStream;
import java.net.URL;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import org.w3c.dom.Document;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.InputSource;
import android.app.Activity;
import android.graphics.drawable.Drawable;
import android.os.Bundle;
import android.sax.Element;
import android.widget.ImageView;
import android.widget.TextView;
public class showReleases extends Activity {
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.releasedetails);
getFeed();
}
public void getFeed() {
Bundle extras = getIntent().getExtras();
try {
URL url2 = new URL("http://www.it-leaked.com/app/details.php?id=" + extras.getString("id"));
DocumentBuilderFactory dbf2 = DocumentBuilderFactory.newInstance();
DocumentBuilder db2 = dbf2.newDocumentBuilder();
Document doc2 = db2.parse(new InputSource(url2.openStream()));
doc2.getDocumentElement().normalize();
NodeList nodeList2 = doc2.getElementsByTagName("item");
String relTitle[] = new String[nodeList2.getLength()];
String relCover[] = new String[nodeList2.getLength()];
for (int i = 0; i < nodeList2.getLength(); i++) {
Node node2 = nodeList2.item(i);
Element fstElmnt2 = (Element) node2;
NodeList nameList2 = ((Document) fstElmnt2).getElementsByTagName("title");
Element nameElement2 = (Element) nameList2.item(0);
nameList2 = ((Node) nameElement2).getChildNodes();
relTitle[i] = ((Node) nameList2.item(0)).getNodeValue();
NodeList coverList2 = ((Document) fstElmnt2).getElementsByTagName("cover");
Element coverElement2 = (Element) coverList2.item(0);
coverList2 = ((Node) coverElement2).getChildNodes();
relCover[i] = ((Node) coverList2.item(0)).getNodeValue();
}
TextView txtView = (TextView)findViewById(R.id.TextView01);
txtView.setText(relTitle[0]);
ImageView imgView =(ImageView)findViewById(R.id.ImageView01);
Drawable drawable = LoadImageFromWebOperations(relCover[0]);
imgView.setImageDrawable(drawable);
}
catch (Exception e) {
TextView txtView2 = (TextView)findViewById(R.id.TextView02);
txtView2.setText("Error: " + e);
}
}
private Drawable LoadImageFromWebOperations(String url)
{
try
{
InputStream is = (InputStream) new URL(url).getContent();
Drawable d = Drawable.createFromStream(is, "src name");
return d;
}catch (Exception e) {
System.out.println("Exc="+e);
return null;
}
}
}
Here is the URL for the XML with an attached id so you can see what it looks like:
http://www.it-leaked.com/app/details.php?id=50969
Any ideas whats going on? By the way i added the number 2 to every variable which has something to do with the XML parsing / fetching just to make sure theres no conflict with the other activity, but im still getting the same error..
I hope you can help me out.
Thanks in advance
This question is a little old but I believe the ClassCastException is due to attempting to cast the elements in the nodelist to android.sax.Element type rather than org.w3c.dom.Element; check the imports.
Looking at your exception (which is java.lang.ClassCastException) the problem is in casting some class to another.
In your code i didn't understand the reason casting Element to Document - Element has getElementsByTagName method which you are using. Look here: http://developer.android.com/reference/org/w3c/dom/Element.html
It is the root of all evil. Element and Document both implementing Node interface, but Document isn't implements Element - that's why Element can't be cast to Document.
Anyway, expcetion line number can determine the exact error position.