I'm somewhat new to network programming and am having some trouble. I am creating a JSON object on an Android device, connecting to a python server via TCP, and sending the JSON string. The connection gets accepted, but I keep losing the end of the string, so
json.loads(json_string)
is failing.
Here is the relevant Android code:
private class Worker implements Runnable
{
#Override
public void run()
{
//create the network socket
try
{
socket = new Socket(address, 4242);
Log.i(TAG, "timeout: " + socket.getSoTimeout());
}
catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
packets = new ArrayList<JSONObject>();
for (jobs.moveToFirst(); jobs.isAfterLast() == false; jobs.moveToNext())
{
String jobName = jobs.getString(jobs.getColumnIndex(JobMetaData.JobTableMetaData.JOB));
Uri.Builder updated = new Uri.Builder();
updated.scheme("content");
updated.authority(JobMetaData.AUTHORITY);
updated.appendPath(jobName);
updated.appendPath("member");
updated.appendPath(JobMetaData.MemberTableMetaData.CHANGED);
updated.appendPath("true");
Cursor changed = getContentResolver().query(updated.build(), null, null, null, null);
Log.d(TAG, "number of members " + changed.getCount());
//create a JSON object out of the editable properties
for (changed.moveToFirst(); changed.isAfterLast() == false; changed.moveToNext())
{
JSONObject json = new JSONObject();
for (String att : changed.getColumnNames())
{
if (ListMetaData.validAtts.contains(att))
{
try
{
json.put(att, changed.getString(changed.getColumnIndex(att)));
}
catch (JSONException e)
{
// TODO Auto-generated catch block
Log.d(TAG, "JSON exception in DatagramService");
e.printStackTrace();
}
}
}
//include the GUID and job name
//for identification
try
{
json.put(JobMetaData.MemberTableMetaData.GUID,
changed.getString(changed.getColumnIndex(JobMetaData.MemberTableMetaData.GUID)));
json.put(JobMetaData.JobTableMetaData.JOB, jobName);
}
catch (JSONException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
packets.add(json);
}
changed.close();
}
Log.d(TAG, "entering send loop");
try
{
out = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
out.flush();
}
catch (IOException e1)
{
// TODO Auto-generated catch block
e1.printStackTrace();
}
for (JSONObject packet : packets)
{
Log.d(TAG, "supposedly sending");
try
{
//now write the data
Log.d(TAG, "packet string: " + packet.toString());
out.write(packet.toString());
out.flush();
}
catch (IOException e)
{
}
}
try
{
out.write("Done");
out.flush();
out.close();
} catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
stopSelf();
}
And the test server that I am using (written in python):
#!/usr/bin/env python
import SocketServer
import json
class MemberUpdateHandler(SocketServer.BaseRequestHandler):
def setup(self):
print self.client_address, "connected"
def handle(self):
while True:
self.JSONString = self.request.recv(8192).strip()
if self.JSONString == "Done":
return
self.handleJSON()
self.update()
def handleJSON(self):
JSONMember = json.loads(self.JSONString)
print "GUID:", JSONMember['ManufacturingGUID']
print "Weight:", JSONMember['Weight']
def update(self):
print "do something here"
if __name__ == "__main__":
ADDRESS = ''
PORT = 4242
HOST = (ADDRESS, PORT)
s = SocketServer.ThreadingTCPServer(HOST, MemberUpdateHandler)
s.serve_forever()
Here is the string that is being sent (it is long):
{"DetailCheckedBy":"","SketchRight":"","DetailLength":"142.75","DetailedDate":"**NOT SET**","EngineerVerifiedBothConns":"False","HoldStatus":"Not held","RevisionLevel":"No Revision","MemberNumber":"28","RequestVerifySectionSize":"False","TieForcesRight":"False","InputBy":"","IFCFinishDate_4":"**NOT SET**","IFCFinishDate_5":"**NOT SET**","Weight":"438.408","IFCTaskUID_1":"","IFCFinishDate_1":"**NOT SET**","ErectorOrder":"","IFCFinishDate_2":"**NOT SET**","IFCFinishDate_3":"**NOT SET**","IFCTaskUID_4":"","IFCTaskUID_5":"","IFCTaskUID_2":"","SketchLeft":"","IFCTaskUID_3":"","ErectorSequences":"","ReasonRejected":"","MemberCategory":"","EngineerVerifiedLeftConn":"False","BarcodeId":"","ManufacturingGUID":"42bbf9cc-52da-4712-a5fc-e37c5a544c14","aess":"False","FabricationComplete":"**NOT SET**","UserComment2":"","UserComment3":"","LoadNumber":"","UserComment1":"","ErectionBolted":"**NOT SET**","RequestVerifyLength":"False","RequestVerifyGrade":"False","Painted":"False","HeatCertNumber":"","Route1Description":"","IsExisting":"No","ReceivedFromApproval":"**NOT SET**","BackCheckedBy":"","BatchNumber":"","CostCodeReference":"","PONumber":"","Piecemark":"B_25","ReleasedForFabrication":"**NOT SET**","MemberDescription":"BEAM","EngineerVerifiedMemberReady":"False","IFCTaskName_2":"","IFCTaskName_1":"","IFCTaskName_4":"","RequestVerifyMemberPosition":"False","IFCTaskName_3":"","Erected":"**NOT SET**","RevisionCheckedBy_3":"","IFCTaskName_5":"","RevisionCheckedBy_2":"","RevisionCheckedBy_1":"","EngineerVerifiedLeftComments":"","RequestVerifyLeftConnMaterial":"False","RequestEngineerVerify":"False","RevisionCheckedDate_3":"**NOT SET**","RevisionCheckedDate_2":"**NOT SET**","RevisionCheckedDate_1":"**NOT SET**","EngineerVerifiedLength":"False","BackCheckedDate":"**NOT SET**","SubmittedForApproval":"**NOT SET**","EngineerVerifiedSpecial":"False","CostCodeDescription":"","IFCStartDate_5":"**NOT SET**","TieForcesLeft":"False","Fireproofed":"False","ErectorAvailable":"False","RequestVerifyRightConnMaterial":"False","DetailCheckedDate":"**NOT SET**","ErectorNonSteelSupported":"False","BeamPent":"False","StockStatus":"","Sequence":"1","RequestVerifyLeftLoad":"False","DetailFinalCheckDate":"**NOT SET**","ErectorMemberPlaced":"**NOT SET**","InstanceStatus":"","EngineerVerifiedRightConn":"False","DateReceived":"**NOT SET**","MemberType":"Beam","ModelCheckDate":"**NOT SET**","ReasonForHold":"","EngineerVerifiedRightComments":"","ReceivedOnJobSite":"**NOT SET**","RequestVerifyRightLoad":"False","CostCodePrice":"0.0","NestStatus":"","DateDue":"**NOT SET**","ShopSequence":"","EngineerVerifiedSectionSize":"False","ActualLength":"144","InputDate":"**NOT SET**","ErectorCity":"Unknown","EngineerVerifiedSpecial_comments":"","Route4Description":"","EngineerVerifiedGrade":"False","RightLocation":"0.0xx144.0xx156.0xx","IFCFinishTime_2":"","IFCFinishTime_1":"","IFCFinishTime_4":"","Route3Description":"","IFCFinishTime_3":"","LoadStatus":"","ErectorLongitude":"","DateModelCompleted":"61299957600000","Grade":"##SEKRIT KODE!!##","IFCFinishTime_5":"","Route2Description":"","RequestVerifyCamber":"False","ProjectedFabricationComplete":"**NOT SET**","DetailedBy":"","DetailFinalCheckBy":"","Description":"W8x35","ProjectedShippedDate":"**NOT SET**","NestName":"","IFCStartDate_2":"**NOT SET**","IFCStartTime_1":"","IFCStartDate_1":"**NOT SET**","IFCStartDate_4":"**NOT SET**","IFCStartDate_3":"**NOT SET**","IFCStartTime_5":"","IFCStartTime_4":"","IFCStartTime_3":"","DateHeld":"**NOT SET**","IFCStartTime_2":"","LeftLocation":"0.0xx0.0xx156.0xx","Job":"Mobile_x_x_x_x_Demo_x_x_x_x_IN_x_x_x_x_2011","SpecialCutWeld":"False","RejectedBy":"","ErectionWelded":"**NOT SET**","RequestVerifyRightConnConfig":"False","Vendor":"","PackageNumber":"","RejectedByErector":"**NOT SET**","ModelCheckedBy":"","ApprovalStatus":"Not reviewed","RequestVerifyLeftConnConfig":"False","ErectorLatitude":"","LotName":"","ActualShipDate":"**NOT SET**","NestId":""}
This is the error I get from the python server:
ValueError: Unterminated string starting at: line 1 column 1435 (char 1435)
which means that the string has been truncated to:
{"DetailCheckedBy":"","SketchRight":"","DetailLength":"142.75","DetailedDate":"**NOT SET**","EngineerVerifiedBothConns":"False","HoldStatus":"Not held","RevisionLevel":"No Revision","MemberNumber":"28","RequestVerifySectionSize":"False","TieForcesRight":"False","InputBy":"","IFCFinishDate_4":"**NOT SET**","IFCFinishDate_5":"**NOT SET**","Weight":"438.408","IFCTaskUID_1":"","IFCFinishDate_1":"**NOT SET**","ErectorOrder":"","IFCFinishDate_2":"**NOT SET**","IFCFinishDate_3":"**NOT SET**","IFCTaskUID_4":"","IFCTaskUID_5":"","IFCTaskUID_2":"","SketchLeft":"","IFCTaskUID_3":"","ErectorSequences":"","ReasonRejected":"","MemberCategory":"","EngineerVerifiedLeftConn":"False","BarcodeId":"","ManufacturingGUID":"42bbf9cc-52da-4712-a5fc-e37c5a544c14","aess":"False","FabricationComplete":"**NOT SET**","UserComment2":"","UserComment3":"","LoadNumber":"","UserComment1":"","ErectionBolted":"**NOT SET**","RequestVerifyLength":"False","RequestVerifyGrade":"False","Painted":"False","HeatCertNumber":"","Route1Description":"","IsExisting":"No","ReceivedFromApproval":"**NOT SET**","BackCheckedBy":"","BatchNumber":"","CostCodeReference":"","PONumber":"","Piecemark":"B_25","ReleasedForFabrication":"**NOT SET**","MemberDescription":"BEAM","EngineerVerifiedMemberReady":"False","IFCTaskName_2":"","IFCTaskName_1":"","IFCTaskName_4":"","RequestVerifyMemberPosition":"False","IFCTaskName_3":"","Erected":"**NOT SET**","RevisionCheckedBy_3":"","IFCTaskName_
Any help would be greatly appreciated. Thanks in advance.
UPDATE:
I have updated the code to reflect my tinkering. The string received by the server is now
{"DetailCheckedBy":"","SketchRight":"","DetailLength":"142.75","DetailedDate":"**NOT SET**","EngineerVerifiedBothConns":"False","HoldStatus":"Not held","RevisionLevel":"No Revision","MemberNumber":"28","RequestVerifySectionSize":"False","TieForcesRight":"False","InputBy":"","IFCFinishDate_4":"**NOT SET**","IFCFinishDate_5":"**NOT SET**","Weight":"438.408","IFCTaskUID_1":"","IFCFinishDate_1":"**NOT SET**","ErectorOrder":"","IFCFinishDate_2":"**NOT SET**","IFCFinishDate_3":"**NOT SET**","IFCTaskUID_4":"","IFCTaskUID_5":"","IFCTaskUID_2":"","SketchLeft":"","IFCTaskUID_3":"","ErectorSequences":"","ReasonRejected":"","MemberCategory":"","EngineerVerifiedLeftConn":"False","BarcodeId":"","ManufacturingGUID":"42bbf9cc-52da-4712-a5fc-e37c5a544c14","aess":"False","FabricationComplete":"**NOT SET**","UserComment2":"","UserComment3":"","LoadNumber":"","UserComment1":"","ErectionBolted":"**NOT SET**","RequestVerifyLength":"False","RequestVerifyGrade":"False","Painted":"False","HeatCertNumber":"","Route1Description":"","IsExisting":"No","ReceivedFromApproval":"**NOT SET**","BackCheckedBy":"","BatchNumber":"","CostCodeReference":"","PONumber":"","Piecemark":"B_25","ReleasedForFabrication":"**NOT SET**","MemberDescription":"BEAM","EngineerVerifiedMemberReady":"False","IFCTaskName_2":"","IFCTaskName_1":"","IFCTaskName_4":"","RequestVerifyMemberPosition":"False","IFCTaskName_3":"","Erected":"**NOT SET**","RevisionCheckedBy_3":"","IFCTaskName_5":"","RevisionCheckedBy_2":"","RevisionCheckedBy_1":"","EngineerVerifiedLeftComments":"","RequestVerifyLeftConnMaterial":"False","RequestEngineerVerify":"False","RevisionCheckedDate_3":"**NOT SET**","RevisionCheckedDate_2":"**NOT SET**","RevisionCheckedDate_1":"**NOT SET**","EngineerVerifiedLength":"False","BackCheckedDate":"**NOT SET**","SubmittedForApproval":"**NOT SET**","EngineerVerifiedSpecial":"False","CostCodeDescription":"","IFCStartDate_5":"**NOT SET**","TieForcesLeft":"False","Fireproofed":"False","ErectorAvailable":"False","RequestVerifyRightConnMaterial":"False","DetailCheckedDate":"**NOT SET**","ErectorNonSteelSupported":"False","BeamPent":"False","StockStatus":"","Sequence":"1","RequestVerifyLeftLoad":"False","DetailFinalCheckDate":"**NOT SET**","ErectorMemberPlaced":"**NOT SET**","InstanceStatus":"","EngineerVerifiedRightConn":"False","DateReceived":"**NOT SET**","MemberType":"Beam","ModelCheckDate":"**NOT SET**","ReasonForHold":"","EngineerVerifiedRightComments":"","ReceivedOnJobSite":"**NOT SET**","RequestVerifyRightLoad":"False","CostCodePrice":"0.0","NestStatus":"","DateDue":"**NOT SET**","ShopSequence":"","EngineerVerifiedSectionSize":"False","ActualLength":"144","InputDate":"**NOT SET**","ErectorCity":"Unknown","EngineerVerifiedSpecial_comments":"","Route4Description":"","EngineerVerifiedGrade":"False","RightLocation":"0.0xx144.0xx156.0xx","IFCFinishTime_2":"","IFCFinishTime_1":"","IFCFinishTime_4":"","Route3Description":"","IFCFinishTime_3":"","LoadStatus":"","ErectorLongitude":"","DateModelCompleted":"61299957600000","Grade":"##SEKRIT KODE!!##","IFCFinishTime_5":"","Route2Description":"","RequestVerifyCamber":"False","ProjectedFabricationComplete":"**NOT SET**","DetailedBy":"","DetailFinalCheckBy":"","Description":"W8x35","ProjectedShippedDate":"**NOT SET**","NestName":"","IFCStartDate_2":"**NOT SET**","IFCStartTime_1":"","IFCStartDate_1":"**NOT SET**","IFCStartDate_4":"**NOT SET**","IFCStartDate_3":"**NOT SET**","IFCStartTime_5":"","IFCStartTime_4":"","IFCStartTime_3":"","DateHeld":"**NOT SET**","IFCStartTime_2":"","LeftLocation":"0.0xx0.0xx156.0xx","Job":"Mobile_x_x_x_x_Demo_x_x_x_x_IN_x_x_x_x_2011","SpecialCutWeld":"False","RejectedBy":"","ErectionWelded":"**NOT SET**","RequestVerifyRightConnConfig":"False","Vendor":"","PackageNumber":"","RejectedByErector":"**NOT SET**","ModelCheckedBy":"","ApprovalStatus":"Not reviewed","RequestVerifyLeftConnConfig":"False","ErectorLatitude":"","LotName":"","ActualShipDate":"**NOT SET**","NestId":""}Done
followed by a mess of whitespace. Enough that gedit has trouble loading it all. One step forward two steps back. :/
sizeof(int) may not be the same on both devices. So you probably should hardcode something if you want to pass a binary integer.
If you evaluate int('\001\002\003\004\005\006\007\010'), you're unlikely to get what you want, and I believe that's close to what you're doing in your Python code. For one thing, the endianness of the two devices might be different, and for another, int() wants to evaluate ASCII or some other encoding, not raw endian-dependent integers.
On the python side, you might find this useful:
http://stromberg.dnsalias.org/~strombrg/bufsock.html
I'm not sure about the out.write() you're using, but at the lower level of send(), there's no guarantee that your entire buffer will be written in a single send() - it's allowed to stop early and just return how much was sent. Hopefully, java protects you from that detail the way bufsock does for python.
Why are you "assuming" that the string has been truncated? print it and see what it actually is.
Also, the string that is being sent (as you posted it) is not enclosed in {}, which means it is not proper JSON... I tried to copy/paste it in the interpreter, this raises a ValueError:
ValueError: Extra data: line 1 column 17 - line 1 column 3952 (char 17 - 3952)
I enclosed it in {} and it worked. You should try to see what the string you are receiving actually is on the python side, and then you can really see what's happening. I assume also, that since you are seeing the "Done" sent, then the content should have been sent completely.