I'm implementing schema validation using libxml2. The schema I'm validating against imports two other schemas with lines like:
<xs:import namespace="http://www.w3.org/XML/1998/namespace"
schemaLocation="http://www.somewebsite.com/xsd/xml.xsd"/>
All three schema files are located in the same directory on the device.
This works well when the device has internet access, but fails when it does not, as libxml2 still attempts to download the imported schemas from the schemaLocation even though I'm passing in XML_PARSE_NONET.
I tried getting libxml2 to load the files locally by editing the schemaLocation attribute to xml.xsd, ./xml.xsd, and file:///data/data/com.company.appname/files/xml.xsd, but all three resulted in the same libxml2 error:
domain: 16
code: 3069 (XML_SCHEMAP_INTERNAL)
message: Internal error: xmlSchemaParse, An internal error occurred.
I also tried removing the schemaLocation attribute entirely, on the off-chance that libxml2 might search for the imported schemas alongside the original schema, but that resulted in the following error when the schema parser hit a line that referenced the imported entities:
<xs:attribute ref="xml:lang" use="required"/>
domain: 16
code: 3004 (XML_SCHEMAP_SRC_RESOLVE)
message: attribute use (unknown), attribute 'ref': The QName value '{http://www.w3.org/XML/1998/namespace}lang' does not resolve to a(n) attribute declaration.
I also looked into manually merging the three schemas into a single file, but as they use different namespaces, this is not possible.
The standard solution for this seems to be the XML catalog, but I've read through libxml2's catalog documentation, and I can't figure out how (or even whether it's possible) to add mappings that will be used by my app when deployed to a device. I think I might need to implement an xmlExternalEntityLoader, but the documentation for that is quite slim.
How can I get libxml2 to import these schemas without network access? Obviously I'd ideally like a robust solution that works with the unedited schema, but I've be content with something quick-and-dirty that involves editing the schema, like my original attempts described above.
The errors described above are from an Android device (using JNI), but I'm having similar problems on iOS, where the solution will also need to work.
One way to do this is to intercept libxml2's call to open the imported URL with a custom xmlExternalEntityLoader.
The basic code for doing this is as follows:
#include <libxml/xmlIO.h>
#include <libxml/parserinternals.h>
xmlExternalEntityLoader defaultLoader = NULL;
xmlParserInputPtr
xmlMyExternalEntityLoader(const char *URL, const char *ID,
xmlParserCtxtPtr ctxt) {
xmlParserInputPtr ret;
const char *fileID = NULL;
/* lookup for the fileID
* The documentation suggests using the ID, but for me this was
* always NULL so I had to lookup by URL instead.
*/
ret = xmlNewInputFromFile(ctxt, fileID);
if (ret != NULL)
return(ret);
if (defaultLoader != NULL)
ret = defaultLoader(URL, ID, ctxt);
return(ret);
}
int main(..) {
...
/*
* Install our own entity loader
*/
defaultLoader = xmlGetExternalEntityLoader();
xmlSetExternalEntityLoader(xmlMyExternalEntityLoader);
...
}
(Slightly adjusted from the sample code in The entities loader section of libxml2's I/O Interfaces documentation.)
Related
I am doing a demo project on graphql queries. Followed all the required steps and downloaded schema.json from my dummy graphql server on graphcool. However when I am writing the .graphql file which contains the graphql queries, android studio is showing me errors for the fields. Even after I have successfully built the project, still the errors persist. I have a dummy graphql server which has type Employee and 5 fields : name,id,height,age and work.
I have tried making a new project and re writing all the code. I have tried rebuilding and cleaning my project, etc,. I have re downloaded the schema.json file and tried deleting and rewriting the .graphql file. Yet the errors are still thrown.
query getAllEmployeeDetails{
allEmployees{
name
id
height
age
work
}
}
this is my .graphql file. Android studio throws errors on all the 5 fields and also throws error on the "allEmployees" too.
The corresponding query on graphql backend is:
query{
allEmployees{
name
id
height
age
work
}
}
which works fine.
Expected result is that android studio shouldn't be throwing an error on the fields.
I have faced the same error you have mentioned in the comment, the whole queries show the same error every field couldn't be resolved but in development the code works fine.
So I searched many times until I found this issue
After opening the .graphqlconfig file there is a button to run a query called "introspection query" here like this snapshot here
A new file will be added automatically to your graphql folder called "schema.graphql", It is a file contains the whole inputs, types & scalars by the end-point, after generating the file, everything will be normal again and the errors will be vanished.
Add a file named .graphqlconfig to the root of your project. The file's content should be something like this:
{
"name": "Remote Schema",
"schemaPath": "remote-schema.graphql",
"extensions": {
"endpoints": {
"Apollo Fullstack Tutorial": {
"url": "https://apollo-fullstack-tutorial.herokuapp.com/",
"headers": {
"user-agent": "JS GraphQL"
},
"introspect": true
}
}
}
}
Change the url to your server's endpoint and change "Remote SWAPI GraphQL Endpoint" to better describe your server.
Also, see here
I had this same issue...
Zizoh and Mahmoud Magdy explained it but ill leave it here with more details to help anyone who needs..
In android studio, in root folder create a .graphqlconfig like this :
change that URL for your server url
]2
after that, click on the play icon in the config file you just created..
]3
And now everything should be working as you would like :)
When using the URIs
String myUri = "https://evil.example.com\\.good.example.org/";
// or
String myUri = "https://evil.example.com\\#good.example.org/";
in Java on Android, the backslash in the host or user information of the authority part of the URI causes a mismatch between how Android’s android.net.Uri and android.webkit.WebView parse the URI with regard to its host.
The Uri class (and cURL) treat evil.example.com\.good.example.org (first example) or even good.example.org (second example) as the URI’s host.
The WebView class (and Firefox and Chrome) treat evil.example.com (both examples) as the URI’s host.
Is this known, expected or correct behavior? Do the two classes simply follow different standards?
Looking at the specification, it seems neither RFC 2396 nor RFC 3986 allows for a backslash in the user information or authority.
Is there any workaround to ensure a consistent behavior here, especially for validation purposes? Does the following patch look reasonable (to be used with WebView and for general correctness)?
Uri myParsedUri = Uri.parse(myUri);
if ((myParsedUri.getHost() == null || !myParsedUri.getHost().contains("\\")) && (myParsedUri.getUserInfo() == null || !myParsedUri.getUserInfo().contains("\\"))) {
// valid URI
}
else {
// invalid URI
}
One possible flaw is that this workaround may not catch all the cases that cause inconsistent hosts to be parsed. Do you know of anything else (apart from a backslash) that causes a mismatch between the two classes?
It's known that Android WebView 4.4 converts some URLs, in the linked issue are some steps described how to prevent that. From your question is not completely clear if your need is based in that issue or something else.
You can mask the backslashes and other signs with there according number in the character-table. In URLs the the number is written in hexademcimal.
Hexadecimal: 5C
Dezimal: 92
Sign: \
The code is the prepended with a % for each sign in the URL, your code looks like this after replacement:
String myUri = "https://evil.example.com%5C%5C.good.example.org/";
// or
String myUri = "https://evil.example.com%5C%5C#good.example.org/";
it might be required still to add a slash to separate domain and path:
String myUri = "https://evil.example.com/%5C%5C.good.example.org/";
// or
String myUri = "https://evil.example.com/%5C%5C#good.example.org/";
Is it possible that the backslashes never shall be used for network-communication at all but serve as escaping for some procedures like regular expressions or for output in JavaScript (Json) or some other steps?
Bonus ;-)
Below is a php-script that prints a table for most UTF-8-signs with the corresponding Numbers in hex and dec. (it still should be wrapped in an html-template including css perhaps):
<?php
$chs = array('0','1','2','3','4','5','6','7','8','9','A','B','C','D','E','F');
$chs2 = $chs;
$chs3 = $chs;
$chs4 = $chs;
foreach ($chs as $ch){
foreach ($chs2 as $ch2){
foreach ($chs3 as $ch3){
foreach ($chs4 as $ch4){
echo '<tr>';
echo '<td>';
echo $ch.$ch2.$ch3.$ch4;
echo '</td>';
echo '<td>';
echo hexdec($ch.$ch2.$ch3.$ch4);
echo '</td>';
echo '<td>';
echo '&#x'.$ch.$ch2.$ch3.$ch4.';';
echo '</td>';
echo '</tr>';
}
}
}
}
?>
Is this known, expected or correct behavior?
IMO, it is not. For both URI and WebView. Because RFC won't allow a backslash, they could have warn it. However it is less important because it does not affect the working at all if the input is as expected.
Do the two classes simply follow different standards?
The URI class and WebView strictly follows the same standards. But due to the fact that they are different implementations, they may behave differently to an unexpected input.
For example, "^(([^:/?#]+):)?((//([^/?#]*))?([^?#]*)(\\?([^#]*))?)?(#(.*))?" this is the regular expression in URI which is used to parse URIs. The URI parsing of WebView is done by native CPP methods. Even though they follow same standards, chances are there for them to give different outcome (At least for unexpected inputs).
Does the following patch look reasonable?
Not really (See the answer of next question).
Do you know of anything else (apart from a backslash) that causes a
mismatch between the two classes?
Because you are so concerned about the consistent behavior, I won't suggest a manual validation. Even the programmers who wrote these classes can't list all of such scenarios.
The solution
If I understand correctly, you need to load URLs which is supplied by untrustable external sources (which attackers can exploit if there is a loop hole), but you need to identify it's host correctly.
In that case, you can parse it using URI class itself and use URI#getHost() to identify the host. But for WebView, instead of passing the original URL string, pass URI#toString().
I have been trying a ( i hope) simple bit of Android hyperloop code directly within a titanium project (using SDK 7.0.1.GA and hyperloop 3).
var sysProp = require('android.os.SystemProperties');
var serialNumber = sysProp.get("sys.serialnumber", "none");
But when the app is run it reports
Requested module not found:android.os.SystemProperties
I think this maybe due to the fact that when compiling the app (using the cli) it reports
hyperloop:generateSources: Skipping Hyperloop wrapper generation, no usage found ...
I have similar code in a jar and if I use this then it does work, so I am wondering why the hyperloop generation is not being triggered, as I assume that is the issue.
Sorry should have explained better.
This is the jar source that I use, the extraction of the serial number was just an example (I need access to other info manufacturer specific data as well), I wanted to see if I could replicate the JAR functionality using just hyperloop rather that including the JAR file. Guess if it's not broke don't fix it, but was curious to see if it could be done.
So with the feedback from #miga and a bit of trial and error, I have come up with a solution that works really well and will do the method reflection that is required. My new Hyperloop function is
function getData(data){
var result = false;
var Class = require("java.lang.Class");
var String = require("java.lang.String");
var c = Class.forName("android.os.SystemProperties");
var get = c.getMethod("get", String.class, String.class);
result = get.invoke(c, data, "Error");
return result;
}
Where data is a string of the system property I want.
I am using it to extract and match a serial number from a Samsung device that is a System Property call "ril.serialnumber" or "sys.serialnumber". Now I can use the above function to do what I was using the JAR file for. Just thought I'd share in case anyone else needed something similar.
It is because android.os.SystemProperties is not class you can import. Check the android documentation at https://developer.android.com/reference/android/os/package-summary.html
You could use
var build = require('android.os.Build');
console.log(build.SERIAL);
to access the serial number.
I'm having a confusing problem. I'm trying to make a Web cleint that uses WSDL.
I'm using C++ RAD Studio 10 Seattle, but the same problem occured in RAD Studio XE8(older version).
1.I create a Multi-Device Application, add one Edit component and one Button.
2.I create a WSDL Importer by changing the location of the WSDL file to : "http://www.w3schools.com/webservices/tempconvert.asmx?WSDL" and leave all other setting to default.
3.On ButtonClick event of the button I write two lines of code :
_di_TempConvertSoap Converter = GetTempConvertSoap(true,
"http://www.w3schools.com/webservices/tempconvert.asmx?WSDL");
Edit1->Text = Converter->CelsiusToFahrenheit("32");
So after these three steps I have one unit, which is the main Unit with the Form and with the button event. And one file "tempconvert.cpp" that the WSDL Importer has generated. It quite actually just translates the WSDL code to a C++ one and defines the method to communicate with the server. In my case I have two methods : FahrenheitToCelsius() and CelsiusToFahrenheit(), in the example I use CelsiusToFahrenheit().
I compile it to 32-bit Windows platform, run it and when I click the button, the result "89.6" appears in the text of the Edit component. So this is working as expected.
But when I change the target platform to "Android" and use my mobile phone "Samsung GT-I8262" with Android 4.1.2 and run the project, it just stops and exits. I debugged the problem and it stops at the first command in "tempconvert.cpp" in RegTypes() method.
// ************************************************************************
//
// This routine registers the interfaces and types exposed by the WebService.
// ************************************************************************ //
static void RegTypes()
{
/* TempConvertSoap */
InvRegistry()->RegisterInterface(__delphirtti(TempConvertSoap), L"http://www.w3schools.com/webservices/", L"utf-8");
InvRegistry()->RegisterDefaultSOAPAction(__delphirtti(TempConvertSoap), L"http://www.w3schools.com/webservices/%operationName%");
InvRegistry()->RegisterInvokeOptions(__delphirtti(TempConvertSoap), ioDocument);
/* TempConvertSoap.FahrenheitToCelsius */
InvRegistry()->RegisterMethodInfo(__delphirtti(TempConvertSoap), "FahrenheitToCelsius", "",
"[ReturnName='FahrenheitToCelsiusResult']", IS_OPTN);
/* TempConvertSoap.CelsiusToFahrenheit */
InvRegistry()->RegisterMethodInfo(__delphirtti(TempConvertSoap), "CelsiusToFahrenheit", "",
"[ReturnName='CelsiusToFahrenheitResult']", IS_OPTN);
/* TempConvertHttpPost */
InvRegistry()->RegisterInterface(__delphirtti(TempConvertHttpPost), L"http://www.w3schools.com/webservices/", L"utf-8");
InvRegistry()->RegisterDefaultSOAPAction(__delphirtti(TempConvertHttpPost), L"");
}
#pragma startup RegTypes 32
Does someone have any idea why this might be happening? I tried on two other Samsung phones and it didn't work. The error that shuts the program down is "Segmentation fault(11)", and more precisely it stops at the following line of code in "System.pas" file :
u_strFromUTF8(PUChar(Dest), MaxDestChars, DestLen, MarshaledAString(Source), SourceBytes, ErrorConv);
Here is some info that I've found about the function:
u_strFromUTF8 - function that converts a UTF-8 string to UTF-16.
UCHAR is a Byte(in Delphi), so PUCHAR is a pointer to Byte.
I cannot se what could possibly go wrong with this function which apparently only converts a string.
So my question is why does the project work on Windows 32 bit version, but on Android it throws Segmentation fault(11)?
I hope I could find a solution for this problem. I will keep looking.
Thank you,
Zdravko Donev :)
UPDATE:
I disassembled the line:
InvRegistry()->RegisterInterface(__delphirtti(TempConvertSoap), L"http://www.w3schools.com/webservices/", L"utf-16");
to get :
TInvokableClassRegistry *Class = InvRegistry();
TTypeInfo *Info = __delphirtti(TempConvertSoap);
UnicodeString Namespace = "http://www.w3schools.com/webservices/";
UnicodeString WSDLEncoding = "utf-8";
Class->RegisterInterface(Info, Namespace, WSDLEncoding);
And I saw that the problem occurs when calling InvRegistry() function, but I still haven't found the problem as I cannot reach the source code of the function.
I found a solution.
I deleted the line
#pragma startup RegTypes 32
and called the method RegTypes() on my own when I create the form and it worked.
I want to access a android device from python to download some photos.
libmtp works from the CLI.
Than pymtp. It's been around for a while but it's designed for python 2 and i'm using python 3. Meanwhile fixed several minor issues but i'm stuck at an error from function get_filelisting
specially this section:
ret = []
next = files
while next:
ret.append(next.contents)
if (next(next.contents) is None):
break
next = next(next.contents)
The error is related to the "next".
That section looks strange to me, i've been coding in python for a while but i'm new to ctypes. Tried a lot of variants, they all failed. The "next" could be confusing with python buildin function so i renamed it to nextpointer and came to this code:
ret = []
nextpointer = files
while nextpointer:
ret.append(nextpointer.contents)
nextpointer = nextpointer.contents.next
It seems to work but did it work by accident ? does it have any design flaws ? Could anyone with experience on python ctypes confirm this a solution ? Any suggestion welcome.
From python2.7 documentation
next(iterator[, default])
Retrieve the next item from the iterator by calling its next() method. If default is given, it is returned if the iterator is
exhausted, otherwise StopIteration is raised.
from python3 documentation
next(iterator[, default])
Retrieve the next item from the iterator by calling its __next__() method. If default is given, it is returned if the iterator is
exhausted, otherwise StopIteration is raised.
Notice that next() method was removed from python3 but the function still exists.
This is all I can say about the next function and .next()/__next__() methods.
I downloaded the pymtp module and get_filelisting() is slightly different from what you posted in your ported code, here it is:
ret = []
next = files
while next:
ret.append(next.contents)
if (next.contents.next == None):
break
next = next.contents.next
If none of this helped you (which probably didn't :D), the version of pymtp library that I am using is 0.0.6 download using pip.