Ay up.
I'm currently in the process of making a game - it's a C++/SDL2/OpenGL combo which on PC works reasonably well. The long-term goal is to make this portable, though given that I lack most of the equipment to shove it onto every platform under the sun, the first "port", as it were, is likely to be for Android-powered phones and tablets.
SDL2 has an official (or "officially endorsed") port to Android, and after many weeks of haggling with Java and Eclipse and a mostly broken OpenGL ES 2.0 renderer, I've got stuff running on my lovely Nexus 7 tablet. It's not fast or pretty, but it's something to build on.
However, my game, as I would imagine many games like it, relies heavily on reading (and writing) data to and from XML files, be it saves, options, levels, you name it. But the Android NDK compresses all but a few file formats upon compilation - SDL2 has functionality to claw out images and audio from the Android-created /assets/ folder (where all my stuff is stored), but I use TinyXML2 for my XML handling, which can't cope.
There seems to be a small handful of possible solutions (it's a problem that turns up quite a bit - I'm well aware the "Similar Questions" box on my right is filled with similar woes as I type this), but each one seems to have either pretty big caveats or make pretty big assumptions about how the code is written. End results often being pointers to wacky variable types that won't fit neatly into tinyxml2, or I get stuck half way through because it turns out a piece of the puzzle isn't there. It's super frustrating.
Method a) seems to involve decompressing the apk at runtime and accessing the assets by brute force. It sounds like a really bad way of doing things and in some cases seems to assume the apk is placed in a specific location. Sounds pretty terrible to me, and complicated to achieve?
Method b) has the Java side effectively pass a pointer to the Android asset manager to the C++ side. This seems better, but examples I've seen of Java-to-C++ Android communication seem geared around very small "Hello World"-style projects where you're just passing across a string or a number. I also read this involved compiling Java headers and doing something novel with that- a process that would then fail because the Android asset manager isn't a standard data type or whatever.
c) SDL_RWops is meant to be geared up for Android and may be a preferable way of reading data from files. But it doesn't solve the subsequent brick walls as I try to convert this stuff into something tinyxml2 will understand.
Method d) seems to suggest you can get away with changing the extension of assets to that they're not compressed. So all my XML files become unofficial PNGs or whatever. That sounds terrible too, not least because I have 40+ XML files that would need to be managed. Supposedly there's a way to adjust what files are or aren't compressed, but some seem to think it's all or nothing, and I'm not sure how it's done regardless.
Understand that when you've been at at this for hours it's easy to let yourself be mislead. While I acknowledge the merit of figuring this stuff out for yourself, clues online are often incomplete, out of date, or are flat-out wrong.
So yes, a simple guide would be lovely. I've wasted many days exploring options and I'm sure it would be beneficial to the world to have a definitive answer.
As said, this works fine through the Windows PC build - reading and writing isn't a problem so I know it's not something stupid like dodgy files. For the Android port I'm using Eclipse and as said, the tablet I'm testing on is a Nexus 7. I have working knowledge of C++ and Java, but these fiddly bits drive me round the bend.
Thanks in advance
Well okay, I've taken a bit of a break and returned to this puzzle, and given that the situation hasn't really changed in terms of online tutorials or documentation or whatever, I've gone for the stupid option (option d) - I wrote a batch file to convert my XML files into JPEGs en masse, and now the program looks for JPEGs to read from and write to. How terrible.
On the plus side, tinyxml2 appears to be loading these fake JPEGs, but things crash as soon as it's given the command to look inside for specific elements, leading me to think it's probably not loading things properly (but at least it's no longer crashing on file load).
Debugging is painful and this whole strategy feels dirty and wrong. And there's nothing more disheartening than struggling with a question that presumably almost every Android game developer working in C++ has already answered in some form. Reading and writing files - why is this a challenge?
Related
All my life I've build my games and programs on top of prebuilt software, Unity,Unreal, Python etc, But I've never really learned how it's standardly done in the industry, ya engines are cool, but they also cause problems such as having limitations with what you can do, as well has having to work around bugs etc. But let's look at other software: MS Office, 7Zip, Evernote, Firefox, Youtube, Amazon Alexa app, Inkscape, Autodesk cad, Photoshop etc. These are just some random examples but they were NOT built in an engine yet manage to be cross-platform. What skills/languages are required to do that? To be clear I'm not asking for a miracle answer that can tell me everything I need to know, but I'm looking for a guide on the tools used to build things like that. Is it C++? Is it OpenGL? Am I still think too high up? I don't know much about it but are that the right direction? I've heard of things like electron but I think you could consider that more of an engine. I can't imagine these companies doing this at the binary level, I would think that would be absurd doing that for every program in such a day in age, also impossible for startups like Mozilla and Evernote to do as brand new companies without a large amount (if any) incoming revenue. In the end, I guess what I'm asking is, what is the industry standard when building cross-platform programs? What are the tools they tend to use? And what sources could I learn from to learn ore about this?
P.S. To be clear I understand things like Evernote, Youtube, Alexa App, etc have back-ends behind them, but I'm not talking about those, I primarily mean the front-end though I would think whatever the standard is would be capable of handling a back-end.
Ex-Middleware Game dev here. To make anything work on any given platform, at the lowest level you will have to start accessing the OS or hardware specific API's (So for example, on windows, CreateFile to open a file, or open() on linux). The C/C++ standard libraries build on top of those API's to provide a somewhat generic platform on which to build cross platform apps.
Realistically though, the C++ stdlib isn't overly helpful in this regard, mainly because things such as graphics, windowing APIs, etc; they are all outside of the remit of the stdlibs. Another big issue for us game devs w/r/t the C++ stdlibs on console, is that their behaviour tends to be targetted towards the 'general case', rather than the specific platform. Take for example the cmath functions. I am not, under any circumstances, going to be calling the extremely inefficient std::sin() implementation. std::sin is great in one regard - it handles denormal numbers, correctly identifies NAN/INF, and has a well described method of reporting errors.
In the games engine world, we tend to spend a lot of time upfront baking assets so that these INFs/NANs cannot make their way into the games computations ever. So handling of that stuff at runtime is a waste of time, so we usually write our own maths approximations (We aren't landing a man on the moon, we're just throwing some polys at the screen, so we usually don't need the accuracy provided by the stdlibs).
So how would a typical cross platform game be organised? You'd probably see a directory structure somewhat akin to this:
game/
platform/ //< contains all OS specific code (timers, mutexs, etc)
vpu/ //< wrappers over the SIMD instructions on the platform
maths/ //< fast versions of cmath + Vectors/Quats/Matrices/etc
graphics/ //< wrappers over the core graphics APIs
sound/ //< wrappers over the platform specific audio stuff
This is pretty much the 'platform' against which all other code is written against (in other words, we basically end up writing our own version of the C++ stdlib for each new platform). Whilst there is quite a lot of work involved in the above, it's usually reasonably straight forward to rewrite the entire code library when a new hardware platform comes along (e.g. Playstation 6, XBox 99, etc). Certainly that's less work than re-writing an entire game.
In some cases there are bits of that work that won't change (e.g. iOS and Android all use ARM CPUs, so the maths routines optimised for ARM NEON will be shared by both, as will the OpenGLES graphics routines).
With any luck, 99.99% of the games code will not need to be modified. With any luck - in many cases we aren't that lucky :( [Although its easier now than it was ten years ago!]
All to often (especially on console), towards the start of a games development you'll target your lovely abstracted core libraries, and all will be good. As you near the end of the project, you'll probably end up with a load of #ifdef XBOX defines that exploit specific performance gains of that specific hardware (we need to meet that 60fps goal, and we don't really care how tbh). In extreme cases you may find that a given platform needs so many platform specific optimisations (in say, the renderer) that it has effectively diverged into an entirely new library for that platform only.
Anyhow. This situation is a little different on PC and android - simply because the variety of hardware is significant (unlike say an XBOX, where they are all identical!). In those cases we will be writing code against an already abstracted API (e.g. OpenGL, OpenAL, D3D, etc), and we will have to insert a lot more runtime error checking than you would on console (for example, on console we may know we have 256Mb of ram. On Android it could be 32Mb, it could be 2Gb, who knows! It doesn't matter, we need to handle failures gracefully).
When it comes to desktop APPs, for windowing APIs, the vast majority of sane people just use QT now (possibly with OpenGL if they need 3D rendering).
I have a Python3 desktop application which I want to convert to an android apk. I saw that Kivy module exists and might be able to pull this off, but I am concerned about it's ability to make the apk work just like the python code. I use many different modules like PIL, opencv, pyserial, threading, watchdog, file_read_backwards etc).
Is this possible or I am asking for too much? And if it is, how can I change/handle for which android version it will the apk be?
threading is in the stdlib, so you have it
file_read_backwards is pure python and doesn't seem to require anything (as per setup.py) although the requirements_dev.txt lists a lot more things, so i'm not sure
PIL has a recipe (https://github.com/kivy/python-for-android/blob/master/pythonforandroid/recipes/pil/init.py), and there is also now a Pillow recipe, which is probably better to use (https://github.com/kivy/python-for-android/blob/master/pythonforandroid/recipes/Pillow/init.py) so that seems fine
pyserial is pure python, and i think i remember people having success with serial over usb on android, though i didn't try myself, it might require a device able to be an usb host and not just client, but i don't know much about that.
opencv has a recipe, it might be a more touchy one, as it hasn't been touched in years (aside from some cosmetic fixes), and i think i've seen people having issues with it, but i'd say it's worth a shot.
I'd say, it's certainly worth a shot, before converting any of your code, try just building a hello world application with kivy for android, then redo the build but adding your dependencies to the requirements one by one, and see if you can solve it or find help when it doesn't, if all go well, then look into porting your code, which is the part where the success will certainly depend more on you than on what kivy/python-for-android can do.
I found this answer that brought me to the idea instead of using the compiled tensorflow graph you might be able to use kivy on your Android phone. That way you could directly talk to the tensorflow graph using python-for-android.
A possible advantage would be to train/adapt the model on the fly. As far as I know otherwise you can only use the final trained model (but this is currently unanswered on stackoverflow). Also cross compiling to Windows Phone might be possible what currently isn't (see here).
I don't know the technical limitations. Anyone can confirm that this is possible and maybe what would be neccessary?
Although WinPhones could be a possibility in the future, there's basically a situation where almost no-one cares as there isn't really much interest about porting it. However, there's a thing in progress about using angle for translating openGL to DirectX, so it could be possible later. There's still this funny app packaging though, so it'll eat a lot of time.
Yet I think it might be possible to use those unofficial converters APK -> WinPhone app. Re TensorFlow: to me it looks like only a recipe is missing so try write one. :P
I have just been given a task at work to help audit a code base for a mobile app. I am not a mobile app programmer, although I've been a software developer for many years now, but know nothing about mobile apps. I was wondering if there's any tips or tools that I can use for this code audit.
I have seen the replies to this older post for a Java EE application, which can't be applied to my case since they're mostly based on having maven to build the app and in my case they use Gradle. Also these replies are from 2011 and perhaps there are more recent ones I'd really be very grateful to hear about.
In itself, the fact of appointing someone with no experience in the target environment seems like a complete nonsense to me, so I'd question the management here.
I do hope for you that you know at very least the languages these apps are written in: probably Java for Android & Objective-C for iOS (your question didn't mention what technologies your past experience concerned). If not, you're bound to just make remarks about comments, file size, and maybe some about naming conventions, which is of little interest compared to a real audit.
Beyond programming languages, iOS and Android are designed in very different ways, with different conventions & patterns. I actually know very few people who are really good in both environments, and there's a reason for this: these are different worlds, each of which you can easily spend your whole time on to learn APIs, common libraries, design philosophy, work-arounds for common issues, and understand a bit of how the internals work.
I don't know how much time you have to perform this task, but I'd suggest you learn how to code a basic app on the target environment, and learn about the key components.
My approach is generally:
gather some context from the team
get the source
build the app & get a taste of what it's doing (I usually hand-draw a screen flow diagram at this stage, it's useful later when you navigate in the code), also take note of bugs, slow features, non-user-friendly stuff (feedback is important to the team)
go to the source code, examine it's macroscopic layout:
. look at the build scripts to see what external libs it's using
. take note of the general package hierarchy, check that the naming is consistent, that packages are not overloaded with junk
. look generally at the class naming: is it consistent? do class names help figure out what's actually inside
. do some basic stats about file sizes: it's something that can quickly indicate some design flaws
now about the code in itself:
. read it until satisfied that you understand the general way it works (drawing a technical flow diagram helps), I like to start by the app entry-point (generally an activity in Android)
. make sure you spot how what you read achieves what you saw while testing the app
. take note of bad coding habits you spot while reading (naming, comments, it can be anything: there's no limit to how bad the code can be ^^)
. take note of unreadable/overly-complex bits of code (but don't spend days just to understand them)
. if you had noticed slow features in the app, it might be worth looking at those bits of code a little more carefully
. have a good night sleep, then re-read all your notes, and try to extract some high-level remarks about the application design
Now, specifically for Android, here the most common list of things to look for, based on my experience:
components life-cycle handling issues (for components like activities, services, fragments and such): symptoms include device rotation and application switches causing issues
thread handling issues (things done on the UI thread, when they should really run in background)
massive activities / services (many people think that creating activities / fragments / services is all that's required in terms of architecture - it is true only for very simple apps)
I won't enter more into the specifics, because people a lot more intelligent than me wrote books about this. And you have to code apps to really get a grasp of those subjects: a lot of them, so that's what you should start with: code apps yourself, otherwise: 1/ your audit will be irrelevant 2/ the team will spot your lack of skills pretty fast - depending on the aim of this audit, you might have a very hard time facing them...
A coworker and I were talking (after a fashion) about an article I read (HTC permission security risk). Basically, the argument came down to whether or not it was possible to log every action that an application was doing. Then someone (an abstract theroetical person) would go through and see if the app was doing what it was supposed to do and not trying to be all malicious like.
I have been programming in Android for a year now, and as far as I know if -- if -- that was possible, you would have to hack Dalvik and output what each process was doing. Even if you were to do that, I think it would be completely indecipherable because of the sheer amount of stuff each process was doing.
Can I get some input one way or the other? Is it completely impractical to even attempt to log what a foriegn application is doing?
I have been programming in Android for a year now, and as far as I know if -- if -- that was possible, you would have to hack Dalvik and output what each process was doing.
Not so much "hack Dalvik" but "hack the android.* class library, and perhaps a few other things (e.g., java.net).
Even if you were to do that, I think it would be completely indecipherable because of the sheer amount of stuff each process was doing.
You might be able to do some fancy pattern matching or something on the output -- given that you have determined patterns of inappropriate actions. Of course, there is also the small matter of having to manually test the app (to generate the output).
Is it completely impractical to even attempt to log what a foriegn application is doing?
From an SDK app? I damn well hope so.
From a device running a modded firmware with the aforementioned changes? I'd say it is impractical unless you have a fairly decent-sized development team, at which point it is merely expensive.
This is both possible and practical if you are compiling your own ROM. Android is based on Linux and I know several projects like this for Linux, like Linux Trace Toolkit. I also know of research into visualizing the results and detecting malicious apps from the results as well.
Another thing functionality like this is often used for is performance and reliability monitoring. You can read about the DTRACE functionality in Solaris to learn more about how this sort of stuff is used in business rather than academia.