In my github actions jobs I would like to have n jobs.
job 1 would run gradle assembleDebug
job 2 to job n would run gradle connectedCheck. The difference between each job is the image used on the android emulator (each with a different android api)
The idea is that the project is built in job1 while job2 to job n would reuse what was already built in job1 and only build the tests (that are made by a task run by connectedCheck). This is in order to avoid having everything built from scratch in job2 to job n.
I already:
save the app/build app/.cxx files into a github cache in job1
restore that cache in job2 to job n.
I tried to touch the files in app/build & app/.cxx before running gradle again (so that I'm sure the files are all older than the source files).
But this still doesn't permit to gain in build time. In other words, the system still rebuilds everything, gradle runs the tasks that were done in job1 again.
Any idea on how to achieve this?
I found that:
touch was useless
the source files must have the same timestamps across the jobs. If github gets the repo with a git checkout, the files will have the timestamp of the time of the checkout. So to have always the same timestamp, you may use git-restore-mtime. There is a github actions chetan/git-restore-mtime-action#v1 for it.
I also had to add the $PROJECT_DIR/.gradle directory.
In addition to that, by running:
gradle with -i option
ninja with -d explain -v options
I found that:
I had other build/ dirs in other places than just in app/build/. Typically in some of the libraries I use. For example in additional_lib/my_lib/build/
this file is also needed: ~/.android/debug.keystore
the files & dirs in ANDROID_SDK_ROOT/ndk/<version> and ANDROID_SDK_ROOT/cmake/<version> must have the same timestamp across the jobs.
So in the job you have to:
be sure that the source files have the same timestamp across the jobs
set fetch-depth to 0
- name: checkout
uses: actions/checkout#v3
with:
fetch-depth: 0
use chetan/git-restore-mtime-action#v1
- name: restore timestamps
uses: chetan/git-restore-mtime-action#v1
be sure that files in these dirs have the same timestamp across the jobs.
ANDROID_SDK_ROOT/ndk/
ANDROID_SDK_ROOT/cmake/
cache these folders:
.gradle/
app/build/
app/.cxx/
additional_lib/my_lib/build/
~/.gradle/
~/.android/debug.keystore
Note: I also added ~/.gradle in the cache, but this is probably not mandatory
Related
Is there a way to build an Android system app from AOSP without having to clone the entire code tree and having to build the entire OS?
Just being able to build the unmodified app from a Linux shell is sufficient, with any toolchain that will do the job. Being able to make modifications in an IDE (Eclipse or Android Studio) is not a requirement (a text editor will do for making changes).
The app in question is CarrierConfig. Most of the app is just assets, the code consists of just one single Java class (~400 lines of code), but with four internal dependencies not exposed through the SDK API:
android.annotation.Nullable
android.os.PersistableBundle.restoreFromXml(XmlPullParser)
android.telephony.TelephonyManager.from(Context)
android.telephony.TelephonyManager#getCarrierIdFromMccMnc(String)
These are what prevents me from simple adding a generic build.gradle and running it through the gradle toolchain. The build artifact is a simple APK file, with which I would then patch the system image.
So how would I build this app without needing the entire AOSP source code (just the actual dependencies, and dependencies of dependencies etc.)?
Not a complete answer (yet), but some snippets I was able to find out so far:
Downloading just individual projects from the source tree
This is what I have been able to piece together from various instructions—untested so far:
mkdir <dir>
cd <dir>
repo init -u <url> -b <branch>
repo sync <project-list>
Where
<dir> is a dir on your system where you are going to keep the source
<url> is the URL for your build, e.g.:
AOSP: https://android.googlesource.com/platform/manifest
LineageOS: https://github.com/LineageOS/android.git
<branch> is the branch to check out (omit -b to check out the master branch)
AOSP branches are found at https://source.android.com/setup/start/build-numbers#source-code-tags-and-builds
LineageOS branches are found at https://github.com/LineageOS/android/branches
<project-list> is a list of projects to fetch (if omitted, repo sync will fetch the entire source tree). Projects can be indicated either by their name or by their path within the source tree, separated with spaces.
(source 1, source 2, source 3, source 4)
Figuring out which repos you need can get tricky, and if your dependencies have further dependencies, this can become a time-comsuming process.
Also I haven’t figured out if the next step actually works with a source tree stripped down in this manner.
Building individual projects
If you just need to build a single project, you can use mmm for that:
. build/envsetup.sh
lunch
mmm path/to/the/project/
(source)
I've synced the entire Android repo, and set up a build environment per the instructions here:
https://source.android.com/source/building
The build instructions seem to be assuming that you want to build the entire Android platform. I'm really interested in building a specific AOSP app, like contacts, SMS, camera, etc. I've seen mirrors of the stock app's code on GitHub, but there doesn't seem to be any build instructions within those, for example:
https://github.com/android/platform_packages_apps_contacts
https://github.com/android/platform_packages_apps_calendar
Is there a build guide for doing this? Am I stuck downloading, modifying, building this huge (100+GB) code set?
Just as you have 'mm' to build a certain target, you can also use 'mma' to build that target with its dependencies. For example:
$ mma Settings -j16
This will scan the project for the dependencies of the Settings app, and will afterward build the dependencies first before commencing the build of the Settings app.
here are compile and module-based compilation commands:
lunch: lunch <product_name>-<build_variant>
tapas: tapas [<App1> <App2> ...] [arm|x86|mips|armv5] [eng|userdebug|user]
croot: Changes directory to the top of the tree.
m: Makes from the top of the tree.
mm: Builds all of the modules in the current directory, but not their dependencies.
mmm: Builds all of the modules in the supplied directories, but not their dependencies.
To limit the modules being built use the syntax: mmm dir/:target1,target2
mma: Builds all of the modules in the current directory, and their dependencies.
mmma: Builds all of the modules in the supplied directories, and their dependencies.
cgrep: Greps on all local C/C++ files.
jgrep: Greps on all local Java files.
resgrep: Greps on all local res/*.xml files.
godir: Go to the directory containing a file.
you can look here for other and more build commands : https://source.android.com/setup/build/building
and here :https://elinux.org/Android_Build_System
And check "build/envsetup.sh" file's comments to see full list.
I am working on an android source code which I have downloaded from source.android.com.
After a full build I went through this site http://elinux.org/Android_Build_System which explains the android build system.
When I make changes in external/webkit code and build it with
make -j4 libwebcore it compiles the corresponding file and updates the libwebcore.so, and it save me a lot of time.
The same thing is applied to applications and also for building apks.
The problem arises when I make changes in the framework and give the command as
make -j4 framework its not compiling the corresponding files.
Can any one help me!
The folder frameworks contains many things, you have to be more specific about telling make what to build.
For example I made a change in:
frameworks/base/cmds/input/src/com/android/commands/input/Input.java.
Now the corresponding Android.mk file is located in:
frameworks/base/cmds/input/Android.mk, which contains a line saying: LOCAL_MODULE := input.
Thus the module being build from the source is called input, so I call:
$ make input
Which rebuilds that specific module.
As a bonus info, you can use the mmm helper and you can specify the path of the module to build like this:
$ mmm frameworks/base/cmds/input
or using mm which just builds the module in you current working directory:
$ cd frameworks/base/cmds/input
$ mm
I normally use mmm as my preferred tool.
Update
Oh, I see you might be talking specifically about the module called framework
I just tried to modify: frameworks/base/core/java/android/app/Dialog.java, and do a: make framework.
This seems to recompile the framework just fine. Which file exactly are you making changes in before running make framework ?
In response to your comment
I just tried to modify frameworks/base/core/java/android/webkit/WebView.java. mmm frameworks/base as well as make framework works perfectly fine for me.
If it does not work for you, can you update your question with additional information about which android version you are building, which commands you are typing exactly, and the output your are seeing?
Here are fuller descriptions of mm, mmm, and other convenient functions provided by sourcing the build/envsetup.sh file:
Invoke . build/envsetup.sh from your shell to add the following functions to your environment:
lunch: lunch <product_name>-<build_variant>
tapas: tapas [<App1> <App2> ...] [arm|x86|mips|armv5] [eng|userdebug|user]
croot: Changes directory to the top of the tree.
m: Makes from the top of the tree.
mm: Builds all of the modules in the current directory, but not their dependencies.
mmm: Builds all of the modules in the supplied directories, but not their dependencies.
To limit the modules being built use the syntax: mmm dir/:target1,target2.
mma: Builds all of the modules in the current directory, and their dependencies.
mmma: Builds all of the modules in the supplied directories, and their dependencies.
cgrep: Greps on all local C/C++ files.
jgrep: Greps on all local Java files.
resgrep: Greps on all local res/*.xml files.
godir: Go to the directory containing a file.
Plese check build/envsetup.sh file's comments to see full list of functions.
I am currently using Git on the command line to help me incrementally add features without breaking existing code. The part that worries me is this line of output from Git after committing:
[optimized_managed_event 6c9a98c] Added managed event insert into my ContentProvider
12 files changed, 202 insertions(+), 16 deletions(-)
rewrite bin/classes.dex (87%)
rewrite bin/classes/com/zeroe/SmartCalProvider.class (85%)
Should I worry about the rewrites if they are .class files and other types that aren't text? I am fairly new to Git, but am pretty comfortable with the command line and I understand the basic workflow for most Git projects:
> git add .
> git commit -m 'comment on commit'
> git checkout [master]
> git merge [branch]
What I am slightly worried about is issues that can occur when committing, then merging since Android projects have a lot of files that it creates itself in different formats.
My question is essentially in anything I need to worry about when doing this in Android development?
Create .gitignore in the root of your project and add at least the following:
*~
*.apk
bin
gen
local.properties
.apt_generated
This way you avoid putting in repository automatically generated files, which usually blows the repository size up without any reason. The only automatically generated files you might want to save are proguard/ files, which might be necessary to unroll the call-stack after the user-generated crash reports.
Also, I found it's very helpful to have giggle utility installed to see what changes you have in your files.
Given: http://developer.android.com/resources/faq/commontasks.html#filelist
What are the best practices for getting your projects into source control? I ask because if you simply right click on your project, choose team, etc. you end up with the /bin & /gen folders, .classpath as well as all the Eclipse related items.
If I'm inheriting a project with .../workspace/projectName et al. included how can I clean that up to include only the items relevant to the aforementioned URL?
I summarized all my findings in a blog post that can be found here: http://www.aydabtudev.com/2011/05/what-goes-into-source-control-android.html
I executed the following commands from within my project folder to get them out of source control:
svn rm --keep-local .classpath
svn rm --keep-local .project
svn rm --keep-local default.properties
svn rm --keep-local proguard.cfg
svn rm --keep-local bin/
svn rm --keep-local gen/
Then I executed the following command to add them to an ignore list:
svn pe svn:ignore .
Add each item above without the associated command like so:
.classpath
.project
bin/
...
I followed that up with a commit and an update to solidify my changes.
svn commit -q -m "Removing files" .
svn update
It would seem the smarter way to do this would be to configure the Ignored Resources under the Eclipse Team preferences.
If you're using SVN, you should selectively add files/directories to your repository.
For example with the following directory structure (quick example from my disk):
res/
src/
build/
.idea/
You do not want the build directory, nor the personal preferences for your IDE (.idea folder) adding, so you would only issue the command: svn add res src
To (I think) answer your second point, I'd manage everything to do with version control from command line initially, and then let your IDE do it.
My apologies if I'm missing the point of the question.
Here are some basic points:
Don't store stuff in version control that your source code produces. For example, if you build a jarfile, don't store that jarfile under source control.
Source control is for source. If you have releases, use a release repository like Artifactory. Don't let the Maven stuff scare you away. Maybe you don't use Maven (now), but a Maven repository tool is in standard format, and makes it easy to find your releases. Artifactory can work with Ant/Ivy, and with a little elbow grease, you can get it to work with C and C++ projects too.
Which brings me to the next statement: Don't store your jarfiles (if you're a Java project) in your source repository. It's convenient, but you'll end up hating yourself for it in the long run. Binary files take a long time to process in many source control systems and they can take up lots of room. What's even worse is that you lose information about them. For example what version of common-utils.jar is checked into Subversion that my project now depends upon. Again, use Artifactory and Ant/Ivy or Maven. If you're non-Java, you can use wget or curl to fetch your dependent libraries out of Artifactory. Again, don't let the whole Maven thing scare you.
If you have a Java project, and you don't use Maven, insist that code is stored in the repository using Maven's standard layout. That is, Java code is stored under src/main/java and non Java files are under src/main/resources. The advantage is that it makes it easy to move from project to project, and new developers can quickly find where things are. Plus, it makes your build.xml files much cleaner. You can use any standard repository layout you want, but by insisting on Maven's standard, you can squelch all complaints. "Hey, I agree with you, but Maven says you put your code under this directory. Sorry, I wish I could help, but my hands are tied"
If you're using Subversion, stick with the standard, trunk, branches, tags style and don't be too fancy. I'm not 100% crazy about the layout myself. (I'd rather have a main under the branches directory and no trunk), but you'll simply confuse developers and make support more difficult, and all for very little gain.
Make sure all projects (if you're using Ant) have standard target names. Again, I borrow Maven's naming convention. Make sure all build.xml use the description parameter in target names, and that internal only targets don't use description. That way, ant -p works. Also make sure that all built artifacts are under the target directory (again, Maven's way). It makes it easy to do a clean if you only have to delete the target directory. The idea of clean is to restore your layout to pristine checkout condition. Makes it much easier to use a tool like Jenkins. Which reminds me...
Use a continuous build tool like Jenkins. It helps you enforce your policy and standards. Unlike many tools, developers actually like Jenkins. And, you can add stuff like automatic testing, checkstyle, etc.
1.
It depends on your workflow. If you expect everybody who will ever work on your project to use eclipse having the .classpath folder in there is good because it keeps all your settings(library paths, external dependencies..)
To the best of my knowledge subclipse doesn't put the /bin folder under version control(it probably happened because of the weird way the repository shaped as you describe in 2.) because eclipse can generate that one on the fly as soon as it has the /src folder.
usually moving everything under /workspace/projectName to / and deleting /workspace is sufficient.