I'm looking reproducibility between android system (AOSP) daily builds against repo's collections of git repositories and the impact of using fast-forward merges.
The problem is that when doing daily builds you will get a tip-of-tree while development is happening elsewhere. If HEAD hasn't changed when the change is pushed, it will do a fast-forward merge and time will be re-written. So effectively the physical state of the repository 3 days ago will be different than asking git to go back three days.
The prime solution I can see is to use git with --no-ff to force merge commits. This injects a lot of noise for smaller commits, and seems to be considered bad practice in keeping a clean tree.
The background on this is trying to have reproducibility in an android build environment. For those that don't know, an android build is a collection of disparate git repositories. My ultimate use case is that I want to be able to say, put the code base in the state it was for developers 3 days ago. With fast-forward merges, we lose some critical information on how to correlate between changes on multiple repo projects (ie: git repositories).
As you've noticed, Git doesn't track the state of a branch over time. Two reasonable options are to either set tags for each build or to create a static manifest with the SHA-1s of all checked out commits at the time of the build:
repo manifest -r -o build_20131104.xml
Those files can either be checked into the manifest git and used like e.g.
repo init -u ... -m build_20131104.xml
to reproduce the prior state or you can save the files elsewhere.
Tagging all the gits litters the tag namespace if you're doing a lot of builds (and Git is currently fairly slow with thousands of tags) but is otherwise pretty convenient. Don't forget to tag the manifest git too.
Related
I'm working with an internal tool for a few users building in AOSP tree.
To enable the build of this tool it must be added to PRODUCT_PACKAGES variable somewhere in the make-system.
PRODUCT_PACKAGES+=mytool
Upto now I have maintained this as uncommitted changes to /build/make/target/product/base_system.mk I could of course fork this module and maintain my patch or maintain it as a diff and patch it when a new workspace is created.
The project now needs to be shared with a handful of coworkers, so this is a good opportunity to make it less hacky and more maintainable as a forked module requires constant merges of upstream patches.
The project only maintain one git repo for manifest and one for the tool itself.
How could I add this assignment of to a Makefile without having to maintain it as a shell script or a forked version of one of the android gits?
You will have to add mytool to the device configuration in device/<vendor>/<device>/device.mk.
Even if you happen to find a way to make it work from within the mytool repository you probably shouldn't, because it is the job of the integrator to add modules to the device. He can hardly keep track if packages are not collected at a single point and I can think of even more problems if the integration is spread over many repositories, potentially each with an own set of access rights.
A solution I use when I want to share an early version without the hassle of integration is providing it in form of local repo manifest which combines the mytool repository together with a patched device/<vendor>/<device> repository.
I have an app with different version.
But every version have different Git repository.
So please let me know how we can manage that every build variant have different repository.
You should have different tags or branches per version, not different repositories.
From the same repo, you would then use git worktree in order to clone the repo once, but checkout the repo multiple times: see "Multiple working directories with Git?".
That way, you have different folders, each one with a different version of your project.
Since only the name/color/logo are changing, you would need:
one repo with all the common code
one configuration file with the right values for name/color/logo per environment
one way for your code to detect in which environment it is deployed: your code you then pick the right value from the config file.
The point is: one project, one Git repo: See the 12 factors app (in particular, the Config section).
We have decided to follow the process of creating a new SVN branch for every new feature that we add to our mobile app. The ultimate goal behind this is to preserve the history for every single code change (this doesn't happen when we manually copy project folders into SVN instead of creating & merging branches).
My problem is that so far I am unable to merge changes from one branch to another.
I have already referred the following posts without success:
How to merge branch with trunk using SVN in android studio.
Android Studio Update Project: Merge vs Rebase vs Branch Default.
How to merge branch to SVN with Android studio.
How do merge specific svn revisions from branch to trunk in Android Studio 2.0.
Here is what I have tried so far:
I have two feature branches as can be seen below in Tortoise SVN:
I want to merge the changes in the branch Feature_A3 into branch Feature_A2. To do this, I am using the Merge from option in Android Studio's VCS from the A2 working copy:
It then asks me to select the branch to merge from (A3) or configure other branches:
I click on Configure Branches, just to show you the existing branch config:
As you can see, A2 is the Trunk and A3 is the Branch. Is this correct?
It then asks me what part of A3 I want to merge into A2. I select the /src directory (where the relevant changes are present):
It then generously gives me three different ways to perform the merge operation:
I select the third option as it directly gives me the changes I need to merge:
I click on Merge Selected and BAM!!! I get this error every time:
There is no clue as to what the "unresolved conflicts" or "skipped items" are. Why am I getting this error, and what should I do to merge the changes in A3 into A2 ??? Can someone please help ? All answers will be appreciated. Thanks ...
I have been following the official Intellij IDEA documentation below:
Integrating Changes To/From Feature
Branches.
Merging, Deleting, and Comparing
Branches.
Please note that:
Currently I am using Subversion, not Git.
The directory structure of my local working copies is not exactly
identical to that of the SVN repos. Could this be the cause of the error ?
THE ANSWER ...
Thanks to Peter Parker and especially Yoav Aharoni for their valuable feedback. As Yoav correctly pointed out, it was indeed the manner in which the branch locations folder was specified. It needs to be the folder containing the branches, not the branch folders themselves: And as Peter rightly said, checking "Include merged revisions" shows the merged history. I am now able to merge from within the IDE itself, and view the merged history in TortoiseSVN.
NO command line! YAY!!!
However, one last problem is that I am unable to view the merged history in Android Studio (Intellij IDEA) as described in Viewing Merge Sources. Does anyone know how to achieve this in Android Studio?
Phew, haven't used SVN in a while... :)
But from what I can remember Branch locations should be the folder containing your branches folders (and not each individual branch folder).
You see, typically a SVN repo follows a standard naming convention and folder structure:
trunk/
branches/
Feature_A2/
Feature_A3/
tags/
v1.01/
v1.02/
and so on...
trunk is where the main development takes place, and branches are for features, long-term or risky projects, or for different stages (such as QA and pre-prod).
So, as far as I remember, Android Studio expects you to set Branch locations to branches folder. In your case I think it should be http://192.168.0.64/svn/.../Android/Feature.
Also, your trunk is not Feature_A2 - Feature_A2 is just another branch.
Although I can't see the content, I think http://192.168.0.64/svn/.../Android/Development might be your trunk.
Which shouldn't bother you much, since you don't have to merge to your trunk, you can also merge between branches (e.g. merge Feature_A3 into Feature_A2).
So, to recap:
Although not mandatory, I recommend renaming your folders to match the conventions (you can easily do it by right clicking in Tortoise SVN, but only AFTER all teammates commit, otherwise merge will be a HELL for them).
Try setting Branch locations to http://192.168.0.64/svn/.../Android/Feature
And Trunk to http://192.168.0.64/svn/.../Android/Development (only if it indeed contains sources, similar to Feature_A2/3)
Consider "tagging" your releases in a tags folder (it's pretty much just copying the trunk/branch folder to tags, but you have a command for that).
If you do so, you can also add tags folder to Branch locations, that way you'll be able to compare your current source with any previous release (which is handy).
P.S: "unresolved conflicts" error can also mean you have unresolved conflicts (duh :)). Conflicts are are usually created when both you and a teammate change the same lines in file (or if he deletes a file you changed) and you update to get his changes.
SVN won't let you merge until you manually resolve these conflicts/changes.
You can find conflicts in the Version Control tab at the bottom, they'll be mark in red.
(But I don't think that was the problem in your case)
Let me know if that works for you!
I'm not an SVN expert, but I think you will need to change you directory structure. I believe that even though it is just a convention, svn uses the directory structure for merging of branches. So your directory Structure should be:
SVN
/Android
/branches
/production
/featureA2
/featureA3
/tags
/trunk
/IOS
/branches
/production
/featureA2
/featureA3
/tags
/trunk
Even though the TortoiseSVN Repo-browser will allow you to move your directories around, a lot of meta-data is stored on each directory and is used to handle the merge process, so you may have to start your repo again. See the subversion best practices guide and strategories for repository layout for more details on how to setup your repos. All that said, I would hesitate to rely on Android Studio's svn integration to do heavy lifting like merging of branches as (although most things in JetBrains tooling is fantastic) it's handling of SVN leaves a lot to be desired.
I want to develop on top of Android using a local Android repository server. I need to add several new git repositories to the hierarchy of gits, and I need to modify existing android sources for a custom tailoring of Android.
What is the "correct" way to clone the entire Android source tree of git repositories, such that I can push/pull to/from a common local repository server, and still easily pull new changes from Android upstream?
I am specifically looking for advice on how to use the repo script for interaction with my own server, and how to set up the manifest git repository and managing branches therein.
On your git server
repo init -u https://android.googlesource.com/platform/manifest --mirror # --mirror is the key
repo sync
Because you specified --mirror, all repos created will be 'bare' repos, which is the git-correct way to create a mirror unless you are a git uberlord.
On your client:
repo init -u git#git.yourserver.com:platform/manifest.git # you may use a different means of accessing your git server; I have to assume something for this example.
no repo sync yet. Your manifest is probably wrong. Look at the symbolic linked .repo/manifest.xml... cat it and read at the top the <remote fetch=""... it probably points back to android.googlesource.com. but if it says '..' I think that means 'come back to my server', so you can skip to step 6). But if it does point back to another server (not yours), then go to step 3.
cd .repo/manifests
vim .repo/manifests/default.xml (or whatever your active manifest.xml is).
Fix default.xml's <remote fetch="CHANGE ME" to point back to your git server
attempt a repo sync. It should pull only from your repository. If it doesn't, stop the repo sync, and try to fix default.xml again.
Once you see repo sync work on your test machine, then commit default.xml back in (git commit; git push), so that others on the team will have 'no-manifest-edit' experience when they repo init; repo sync from your server.
Once you see a 'no manifest edit' repo init; repo sync work, then go to your default.xml again, and just start adding new XML elements along with the tons of other existing Android project elements; these new elements will point to your custom projects. For these new projects, just git init them as you normally would, and make sure they have a branch matching the same <default revision="whatever_branch_you_see_here" so that a repo sync will succeed when it encounters these new projects.
If you do indeed set a default branch in your manifest <default revision="" element, then just have everyone make a local branch set to follow the remote branch specified in the revision attribute. So, for example, if <default revision="branch_a" is in your manifest, after you do a repo sync, when you cd into a sub-project of interest, do a :
git checkout -b branch_a origin/branch_a
Then, if the user git push's (there is no repo command to push, as far as I'm aware), and if someone else does a ./repo sync after that push, they will get those changes from the original user... as long as you are using the same manifest and actually pushing to the default revision (branch) specified by that manifest.
That's the simplest recipe. If you want to make actual feature branches, then you'll have to edit your manifest more regularly if you want 'repo sync' to just work... and you will have to communicate to the rest of the team to grab your version of the manifest when you do that. Alternatively, if it's just one or two git repos you are touching, then you could just forego repo sync and git push/pull like normal on those repos and ignore the rest of the quiet tree for the times you are heavily iterating. This sounds, ultimately to me, like a simpler path. I'd ignore repo as much as you possibly can; only using it for 'all project' syncs and leaving it alone for the times you are focusing on 1 or 2 projects.
Regarding getting updates from upstream. I think the way to do that is to change your default.xml to point back to your original git location (like android.googlesource.com), do a repo sync to cause all the new stuff to merge in, and once done with the sync, commit back up to your repo. I haven't done this yet; so I can't be too specific, but this is how I plan to do it.
I'm ignoring server administration details above. For instance, you need to call the repo init in a certain directory on your git server to cause it to be an available git repository; I'm assuming you know how to administer your git server.
I am working with a vendor who provides a patchset to the linux kernel in order to support android on top of their platform. This means that they base their string of patches on a specific linux version, and in their string of patches some of the android patches are included (cherry-picked I assume), which are applied to the same linux version.
So the history looks something like this when imported into git along with our changes which are applied on top:
v2.6.x.y v_rel_x.y o_rel_z
l--l--l---------v--v--a--v--a--a--v--v--v--------o--o--o
Where l are linux commits, v are vendor commits, a are android commits, and o are our commits.
What makes this complicated is that the android git kernel source based on the same linux kernel version is completely separate, looking like this:
v2.6.x.y v2.6.x.y+1
l--l--l---------l---l
\ \ android-2.6.x
\ a--a--a--a--a
\
\ v_rel_x.y o_rel_z
v--v--a--v--a--a--v--v--v--------o--o--o
Now, I want to include all of the android patches in the android-2.6.x release, however I also want all the vendors patches to support their platform. Unfortunately quite a few of the changes in the v2.6.x.y+1..android-2.6.x changeset are already applied to the v_rel_x.y branch. Thus a simple merge from android-2.6.x to o_rel_z creates a huge number of conflicts, which are simply impossible to solve by hand.
Do you have any idea on how to perform the merge from android-2.6.x to o_rel_z reliably?
And yes, there are really a huge number of commits on each branch, and the android patches on the v_rel_x.y are completely entangled with the vendor patches.
My two cents:
The reason of conflict is not that you have some of the commits cherry picked, but because some of the commits are changing the same files in the same places and git cannot apply the diff cleanly.
In case of conflicting changes you don't have many choices:
a) you merge and resolve conflicts to bring the code to the desirable state
b) you're trying to resolve conflicts automatically by merging using merging strategies like --ours or --theirs depending on what you consider more correct for your purposes.
See here Git merge documentation
Another approach you may try is to rebase your 0_rel_z branch on top of the updated upstream android-2.6.x
Finally, your problem is not the conflicts them-selves, but the amount of them that you have to deal with at once. So I would take a look at improving the process how you are managing the updates, may be to make this more often (daily vs weekly, for example) and constantly rebasing your development branches on top of the updating upstream until you're ready to finalize the development of a feature.
Hope that helps!