Develop Reference

Repo with non android projects

I am working on a project where I am trying to unite several modules into one solution. The modules are each in their own folder and are git repositories. These are all stored in C:\sourcecode\Modules Eventually they will be on GitHub. After deep reviews of different methods of using a Solutions made up of Module stored in git repositories, I decided to try Google's Repo that was built for AOSP.
I installed all the tools based on the Repo requirements here https://source.android.com/setup/develop and created a folder C:\sourcecode\Repotest in that folder I created a file called default.xml. The contents of that folder are very simple:
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<remote name="origin"
fetch="/c/sourcecode/Modules" />
<default revision="master"/>
<project path="AnalystQualification" name="AnalystQualification" />
</manifest>
I launch git-bash as administrator and run:
$ cd /c/sourcecode/Repotest/
$ repo init -u default.xml
I get the output:
$ repo init -u default.xml
Downloading manifest from default.xml
Traceback (most recent call last):
File "C:\sourcecode\Repotest\.repo\repo\main.py", line 630, in <module>
_Main(sys.argv[1:])
File "C:\sourcecode\Repotest\.repo\repo\main.py", line 604, in _Main
result = run()
File "C:\sourcecode\Repotest\.repo\repo\main.py", line 597, in <lambda>
run = lambda: repo._Run(name, gopts, argv) or 0
File "C:\sourcecode\Repotest\.repo\repo\main.py", line 266, in _Run
result = cmd.Execute(copts, cargs)
File "C:\sourcecode\Repotest\.repo\repo\subcmds\init.py", line 531, in Execute
self._SyncManifest(opt)
File "C:\sourcecode\Repotest\.repo\repo\subcmds\init.py", line 232, in _SyncManifest
default_branch = m.ResolveRemoteHead()
File "C:\sourcecode\Repotest\.repo\repo\project.py", line 1926, in ResolveRemoteHead
output = self.bare_git.ls_remote('-q', '--symref', '--exit-code', name, 'HEAD')
File "C:\sourcecode\Repotest\.repo\repo\project.py", line 3040, in runner
raise GitError('%s %s: %s' %
error.GitError: manifests ls-remote: fatal: invalid gitfile format: default.xml
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Downloading Repo source from https://gerrit.googlesource.com/git-repo
I think this error is telling me that setup in default.xml cant see the .git folder in the modules/analystqualification directory but I can see it if I ls -l
$ ls -l /C/sourcecode/Modules/AnalystQualification
total 4
drwxr-xr-x 1 adam.wheeler 1049089 0 Jan 12 10:01 Setup_AQ/
Any help is appreciated here. Thanks!
Adam

repo init -u url_to_manifest_repo -m foo.xml -b manifest_repo_branch
url_to_manifest_repo should be a git repository that tracks foo.xml. It can be either in the local disk or in a remote hosting server. In your case, it's /c/sourcecode/Repotest. Make sure that /c/sourcecode/Repotest is a git repository and default.xml has been trakced.
foo.xml is the manifest file's path relative to url_to_manifest_repo root. -m foo.xml can be omitted. If so, -m default.xml is used by default.
manifest_repo_branch is the branch that holds the specific version of foo.xml. If -b manifest_repo_branch is omitted, it defaults to -b master.
So in your case the command would be:
repo init -u /c/sourcecode/Repotest -m default.xml -b master
or simply:
repo init -u /c/sourcecode/Repotest
For a local repository, it could also be -u file:///c/sourcecode/Repotest.

How do you update a mirrored repo workspace?

I created a mirrored repo workspace like this:
repo init -u $url -b $branch -m $manifest --mirror
repo sync
I would like to update the mirror to use a different manifest or an updated manifest and then re-sync to pull down any new or missing changes. If I try to run the same init command, it complains:
fatal: --mirror is only supported when initializing a new workspace.
Either delete the .repo folder in this workspace, or initialize in another location.
I suppose that I could manually fetch within .repo/manifests and then update the symlink, .repo/manifest.xml. I'd like to use the repo tool instead, though, if possible.
Or I could remove the entire .repo directory and then run the init command again. I'm avoiding this because unfortunately my manifest file is inside a 12GB repository. So, re-cloning is time consuming.

I ended up abandoning this idea. Instead, I create a throw-away git repo locally containing the manifest. Then I init from that. The shell functions below will create the temporary manifest repo, create/update the mirror, and then create a working repo for building.
function parameters
mirror
/path/to/mirror
url
/path/to/temporary/manifests_repo
branch
branch name to use.
This is not strictly necessary, but it makes it easy to label your build directories. I use the hash from our main repo corresponding to the current manifest.xml. Then, I can easily see which revision a build directory is based upon:
$ repo info Manifest
...
Manifest merge branch: refs/heads/BRANCH_NAME
manifest
relative/path/to/manifest_file from the current directory
build_dir
/path/to/new_build_directory
The reason for creating the mirror is to save time fetching as this code was designed for an automated build server. Notice that the build directory does a shallow clone from the mirror and even gets its copy of Google's repo tool from a local bundle allowing for offline building. The sync step uses options to only fetch the changesets required by the manifest and excludes other branches and commits.
function pushd_quiet() {
pushd "$#" >/dev/null
}
function popd_quiet() {
popd "$#" >/dev/null
}
function build_android_os_update_mirror() {
local mirror="$1"
local url="$2"
local branch="$3"
local manifest="$4"
# Create a temporary manifest repo
rm -rf "${url}"
mkdir -p "${url}"
cp -rf "${manifest}" "${url}"
pushd_quiet "${url}"
{
git init
git add .
git commit -m $branch
git branch -m $branch
popd_quiet
}
# Update mirror
mkdir -p $mirror
pushd_quiet $mirror
{
rm -rf .repo
repo init -u $url -b $branch -m $(basename $manifest) --mirror \
--repo-url ${BUILD_HOME}/tools/git-repo-clone.bundle
repo sync -c --optimized-fetch -j 16
popd_quiet
}
}
function build_android_os_create_build_tree() {
local build_dir="$1"
local mirror="$2"
local url="$3"
local branch="$4"
local manifest="$5"
rm -rf $build_dir
mkdir -p $build_dir
pushd_quiet $build_dir
{
repo init -u $url -b $branch -m $(basename $manifest) --reference=$mirror --depth=1 \
--repo-url ${BUILD_HOME}/tools/git-repo-clone.bundle
repo sync -d -c --optimized-fetch -j 16
popd_quiet
}
}

Building webrtc library

I am trying to build webrtc android demo application. I was following README applied by the project. First thing was mentioned about gclient but when i tried to run it "no external or internal command". Then i got this link according to the this link i cloned the depot_tool repository but when at the step when i try to run gclient i got this
So after hours of trying i am not able to run it. if any one has experience with this WebRTC library please provide some help. Any guidance? Any Hlep will be appreciated

I heard few things were changed from svn to GIT. We tried building webRTC to windows7 using visual studio. See if you can use any parts on this.
Get depot_tools software and extract to a location (set this path in environment variable).
cd to depot_tools directory and hit 'gclient' (this should take sometime)
Install python 2.7 and put it path.
create a folder of your choice for webRTC and open command prompt there.
Now hit, gclient config http://webrtc.googlecode.com/svn/trunk
then, gclient sync.
Then set GYP_GENERATORS=msvs environment variable (use ninja if you want that).
Also set msvs_version=2012 (depending on your visual studio version)
now run, gclient runhooks --force
This should generate solutions files for your folder. fireup the libjingle/webrtc.sln files and start building it. I'm not sure if it builds as is. But we had hicupps with that.
See this link on WebRTC native development if you haven't already. From my personal experience building this tricky job initially.
Also note that links might be deprecated as they moved to git, don't know if they are still maintaining it.

How to compile WEBRTC On ubuntu
download script and run:
https://cs.chromium.org/chromium/src/build/install-build-deps.sh
./build/install-build-deps.sh --no-chromeos-fonts
Now download depot tools
$ git clone --depth 1 https://chromium.googlesource.com/chromium/tools/depot_tools.git
Put it in Path
$ export PATH=`pwd`/depot_tools:"$PATH"
Define your settings:
export GYP_DEFINES="target_arch=x64 host_arch=x64 build_with_chromium=0 use_openssl=0 use_gtk=0 use_x11=0 include_examples=0 include_tests=1 fastbuild=1 remove_webcore_debug_symbols=1 include_pulse_audio=0 include_internal_video_render=0 clang=1 "
make file to download the source
mkdir webrtc-checkout
cd webrtc-checkout
I chose branch 50 you can change it
gclient config https://chromium.googlesource.com/external/webrtc.git#branch-heads/50 --name=src
gclient sync --force --with_branch_heads
gclient sync --force --with_branch_heads --nohooks
cd src
git checkout branch-heads/50
gclient runhooks
ninja -C ./out/Release
Collect your LIB & Include file to use it in your project
mkdir ../lib/
find ./ -name "*.o" -and -not -name do_not_use -and -not -name protoc -and -not -name genperf -exec ar crs ../lib/libwebrtc.a {} +
mkdir ../include
find ./ -name *.h -exec cp --parents '{}' ../include ';'

Restore deleted file from Android source code

I'm building Android source and deleted some .java files from the framework. I made a repo sync hoping that this restore the deleted file but it didn't happen.
How can I restore this and other deleted files from original source tree?

Check damages:
$ repo status
project device/samsung/maguro/ (*** NO BRANCH ***)
-m full_maguro.mk
project frameworks/base/ (*** NO BRANCH ***)
-- media/java/android/media/#AudioSystem.java#
-d media/java/android/media/AudioManager.java
project frameworks/opt/telephony/ (*** NO BRANCH ***)
-d src/java/com/android/internal/telephony/CallManager.java
project packages/apps/Phone/ (*** NO BRANCH ***)
-d src/com/android/phone/InCallScreen.java
-d src/com/android/phone/PhoneUtils.java
restore files:
$ repo forall -c 'git reset --hard ; git clean -fdx'

If you are using Eclipse you can right-click on the project or folder and select "Restore From Local History".
If the files are already purged from your local history for some reason then it will depend on which source control system you are using and whether you committed your changes.

How to download google source code for android

As you know, there is a list of several hundred projects in https://android.googlesource.com/. I'd like to download them all in windows machine. According to Google's document,
To install, initialize, and configure Repo:
$ curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
$ chmod a+x ~/bin/repo
To clone the entire platform, install repo, and run:
mkdir mydroid
cd mydroid
repo init -u https://android.googlesource.com/platform/manifest
repo sync
In my machine, however, I cannot "repo init" in Git Bash because it says it does not have python. I have python installed but git bash does not recognize it. Note that I set the python directory to the system path too. If anybody can give a tip, I would appreciate it. Thanks
UPDATE: I believe it's problem with new version of Git Bash for Windows. System path is not applied to Git Bash at all - I could easily test if system path worked with command prompt. Anyway, I tried this instead and it actually ran with error of course.
/c/python27/python.exe ../bin/repo init -u https://android.googlesource.com/platform/manifest
The error message is
$ /c/python27/python.exe ../bin/repo init -u https://android.googlesource.com/platform/manifest
Traceback (most recent call last):
File "../bin/repo", line 91, in <module>
import readline
ImportError: No module named readline
OK. I passed this error by installing pyreadline in windows:
easy_install pyreadline
If you got an error, you must install setuptools from
http://pypi.python.org/pypi/setuptools#files
And finally ran the command again to get this:
$ repo init -u https://android.googlesource.com/platform/manifest
fatal: unable to start d:\mywork\dev\GoogleAndroid\working_dir\.repo\repo/main.py
fatal: [Errno 8] Exec format error

With one click, download the latest code as .tar.gz file, from here
https://android.googlesource.com/platform/frameworks/base/+archive/master.tar.gz, the android could be found under core folder
Edit
Alternative here:
http://grepcode.com/project/repository.grepcode.com/java/ext/com.google.android/android/
Just select the version then a download options within.

If you consider, as an example, this other program "sympy" which also needs git bash and python, it is only a matter to add python to your PATH prior to launching the git bash session.
Install Python from:
http://python.org/download/
by downloading the "Python 2.7 Windows installer" (or Python 2.6 or 2.5) and running it.
Add python directory to your system environment path variable
(My Computer -> Advanced -> Environment Variables -> Path -> Edit).
Note that the repo script itself must be in the path, as mentioned in the Version Control page of android:
Repo is a repository management tool that we built on top of Git. Repo unifies the many Git repositories when necessary, does the uploads to our revision control system, and automates parts of the Android development workflow.
Repo is not meant to replace Git, only to make it easier to work with Git in the context of Android.
The repo command is an executable Python script that you can put anywhere in your path.

This answer explains how to fix this error:
fatal: unable to start c:\path\.repo\repo/main.py
fatal: [Errno 8] Exec format error
Summary: I finally used the python packaged by Cygwin.
Details: Below is the full story.
The tip from the repo bug tracking is to add '/c/app/Python27/python ':
line 136 in v1.20
REPO_MAIN = '/c/app/Python27/python ' + S_repo + '/main.py'
line 735 in v1.20 (beginning of function main)
wrapper_path = '/c/app/Python27/python ' + os.path.abspath(__file__)
But we get the error TypeError: coercing to Unicode: need string or buffer, NoneType found
Therefore I reverted these changes above and performed the other changes below (on version 1.20):
line 136, replaced single slash by double back-slash:
REPO_MAIN = S_repo + '\\main.py'
line 766, added python absolute path as first element of me:
me = ['C:\\app\\Python27\\python.exe', repo_main,
'--repo-dir=%s' % rel_repo_dir,
'--wrapper-version=%s' % ver_str,
'--wrapper-path=%s' % wrapper_path,
'--']
line 776, replaced os.execv(repo_main, me) by
os.execv('C:\\app\\Python27\\python.exe', me)
However we get still an error:
$ Traceback (most recent call last):
File "c:\path\.repo\repo\main.py", line 39, in <module>
from subcmds.version import Version
File "c:\path\.repo\repo\subcmds\__init__.py", line 36, in <module>
['%s' % name])
File "c:\path\.repo\repo\subcmds\forall.py", line 17, in <module>
import fcntl
ImportError: No module named fcntl
The Python v2.7 fcntl documentation says fcntl is available for platform Unix only.
I finally reverted again all changes in repo script and installed Cygwin including its python and git packages: it succeeded as a charm.
But, as the symlinks simulated by Cygwin are not recognized by the MSysGit, we have to use the Cygwin git. And GUIs on top of git are not fully compliant with Cygwin git...
(see also my other post)
Edit:
Cygwin can use native NTFS symlinks (just set CYGWIN=winsymlinks:native and be Admin). Therefore MSysGit can be used and any other GUI based on it :-)