Related
I am making an NW.js app on macOS, and want to run the app in dev mode
by double-clicking on an icon.
In the first step, I'm trying to make my shell script work.
Using VS Code on Windows (I wanted to gain time), I have created a run-nw file at the root of my project, containing this:
#!/bin/bash
cd "src"
npm install
cd ..
./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &
but I get this output:
$ sh ./run-nw
: command not found
: No such file or directory
: command not found
: No such file or directory
Usage: npm <command>
where <command> is one of: (snip commands list)
(snip npm help)
npm#3.10.3 /usr/local/lib/node_modules/npm
: command not found
: No such file or directory
: command not found
Some things I don't understand.
It seems that it takes empty lines as commands.
In my editor (VS Code) I have tried to replace \r\n with \n
(in case the \r creates problems) but it changes nothing.
It seems that it doesn't find the folders
(with or without the dirname instruction),
or maybe it doesn't know about the cd command ?
It seems that it doesn't understand the install argument to npm.
The part that really weirds me out, is that it still runs the app
(if I did an npm install manually)...
Not able to make it work properly, and suspecting something weird with
the file itself, I created a new one directly on the Mac, using vim this time.
I entered the exact same instructions, and... now it works without any
issues.
A diff on the two files reveals exactly zero difference.
What can be the difference? What can make the first script not work? How can I find out?
Update
Following the accepted answer's recommendations, after the wrong line
endings came back, I checked multiple things.
It turns out that since I copied my ~/.gitconfig from my Windows
machine, I had autocrlf=true, so every time I modified the bash
file under Windows, it re-set the line endings to \r\n.
So, in addition to running dos2unix (which you will have to
install using Homebrew on a Mac), if you're using Git, check your
.gitconfig file.
Yes. Bash scripts are sensitive to line-endings, both in the script itself and in data it processes. They should have Unix-style line-endings, i.e., each line is terminated with a Line Feed character (decimal 10, hex 0A in ASCII).
DOS/Windows line endings in the script
With Windows or DOS-style line endings , each line is terminated with a Carriage Return followed by a Line Feed character. You can see this otherwise invisible character in the output of cat -v yourfile:
$ cat -v yourfile
#!/bin/bash^M
^M
cd "src"^M
npm install^M
^M
cd ..^M
./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &^M
In this case, the carriage return (^M in caret notation or \r in C escape notation) is not treated as whitespace. Bash interprets the first line after the shebang (consisting of a single carriage return character) as the name of a command/program to run.
Since there is no command named ^M, it prints : command not found
Since there is no directory named "src"^M (or src^M), it prints : No such file or directory
It passes install^M instead of install as an argument to npm which causes npm to complain.
DOS/Windows line endings in input data
Like above, if you have an input file with carriage returns:
hello^M
world^M
then it will look completely normal in editors and when writing it to screen, but tools may produce strange results. For example, grep will fail to find lines that are obviously there:
$ grep 'hello$' file.txt || grep -x "hello" file.txt
(no match because the line actually ends in ^M)
Appended text will instead overwrite the line because the carriage returns moves the cursor to the start of the line:
$ sed -e 's/$/!/' file.txt
!ello
!orld
String comparison will seem to fail, even though strings appear to be the same when writing to screen:
$ a="hello"; read b < file.txt
$ if [[ "$a" = "$b" ]]
then echo "Variables are equal."
else echo "Sorry, $a is not equal to $b"
fi
Sorry, hello is not equal to hello
Solutions
The solution is to convert the file to use Unix-style line endings. There are a number of ways this can be accomplished:
This can be done using the dos2unix program:
dos2unix filename
Open the file in a capable text editor (Sublime, Notepad++, not Notepad) and configure it to save files with Unix line endings, e.g., with Vim, run the following command before (re)saving:
:set fileformat=unix
If you have a version of the sed utility that supports the -i or --in-place option, e.g., GNU sed, you could run the following command to strip trailing carriage returns:
sed -i 's/\r$//' filename
With other versions of sed, you could use output redirection to write to a new file. Be sure to use a different filename for the redirection target (it can be renamed later).
sed 's/\r$//' filename > filename.unix
Similarly, the tr translation filter can be used to delete unwanted characters from its input:
tr -d '\r' <filename >filename.unix
Cygwin Bash
With the Bash port for Cygwin, there’s a custom igncr option that can be set to ignore the Carriage Return in line endings (presumably because many of its users use native Windows programs to edit their text files).
This can be enabled for the current shell by running set -o igncr.
Setting this option applies only to the current shell process so it can be useful when sourcing files with extraneous carriage returns. If you regularly encounter shell scripts with DOS line endings and want this option to be set permanently, you could set an environment variable called SHELLOPTS (all capital letters) to include igncr. This environment variable is used by Bash to set shell options when it starts (before reading any startup files).
Useful utilities
The file utility is useful for quickly seeing which line endings are used in a text file. Here’s what it prints for for each file type:
Unix line endings: Bourne-Again shell script, ASCII text executable
Mac line endings: Bourne-Again shell script, ASCII text executable, with CR line terminators
DOS line endings: Bourne-Again shell script, ASCII text executable, with CRLF line terminators
The GNU version of the cat utility has a -v, --show-nonprinting option that displays non-printing characters.
The dos2unix utility is specifically written for converting text files between Unix, Mac and DOS line endings.
Useful links
Wikipedia has an excellent article covering the many different ways of marking the end of a line of text, the history of such encodings and how newlines are treated in different operating systems, programming languages and Internet protocols (e.g., FTP).
Files with classic Mac OS line endings
With Classic Mac OS (pre-OS X), each line was terminated with a Carriage Return (decimal 13, hex 0D in ASCII). If a script file was saved with such line endings, Bash would only see one long line like so:
#!/bin/bash^M^Mcd "src"^Mnpm install^M^Mcd ..^M./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &^M
Since this single long line begins with an octothorpe (#), Bash treats the line (and the whole file) as a single comment.
Note: In 2001, Apple launched Mac OS X which was based on the BSD-derived NeXTSTEP operating system. As a result, OS X also uses Unix-style LF-only line endings and since then, text files terminated with a CR have become extremely rare. Nevertheless, I think it’s worthwhile to show how Bash would attempt to interpret such files.
On JetBrains products (PyCharm, PHPStorm, IDEA, etc.), you'll need to click on CRLF/LF to toggle between the two types of line separators (\r\n and \n).
I was trying to startup my docker container from Windows and got this:
Bash script and /bin/bash^M: bad interpreter: No such file or directory
I was using git bash and the problem was about the git config, then I just did the steps below and it worked. It will configure Git to not convert line endings on checkout:
git config --global core.autocrlf input
delete your local repository
clone it again.
Many thanks to Jason Harmon in this link:
https://forums.docker.com/t/error-while-running-docker-code-in-powershell/34059/6
Before that, I tried this, that didn't works:
dos2unix scriptname.sh
sed -i -e 's/\r$//' scriptname.sh
sed -i -e 's/^M$//' scriptname.sh
If you're using the read command to read from a file (or pipe) that is (or might be) in DOS/Windows format, you can take advantage of the fact that read will trim whitespace from the beginning and ends of lines. If you tell it that carriage returns are whitespace (by adding them to the IFS variable), it'll trim them from the ends of lines.
In bash (or zsh or ksh), that means you'd replace this standard idiom:
IFS= read -r somevar # This will not trim CR
with this:
IFS=$'\r' read -r somevar # This *will* trim CR
(Note: the -r option isn't related to this, it's just usually a good idea to avoid mangling backslashes.)
If you're not using the IFS= prefix (e.g. because you want to split the data into fields), then you'd replace this:
read -r field1 field2 ... # This will not trim CR
with this:
IFS=$' \t\n\r' read -r field1 field2 ... # This *will* trim CR
If you're using a shell that doesn't support the $'...' quoting mode (e.g. dash, the default /bin/sh on some Linux distros), or your script even might be run with such a shell, then you need to get a little more complex:
cr="$(printf '\r')"
IFS="$cr" read -r somevar # Read trimming *only* CR
IFS="$IFS$cr" read -r field1 field2 ... # Read trimming CR and whitespace, and splitting fields
Note that normally, when you change IFS, you should put it back to normal as soon as possible to avoid weird side effects; but in all these cases, it's a prefix to the read command, so it only affects that one command and doesn't have to be reset afterward.
Coming from a duplicate, if the problem is that you have files whose names contain ^M at the end, you can rename them with
for f in *$'\r'; do
mv "$f" "${f%$'\r'}"
done
You properly want to fix whatever caused these files to have broken names in the first place (probably a script which created them should be dos2unixed and then rerun?) but sometimes this is not feasible.
The $'\r' syntax is Bash-specific; if you have a different shell, maybe you need to use some other notation. Perhaps see also Difference between sh and bash
Since VS Code is being used, we can see CRLF or LF in the bottom right depending on what's being used and if we click on it we can change between them (LF is being used in below example):
We can also use the "Change End of Line Sequence" command from the command pallet. Whatever's easier to remember since they're functionally the same.
One more way to get rid of the unwanted CR ('\r') character is to run the tr command, for example:
$ tr -d '\r' < dosScript.py > nixScript.py
I ran into this issue when I use git with WSL.
git has a feature where it changes the line-ending of files according to the OS you are using, on Windows it make sure the line endings are \r\n which is not compatible with Linux which uses only \n.
You can resolve this problem by adding a file name .gitattributes to your git root directory and add lines as following:
config/* text eol=lf
run.sh text eol=lf
In this example all files inside config directory will have only line-feed line ending and run.sh file as well.
For Notepad++ users, this can be solved by:
The simplest way on MAC / Linux - create a file using 'touch' command, open this file with VI or VIM editor, paste your code and save. This would automatically remove the windows characters.
If you are using a text editor like BBEdit you can do it at the status bar. There is a selection where you can switch.
For IntelliJ users, here is the solution for writing Linux script.
Use LF - Unix and masOS (\n)
Scripts may call each other.
An even better magic solution is to convert all scripts in the folder/subfolders:
find . -name "*.sh" -exec sed -i -e 's/\r$//' {} +
You can use dos2unix too but many servers do not have it installed by default.
For the sake of completeness, I'll point out another solution which can solve this problem permanently without the need to run dos2unix all the time:
sudo ln -s /bin/bash `printf 'bash\r'`
Has anyone successfully modified files in an Android Studio project throughout the repository history? I'm trying to remove some sensitive data that was entered mistakenly and spans several commits (it's a phone number). I'm attempting to run the following command:
git filter-branch --tree-filter "find . -name '*.java' -print0 | xargs sed -i '' -e 's/xxxxxxxxxx/0000000000/g'"
But I'm immediately getting the error
Rewrite c61760bca0273b8597299146fa5c43f984a50e3c (1/22)sed: can't read : No such file or directory
tree filter failed: find . -name '*.java' -print0 | xargs sed -i '' -e 's/xxxxxxxxxx/0000000000/g'
where xxxxxxxxxx is the number to be switched out. The files in question are stored in a deeper directory, but for some reason sed can't even find them. I'm guessing the error means it fails on the first commit because sed can't read the files? I'm unsure why, because they are definitely local.
Can anyone shed some light on my trouble? Many thanks in advance.
You don't need to worry about obscure flags if you use the BFG, rather than git-filter-branch. The BFG is designed specifically for the case of remove unwanted data- where you don't really care where the bad data is, in what obscure file - you just want it gone.
To use the BFG, create a unwanted.txt file, containing just one line like this:
xxxxxxxxxx==>0000000000
Then run the BFG with this command:
$ java -jar bfg.jar -fi '*.java' --replace-text unwanted.txt my-repo.git
Your entire repository history will be scanned, and all .java files (under 1MB in size) will have the substitutions performed: any matching string (that isn't in your latest commit) will be replaced.
The BFG is typically hundreds of times faster than running git-filter-branch on a big repo and the options are tailored around these two common use-cases:
Removing Crazy Big Files
Removing Passwords, Credentials & other Private data
Full disclosure: I'm the author of the BFG Repo-Cleaner.
I'm not an expert of linux/unix shell commands.
Sorry for being so specific, but I would like to ask:
Suppose I have a folder and I need to check that it's empty of "simple" files (all files excluding folders/directories), and also ignore a specific folder inside it. How do I do that?
Do note that I intend to run the command on Android (without using the API, because I intend to use it for checking out the info of protected folders), so it might have less commands than the normal shell of linux/unix .
If needed, I can run multiple shell commands, and check the output of each of them before going to the next command.
This may help you
find /path/to/dirs/* -type d -empty \( ! -iname "DIR TO IGNORE" \)
Please check out the below examples, this will list all empty folders within /home/test - excluding the ruby folder
[user#server:/home/test]# mkdir shell bash php ruby perl
[user#server:/home/test]# touch shell/testing.sh
[user#server:/home/test]# find /home/test/* -type d -empty \( ! -iname "ruby" \)
/home/test/bash
/home/test/perl
/home/test/php
OK, I've handled it by checking for each of the files recursively, and ignored those that are in the path i wanted to ignore using Java.
The terminal command is:
find /PATH/TO/FOLDER/* -type f
If anyone else can also make it skip a specific folder, that would be nice.
I would like to use the apitrace project on android. I followed the instructions from the readme file.
But get no trace where created.
I run this command
adb shell TRACE_FILE=/data/test.trace LD_PRELOAD=/data/egltrace.so am start -n APP_NAME
How can I make it work?
I tried following the instructions in Dalvik.markdown of the original distribution of apitrace, but without success.
The instructions say to set two properties: wrap._process_name_ and debug.apitrace.procname. The former has to be set, according to those instructions, to LD_PRELOAD=/data/egltrace.so. When launching the application I still wouldn't get any trace generated nor any apitrace-related message in the logcat.
I had more success by putting the LD_PRELOAD instruction in a script and using that as the wrapper. This is the script that I use, called /data/apitrace.sh:
LD_PRELOAD=/data/egltrace.so exec $#
You can also set the TRACE_FILE environment variable to specify the path to which the trace file should be written to. Otherwise it will be _/data/app_process.trace_. For example:
TRACE_FILE=/data/apitraces/mytrace.trace LD_PRELOAD=/data/egltrace.so exec $#
I believe apitrace takes care of adding numbers to the filename to prevent overwriting existing ones. So you'll have mytrace.trace, mytrace.1.trace, and so on.
So with this script in place I set the properties like so:
adb shell setprop wrap._process_name_ /data/apitrace.sh
adb shell setprop debug.apitrace.procname _process_name_
And then I launch the application. I see in the logcat something like the following:
I/dalvikvm( 5980): Exec: /system/bin/sh -c /data/apitrace.sh /system/bin/app_process /system/bin --application '--nice-name=_process_name_' com.android.internal.os.WrapperInit 25 17 'android.app.ActivityThread'
D/apitrace( 5991): apitrace: loaded
D/apitrace( 5991): apitrace[5991]: enabled for _process_name_
D/apitrace( 5991): apitrace: tracing to /data/app_process.trace
I'm using CyanogenMod 10.1.3, which is based on Android 4.2.2.
This question already has answers here:
Filter LogCat to get only the messages from My Application in Android?
(37 answers)
Closed 5 years ago.
How can I filter Android logcat output by application? I need this because when I attach a device, I can't find the output I want due to spam from other processes.
Edit: The original is below. When one Android Studio didn't exist. But if you want to filter on your entire application I would use pidcat for terminal viewing or Android Studio. Using pidcat instead of logcat then the tags don't need to be the application. You can just call it with pidcat com.your.application
You should use your own tag, look at: http://developer.android.com/reference/android/util/Log.html
Like.
Log.d("AlexeysActivity","what you want to log");
And then when you want to read the log use>
adb logcat -s AlexeysActivity
That filters out everything that doesn't use the same tag.
According to http://developer.android.com/tools/debugging/debugging-log.html:
Here's an example of a filter expression that suppresses all log messages except those with the tag "ActivityManager", at priority "Info" or above, and all log messages with tag "MyApp", with priority "Debug" or above:
adb logcat ActivityManager:I MyApp:D *:S
The final element in the above expression, *:S, sets the priority level for all tags to "silent", thus ensuring only log messages with "View" and "MyApp" are displayed.
V — Verbose (lowest priority)
D — Debug
I — Info
W — Warning
E — Error
F — Fatal
S — Silent (highest priority, on which nothing is ever printed)
Hi I got the solution by using this :
You have to execute this command from terminal. I got the result,
adb logcat | grep `adb shell ps | grep com.package | cut -c10-15`
I am working on Android Studio, there is a nice option to get the message using package name.
On the "Edit Filter Configuration" you can create a new filter by adding your package name on the "by package name".
If you could live with the fact that you log are coming from an extra terminal window, I could recommend pidcat (Take only the package name and tracks PID changes.)
Suppose your application named MyApp contains the following components.
MyActivity1
MyActivity2
MyActivity3
MyService
In order to filter the logging output from your application MyApp using logcat you would type the following.
adb logcat MyActivity1:v MyActivity2:v MyActivity3:v MyService:v *:s
However this requires you to know the TAG names for all of the components in your application rather than filtering using the application name MyApp. See logcat for specifics.
One solution to allow filtering at the application level would be to add a prefix to each of your unique TAG's.
MyAppActivity1
MyAppActivity2
MyAppActivity3
MyAppService
Now a wild card filter on the logcat output can be performed using the TAG prefix.
adb logcat | grep MyApp
The result will be the output from the entire application.
put this to applog.sh
#!/bin/sh
PACKAGE=$1
APPPID=`adb -d shell ps | grep "${PACKAGE}" | cut -c10-15 | sed -e 's/ //g'`
adb -d logcat -v long \
| tr -d '\r' | sed -e '/^\[.*\]/ {N; s/\n/ /}' | grep -v '^$' \
| grep " ${APPPID}:"
then:
applog.sh com.example.my.package
When we get some error from our application, Logcat will show session filter automatically. We can create session filter by self. Just add a new logcat filter, fill the filter name form. Then fill the by application name with your application package. (for example : my application is "Adukan" and the package is "com.adukan", so I fill by application name with application package "com.adukan")
If you use Eclipse you are able to filter by application just like it is possible with Android Studio as presented by shadmazumder.
Just go to logcat, click on Display Saved Filters view, then add new logcat filter. It will appear the following:
Then you add a name to the filter and, at by application name you specify the package of your application.
On my Windows 7 laptop, I use 'adb logcat | find "com.example.name"' to filter the system program related logcat output from the rest. The output from the logcat program is piped into the find command. Every line that contains 'com.example.name' is output to the window. The double quotes are part of the find command.
To include the output from my Log commands, I use the package name, here "com.example.name", as part of the first parameter in my Log commands like this:
Log.d("com.example.name activity1", "message");
Note: My Samsung Galaxy phone puts out a lot less program related output
than the Level 17 emulator.
I use to store it in a file:
int pid = android.os.Process.myPid();
File outputFile = new File(Environment.getExternalStorageDirectory() + "/logs/logcat.txt");
try {
String command = "logcat | grep " + pid + " > " + outputFile.getAbsolutePath();
Process p = Runtime.getRuntime().exec("su");
OutputStream os = p.getOutputStream();
os.write((command + "\n").getBytes("ASCII"));
} catch (IOException e) {
e.printStackTrace();
}
This is probably the simplest solution.
On top of a solution from Tom Mulcahy, you can further simplify it like below:
alias logcat="adb logcat | grep `adb shell ps | egrep '\bcom.your.package.name\b' | cut -c10-15`"
Usage is easy as normal alias. Just type the command in your shell:
logcat
The alias setup makes it handy. And the regex makes it robust for multi-process apps, assuming you care about the main process only.
Of coz you can set more aliases for each process as you please. Or use hegazy's solution. :)
In addition, if you want to set logging levels, it is
alias logcat-w="adb logcat *:W | grep `adb shell ps | egrep '\bcom.your.package.name\b' | cut -c10-15`"
Yes now you will get it automatically....
Update to AVD 14, where the logcat will automatic session filter
where it filter log in you specific app (package)
On the left in the logcat view you have the "Saved Filters" windows. Here you can add a new logcat filter by Application Name (for example, com.your.package)
What I usually do is have a separate filter by PID which would be the equivalent of the current session. But of course it changes every time you run the application. Not good, but it's the only way the have all the info about the app regardless of the log tag.
Generally, I do this command "adb shell ps" in prompt (allows to see processes running) and it's possible to discover aplication's pid. With this pid in hands, go to Eclipse and write pid:XXXX (XXXX is the application pid) then logs output is filtered by this application.
Or, in a easier way... in logcat view on Eclipse, search for any word related with your desired application, discover the pid, and then do a filter by pid "pid:XXXX".
you can achieve this in Eclipse logcat by entering the following to the search field.
app:com.example.myapp
com.example.myapp is the application package name.
my .bash_profile function, it may be of any use
logcat() {
if [ -z "$1" ]
then
echo "Process Id argument missing."; return
fi
pidFilter="\b$1\b"
pid=$(adb shell ps | egrep $pidFilter | cut -c10-15)
if [ -z "$pid" ]
then
echo "Process $1 is not running."; return
fi
adb logcat | grep $pid
}
alias logcat-myapp="logcat com.sample.myapp"
Usage:
$ logcat-myapp
$ logcat com.android.something.app
In Android Studio in the Android Monitor window:
1. Select the application you want to filter
2. Select "Show only selected application"
Use fully qualified class names for your log tags:
public class MyActivity extends Activity {
private static final String TAG = MyActivity.class.getName();
}
Then
Log.i(TAG, "hi");
Then use grep
adb logcat | grep com.myapp
The Android Device Monitor application available under sdk/tools/monitor has a logcat option to filter 'by Application Name' where you enter the application package name.
On Linux/Un*X/Cygwin you can get list of all tags in project (with appended :V after each) with this command (split because readability):
$ git grep 'String\s\+TAG\s*=\s*' | \
perl -ne 's/.*String\s+TAG\s*=\s*"?([^".]+).*;.*/$1:V/g && print ' | \
sort | xargs
AccelerometerListener:V ADNList:V Ashared:V AudioDialog:V BitmapUtils:V # ...
It covers tags defined both ways of defining tags:
private static final String TAG = "AudioDialog";
private static final String TAG = SipProfileDb.class.getSimpleName();
And then just use it for adb logcat.
I have found an app on the store which can show the name / process of a log.
Since Android Studio just puts a (?) on the logs being generated by the other processes, I found it useful to know which process is generating this log. But still this app is missing the filter by the process name. You can find it here.
use first parameter as your application name.
Log.d("your_Application_Name","message");
and in LogCat : create Filter ---> Filter Name & by Log Tag: is equal to 'your_Application_Name'
it will create new tab for your application.
Add your application's package in "Filter Name" by clicking on "+" button on left top corner in logcat.
to filter the logs on command line use the below script
adb logcat com.yourpackage:v
The log cat output can be filtered to only display messages from your package by using these arguments.
adb com.your.package:I *:s
Edit - I spoke to soon.
adb com.your.package:v