Why is bash-complete so slow!

So, I've been using home-brew for a little while on my macbook, and I noticed that after I installed a few things with it, my tab completion was REALLY slow.... A little poking around was in order.

First I looked in my ~/.bash_profile, and I saw the following

if [ -f $(brew --prefix)/etc/bash_completion ]; then
  . $(brew --prefix)/etc/bash_completion
fi

That pointed me in the direction I needed to go... Lets run the subtle command to see where it points!

MacBook-Pro-2:puppet-sandbox dawiest$ brew --prefix
/usr/local

So lets see how many files (and lines) are contained in those files..

MacBook-Pro-2:bash_completion.d dawiest$ ls /usr/local/etc/bash_completion.d/ | wc -l
  186
MacBook-Pro-2:puppet-sandbox dawiest$ find /usr/local/etc/bash_completion{,.d/*} |xargs  wc -l | grep total
wc: /usr/local/etc/bash_completion.d/helpers: read: Is a directory
   25412 total

Holy smokes!  186 files, with 25k lines!  No wonder it takes so long!

After I opened up the list of files in the bash_completion.d/ directory, removed the ones I didn't care about (they are all just symlinks pointing back to ../../Cellar/bash-completion/1.3/etc/bash_completion.d/), my auto-complete's in my shell are noticeably faster! 

MacBook-Pro-2:puppet-sandbox dawiest$ ls /usr/local/etc/bash_completion.d/ | wc -l
      50
MacBook-Pro-2:puppet-sandbox dawiest$ find /usr/local/etc/bash_completion{,.d/*} |xargs  wc -l | grep total
wc: /usr/local/etc/bash_completion.d/helpers: read: Is a directory
   14115 total

50 files, and 14k lines to execute, but I think I have removed the biggest offenders.  I should figure out a way to profile each of the scripts to see how long they each take to source!  It may be better to find the biggest offenders, and remove them if they aren't part of your usual command set.  

TiL - remember how to shell quote!

Just the other day, I was doing something over and over.... trying to kill a stubborn tomcat server that wouldn't gracefully die.  sudo service tomcat stop just wouldn't cut it.

I was getting tired of repeatedly typing 'ps aux | grep tomcat', copying out the Process id, and passing it to kill -9.   Time for some simple automation... an alias!

So, how to find the Process?  First, we use ps to display a list of running processes, and the grep utility to select only the lines that contain tomcat.

vagrant@client1:~$ ps aux | grep tomcat

tomcat7   6355  0.9 26.2 1050336 63972 ?       Sl   00:37   0:04 /usr/lib/jvm/default-java/bin/java -Djava.util.logging.config.file=/var/lib/tomcat7/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -Xmx128m -XX:+UseConcMarkSweepGC -Djava.endorsed.dirs=/usr/share/tomcat7/endorsed -classpath /usr/share/tomcat7/bin/bootstrap.jar:/usr/share/tomcat7/bin/tomcat-juli.jar -Dcatalina.base=/var/lib/tomcat7 -Dcatalina.home=/usr/share/tomcat7 -Djava.io.tmpdir=/tmp/tomcat7-tomcat7-tmp org.apache.catalina.startup.Bootstrap start

vagrant   7504  0.0  0.3  10460   940 pts/0    S+   00:46   0:00 grep --color=autotomcat

I needed to make sure I was only grabbing the tomcat process.  Since I only have the tomcat server being run by the tomcat user, and the output of PS starts with the user who is running the process, I could use a regular expression that matches from the beginning of the line to the word tomcat.  To make sure I can use the beginning of line char (^) in my regex, I use the '-E, --extended-regexp' argument.

My command looks something like this

vagrant@client1:~$ ps aux | grep -E '^tomcat'

tomcat7   6355  0.7 25.6 1050336 62424 ?       Sl   00:37   0:05 /usr/lib/jvm/default-java/bin/java -Djava.util.logging.config.file=/var/lib/tomcat7/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -Xmx128m -XX:+UseConcMarkSweepGC -Djava.endorsed.dirs=/usr/share/tomcat7/endorsed -classpath /usr/share/tomcat7/bin/bootstrap.jar:/usr/share/tomcat7/bin/tomcat-juli.jar -Dcatalina.base=/var/lib/tomcat7 -Dcatalina.home=/usr/share/tomcat7 -Djava.io.tmpdir=/tmp/tomcat7-tomcat7-tmp org.apache.catalina.startup.Bootstrap start

I then need to pull out the second record, which contains the process ID.  For that, I use awk.   It simply takes the output and breaks it into 'records', by default it uses whitespace.  Using 'print $2' prints out the second record.

Here is the updated command.

vagrant@client1:~$ ps aux | grep -E '^tomcat' | awk '{ print $2 }'

6355

Most important note about the above command.... I initially forgot to use the single quote ('), and I accidentally used the double quote (").  What happens in that case is that your shell will try to do a variable substation, and replace $2 with the variable 2 ( which is usually used when you are passing arguments to a shell script, $0 is the name of the script, $1 is the first argument, $2 is the second, etc...).  Since there was nothing in $2, it turned the awk command into 'print', which simply prints the entire line.

Here is the command without output using the wrong quotes.

vagrant@client1:~$ ps aux | grep -E '^tomcat' | awk "{ print $2 }"

tomcat7   6355  0.6 25.6 1050336 62424 ?       Sl   00:37   0:05 /usr/lib/jvm/default-java/bin/java -Djava.util.logging.config.file=/var/lib/tomcat7/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -Xmx128m -XX:+UseConcMarkSweepGC -Djava.endorsed.dirs=/usr/share/tomcat7/endorsed -classpath /usr/share/tomcat7/bin/bootstrap.jar:/usr/share/tomcat7/bin/tomcat-juli.jar -Dcatalina.base=/var/lib/tomcat7 -Dcatalina.home=/usr/share/tomcat7 -Djava.io.tmpdir=/tmp/tomcat7-tomcat7-tmp org.apache.catalina.startup.Bootstrap start

Now, that will return only the PID... but what to do to it?  I need to pass it as an argument to the kill command, so I'll run it in a subshell by putting it between two backpacks (`).  I use the -9 argument to force kill to kill it totally.  Remember to use sudo if you aren't running as the tomcat user!

My full command now looks like this

vagrant@client1:~$ sudo kill -9 `ps aux | grep -E '^tomcat' | awk '{ print $2 }'`

When successful, this command has no output.

Remember to properly quote your shell variables, happy command line fu everyone!

Arduino on OS X

Hi everybody!

Quick little blurb today..

Today I installed Arduino using homebrew/cask on my laptop, and was running into a little issue...

chmod: Unable to change file mode on /opt/homebrew-cask/Caskroom/arduino/1.6.5-r5/Arduino.app/Contents/Java/hardware/tools/avr/bin/avrdude_bin: Operation not permitted

Part of my issue was that I have an unusual homebrew install (installed with my 'admin' user, but my day to day user is not an admin).

I ended up following the steps outlined on this stackexchange post, and it didn't actually solve my problem, but it got my homebrew setup a lot happier for my non-admin user.

Then, I did the last thing any good programmer will do, and actually read what the error message was telling me!  The arduino process was trying to chmod a file, which meant that it's mode must be incorrect...

bash-3.2$ ls -la /opt/homebrew-cask/Caskroom/arduino/1.6.5-r5/Arduino.app/Contents/Java/hardware/tools/avr/bin/avrdude_bin 

-rw-rw-r--  1 admin  admin  369784 Apr 14 16:30 /opt/homebrew-cask/Caskroom/arduino/1.6.5-r5/Arduino.app/Contents/Java/hardware/tools/avr/bin/avrdude_bin

I did an inspection of all the files in the avr/bin/ directory, and none of them had owner OR group execute permissions!

A quick little command fixed up my issue!

bash-3.2$ chmod g+x /opt/homebrew-cask/Caskroom/arduino/1.6.5-r5/Arduino.app/Contents/Java/hardware/tools/avr/bin/*

I can now load sketches to my Arduino, and connect to the serial port!

Thanks for reading everyone!  I hope this helps someone with the same problem!