Notes

Using xargs to feed multiple CPUs

One little-known feature of xargs is that it can run multiple jobs in parallel via the -P or --max-procs argument.

via www.tummy.com

Recommended Basic Textmate Settings

This is on Textmate 1.5.9 on Mac OS 10.6 (Snow Leopard). These are just basic tweaks to make the editor slightly more useful and to avoid mixing tabs and spaces in code.

Tabs and Spaces

Please do this! Mixed tabs and spaces in code is irritating and screws up diffs and code display.

In the bottom of the document window click Tab Size: and select 4.

In the bottom of the document window click Tab Size: and select Soft Tabs (Spaces).

It should now read Soft Tabs: 4.

Text Display Preferences

From the Textmate menu select Preferences.

Click the General tab.

Select Highlight current line.

Select Show right margin indicator.

Click the Fonts & Colors tab.

I like the Slush & Poppies theme, but that’s just a personal preference.

If you're on Mac OS 10.6 (Snow Leopard) click Select… next to Font and choose Menlo, Regular, 12 pt. which is a much nicer fixed width font.

View Preferences

Line numbers make code navigation easier:

From the View menu select Gutter -> Line Numbers.

I also like to show invisibles:

From the View menu select Show Invisibles.

Perltidy and PBP

Recently I've been cleaning up some Perl code for release. I've been using perltidy as part of that process. Here's what I came up with for perltidy flags, a minor variation on those suggested by Perl Best Practices.

The original Perl Best Practices flags for perltidy combined with the PBP errata notes results in:

perltidy -l=78 -i=4 -ci=4 -st -se -vt=2 -cti=0 -pt=1 -bt=1 -sbt=1 -bbt=1 -nsfs -nolq -wbb="% + - * / x != == >= <= =~ < > | & **= += *= &= <<= &&= -="

Note that this is different than perltidy's -pbp flag! perltidy doesn't take into account the typo in the -wbb setting in PBP, which is corrected in the errata.

I removed the standard output and standard error output flags, bumped up the line length from 78 to 115, and adjusted the vertical tightness to always break a line after an opening token. The result:

perltidy -l=115 -i=4 -ci=4 -vt=0 -cti=0 -pt=1 -bt=1 -sbt=1 -bbt=1 -nsfs -nolq -wbb="% + - * / x != == >= <= =~ < > | & **= += *= &= <<= &&= -="

In summary, perltidy is a land of contrasts:

-l=115  # Max line length is 115 cols
-i=4   # Indent level is 4 cols
-ci=4  # Continuation indent is 4 cols
-vt=0  # Always break a line after opening token 
-cti=0 # No extra indentation for closing brackets
-pt=1  # Medium parenthesis tightness
-bt=1  # Medium brace tightness
-sbt=1 # Medium square bracket tightness
-bbt=1 # Medium block brace tightness
-nsfs  # No space before semicolons
-nolq  # Don't outdent long quoted strings
-wbb="% + - * / x != == >= <= =~ < > | & **= += *= &= <<= &&= -="
       # Break before operators

Option-arrow cursor movement in Terminal

These instructions from Mac OS X Hints still work fine.

Screenshots below for Terminal in Mac OS X Snow Leopard (10.6).

Open Preferences.

Click the Settings tab.

Click the Keyboard tab.

Make sure that Use option as meta key is selected. This causes option-b to move the cursor back on word boundaries and option-f to move the cursor forward on word boundaries.

Then, click the + button.

Add a mapping to enable option-← to move the cursor back on word boundaries.

• Key: cursor left
• Modifier: option
• Action: send string to shell:
• \033b

Click OK. Click + again.

Add a mapping to enable option-→ to move the cursor forward on word boundaries.

• Key: cursor right
• Modifier: option
• Action: send string to shell:
• \033f

Click OK.

Done!

Python Development and Library Management on Ubuntu

Apologies in advance for the Ubuntu-centricity of this post title. I haven't had a chance to try this out on Debian, so I can't be sure that it works quite as smoothly, although I assume it does. All of this was done on Ubuntu 9.10 (Karmic Koala).

First, install python, the python development headers and libraries, and python's setuptools (setuptools includes easy_install). easy_install allows you to install python libraries that are not available via the Ubuntu package system.

sudo aptitude install python python-all python-dev python-all-dev python-setuptools

Second, use easy_install to install pip, which is superior to easy_install for package and library management. We use easy_install to get the latest version of pip, which is not available in the Ubuntu packages. Since it's not installed with aptitude, pip will end up under /usr/local as it should.

sudo easy_install pip

Finally, use pip to install virtualenv, which allows you to build out environments containing specific versions of python libraries for specific applications and development environments.

sudo pip install virtualenv

At this point I don't have hard and fast rules for which method I use to install python dependencies, but roughly it's:

• Use aptitude and the Ubuntu sources for stable packages and tools, or packages and tools which are linked to other functionality and libraries better handled with the system package management. e.g. I'd install MySQL database support for python with the python-mysqldb package.

• Use pip to install to /usr/local for tools which are useful system wide and are either out of date or not available in the Ubuntu sources. e.g. pep8.

• Use pip to install to a specific virtualenv for a specific application development. e.g. if I started a Django project which had some specific dependencies I'd set up a virtualenv for that project and install the libraries it depended on there.

For more on why you might want to use these tools for python development:

• Tools of the Modern Python Hacker: Virtualenv, Fabric and Pip

• A primer on virtualenv

And thanks to Andrew who (unbeknownst to him) was the trigger for me writing this post. He also took the time to walk me through pip and virtualenv a few weeks ago.

Date One Liner

Say it was a Thursday and you wanted to print out today's date as well as the date of every Thursday for the next year. You could do it like this:

NUM=52; for ((POS=0; POS<$NUM ; POS++)); do echo `date -v +${POS}w +"%Y-%m-%d"`; echo; done

DEBEMAIL and DEBFULLNAME

Note to self: Set the DEBEMAIL and DEBFULLNAME environment variables when building or rebuilding custom/personal Debian .deb packages. Like so for (ba)sh:

export DEBEMAIL="my@emailaddress.com"
export DEBFULLNAME="Full Name"

Terminal.app font in Snow Leopard

Before upgrading to Snow Leopard I used Deja Vu Sans Mono (a variant of Bitstream Vera Sans Mono) as my Terminal font. I now prefer Menlo, Apple's own variant of Vera Sans Mono, which ships with Mac OS X 10.6. I use 12 pt. Menlo in Terminal and when programming and editing code. A comparison of Menlo and Deja Vu Sans Mono is below (via Jesse Burgheimer on Typophile).

See also: Terminal.app colors in Snow Leopard

Build a binary .deb package without signing

Building a Debian (or Ubuntu) .deb package for your own use and want to skip signing, etc.? Quoth the debuild(1) man page:

The typical command line options to build only the binary package(s) without signing the .changes file (or the non-existent .dsc file):

debuild -i -us -uc -b

Change the "-b" to "-S" to build only a source package.

Terminal.app colors in Snow Leopard

I’ve had the new version of Mac OS X — 10.6 aka Snow Leopard —  installed for a while, but I’ve only just gotten around to tweaking the configuration of Terminal.app.

Previously: Backspace/Delete/Color in the Terminal from June 2008 covers the process I went through to configure Terminal.app for Mac OS X Leopard (10.5).

Background: Fixing colors in Terminal.app on 10.6 contains all the relevant information on getting SIMBL and TerminalColors installed for Terminal.app, but only if you read the entire post and the entire comment thread which tracks the various updates to SIMBL itself and the 32-bit/64-bit enabled versions of the relevant applications and bundles.

Solution: SIMBL now works on Snow Leopard with both 32-bit and 64-bit applications. The Visor bundle (“a system-wide terminal accessible via a hot-key” which started its life at blacktree.com) now contains both TerminalColors and CopyOnSelect and is under active development. And it's handy to have a shell quickly accessible when I need one.

• Download SIMBL (the latest 0.9.x version) and run the package installer.
• mkdir -p ~/Library/Application\ Support/SIMBL/Plugins
• Download Visor. Unzip Visor.
• mv Visor.bundle ~/Library/Application\ Support/SIMBL/Plugins
• Quit Terminal. Launch Terminal.
• Choose Terminal > Preferences. Click Visor. Deselect the checkbox labeled Show on Reopen. This will stop the system-wide terminal from being displayed when Terminal is brought into the foreground.

I still use this terminal theme as a starting point for my own custom color scheme.

If you just want to tweak colors sans Visor, try the version of TerminalColours timmfin released. (The download link is in the README on that page.) It includes Snow Leopard fixes, 32-bit and 64-bit support, and works with SIMBL 0.9.x.

Another option is Yoshimasa Niwa's TerminalColoreopard. The 0.2.5 beta works with Snow Leopard, 64-bit, and SIMBL 0.9.x.

Apple Publications Style Guide 2008

“The Apple Publications Style Guide provides editorial guidelines for text in Apple instructional publications, technical documentation, reference information, training programs, and the software user interface.”

Via the Mac OS X Reference Library.

Running Devel::NYTProf under FastCGI under lighty

Here's what I put in my lighttpd.conf:

$HTTP["url"] =~ "^/hello.fcgi" {
  fastcgi.server += ( ".fcgi" => (
    "hello" => (
      "socket" => "/tmp/hello-fcgi.socket",
      "bin-path" => "/var/www/hello.fcgi",
      "max-procs" => 1,
      "bin-environment" => ( 
        "PERL5OPT" => "-d:NYTProf",
        "NYTPROF" => "addpid=1:trace=2:file=/tmp/nytprof.out:",
      ),
    )
  ) )
}

The relevant parts are:

"bin-environment" => ( 
  "PERL5OPT" => "-d:NYTProf",
  "NYTPROF" => "addpid=1:trace=2:file=/tmp/nytprof.out:",
),

Thanks to this use.perl.org post for the tip to include the trailing colon in NYTPROF.

Blinking Cursor in Gnome Terminal

Well, at least you can turn it off again.

Wandering Thoughts:

In the modern version of gnome-terminal, there is a hidden gconf setting that controls this if you don't want to turn off the Gnome-wide setting. Fire up gconf-editor, navigate to apps/gnome-terminal/profiles/Default, and set the cursor_blink_mode key to have the value of 'off'. If you have any other profiles, you will have to create and set this key in them as well.

Cables

hhgregg: DOA

Recession-busting pro tip: buy your $10 cables for $10 at Monoprice, buy your electronics at NewEgg, and don’t buy a $1700 washing machine from anyone.

I too was happy with my recent cable purchases from Monoprice.

Default Permissions with SFTP in Transmit

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

 

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

 

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

 

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

UTF 8 and my.cnf

Settings for mysql that will help ensure Unicode top to bottom.

[mysqld]
default-character-set=utf8
default-collation=utf8_general_ci
character-set-server=utf8
collation-server=utf8_general_ci
init-connect='SET NAMES utf8'
 
[client]
default-character-set=utf8

Safely Dumping an InnoDB Database

mysqldump -R -q --single-transaction db | gzip -q -9 - > db-`date +%Y-%m-%d-%H%M%S`.sql.gz

-R Dump stored routines (functions and procedures) from the dumped databases. (Reference.)

-q This option is useful for dumping large tables. It forces mysqldump to retrieve rows for a table from the server a row at a time rather than retrieving the entire row set and buffering it in memory before writing it out. (Reference.)

--single-transaction This option issues a BEGIN SQL statement before dumping data from the server. It is useful only with transactional tables such as InnoDB and BDB, because then it dumps the consistent state of the database at the time when BEGIN was issued without blocking any applications. (Reference.)

Piping across ssh

This is a reminder to myself that not only is it possible to pipe across ssh it makes it easy to do things such as add your public key to the authorized keys file on a remote server.

ssh server 'mkdir -p .ssh && chmod 700 .ssh'

cat ~/.ssh/id_dsa.pub | ssh server 'cat >> .ssh/authorized_keys'

ssh server chmod 600 .ssh/authorized_keys

Quick and Dirty Timestamp

A useful invocation of the date command for adding a timestamp into eg the filenames of tarballs and database dumps is:

date +%Y-%m-%d-%H%M%S

Throw it into a one line shell script in your ~/bin or just set up a "timestamp" alias in your dotfiles. Then it's easy to put in backticks like so:

tar cvzf archive-`timestamp`.tar.gz archive/

or

mysqldump -u root -p database | gzip -9 > database-`timestamp`.tar.gz

This results in files with names such as:

archive-2008-07-16-232750.tar.gz

PostgreSQL Dumps

I have found that the most convenient way to dump a single PostgreSQL database so that it does not contain any user information, grant statements, etc. is like so:

pg_dump -c -O -x -U user dbname

This will send the database to stdout in the textual SQL format. Pipe through gzip and into a file to save the dump:

pg_dump -c -O -x -U user dbname | gzip > dumpname.sql.gz

Explanation of the relevant flags from the pg_dump manpage:

-c
--clean
Output commands to clean (drop) database objects prior to (the commands for) creating them.

-O
--no-owner
Do not output commands to set ownership of objects to match the original database. By default, pg_dump issues ALTER OWNER or SET SESSION AUTHORIZATION statements to set ownership of created database objects. These statements will fail when the script is run unless it is started by a superuser (or the same user that owns all of the objects in the script). To make a script that can be restored by any user, but will give that user ownership of all the objects, specify -O.

-x
--no-privileges
--no-acl
Prevent dumping of access privileges (grant/revoke commands).

Backspace/Delete/Color in the Terminal

Update (12/2009): Terminal.app colors in Snow Leopard

Original post (6/2008): The Mac OS X Terminal gets a little better with each release of the OS, and the version in 10.5 (Leopard) was no exception. But there are still a few little problems. Like the fact that by default the backspace and delete keys don't work properly and consistently when sshing to machines running different unix variants. And problems with color working consistently with apps that use ncurses. All of this hearkens back to some sort of inconsistent/buggy implementation of the turtles-all-the-way-down unix terminal emulation that underlies it all. There isn't really a "solution" to this problem in the traditional sense, but you can get close enough if you (kind of) understand the issues at play. Some advice from over the years:

• 2004: grant: backspace/delete
• 2004: evan_tech -- Debian, Terminal.app, backspace, and color
• 2006: Setting up the Mac OS X 10.4 Terminal.app
• 2007: Fixing the delete key in Terminal.app
• 2008: OS X Terminal Emulation Woes

I ultimately just took the advice of the last article and set my terminal to identify itself as dtterm in the Advanced tab of the Settings pane in the preferences. This (mostly) did the trick. I also selected Delete sends Ctrl-H in the Advanced tab and Use option as meta key in the Keyboard tab. This makes backspace more consistent a and allows me to eg use Meta-B and Meta-F to move backward and forward on word boundaries in the shell.

Another helpful tool to get a more readable black Terminal theme is this SIMBL hack which allows customization of ANSI colors. After installing it I used this terminal theme as a starting point for my own theme customization.