ropen: Remote "open" command for opening remote files locally on OS X

The Problem
———–

Most Mac OS X power users know about the [“open”](http://tuvix.apple.com/documentation/Darwin/Reference/ManPages/man1/open.1.html) command line tool which opens the files specified as arguments in their default (or a specified) OS X application. Additionally, many OS X text editors, such as TextMate (“mate”) and SubEthaEdit (“see”), come with command line tools which can be used to open files.

These are great when working locally, but obviously do no work remotely. Often when working on remote servers you end up using command line editors which you may not be as familiar with.

ropen’s Solution
—————-

The [ropen](http://github.com/tlrobinson/ropen) tool solves this problem using two simple shell scripts, which make use of MacFuse’s sshfs. You run the “ropen” program on your remote machine(s) when you want to open a remote file locally (this is equivalent to the OS X “open” command). The “ropend” daemon runs on your local OS X machine waiting for open requests, and the “ropen.php” PHP script proxies requests from ropen to ropend.

How it works
————

1. When ropen is executed it makes an HTTP request to ropen.php with the paths to be opened and application to open them with, if any, as well as the SSH user, host, and port of the remote machine.
2. ropen.php stores this open request in a queue that is tied to ROPEN_SECRET via PHP’s sessions.
3. ropend polls ropen.php every 1 second waiting for open requests. When it receives one it mounts the remote filesystem using sshfs (if it’s not already mounted) and opens the files or directories specified.

More information
————

See more information about ropen on the [ropen project page](http://github.com/tlrobinson/ropen).

Determining the absolute absolute path of a shell script

In the course of working on projects like server-side Objective-J, jack, and now narwhal, I’ve often had to write shell scripts that needed to know their location in the filesystem. Rather than hardcoding it, I prefer to infer it automatically at runtime. Unfortunately this isn’t as easy as you would expect.

If the script is invoked with an absolute path (“/foo/bar/baz”) or from your PATH (“baz”), then “$0” in the script will contain the absolute of the script (“/foo/bar/baz”). However, if it is invoked using a relative path (“./bar/baz” from “/foo”) then $0 will contain the relative path (“./bar/baz”). Furthermore, if the path to the script is actually a symbolic link, you’ll get the symlink’s path instead of the original.

Surprisingly, I couldn’t find a definitive solution that handles all these cases, so I took the various ones I did find and created one which I think handles all the cases I’m aware of:

If you don’t want to resolve the symlinks remove the second half.

Open new Terminal tab in current directory (updated!)

This is an updated shell script / AppleScript for opening a new tab in your current directory (or the specified directory). The last version was for the pre-tabbed version of Terminal.

#!/bin/sh –

if [ $# -ne 1 ]; then
    PATHDIR=`pwd`
else
    PATHDIR=$1
fi

/usr/bin/osascript <<-EOF
activate application "Terminal"
tell application "System Events"
    keystroke "t" using {command down}
end tell
tell application "Terminal"
    repeat with win in windows
        try
            if get frontmost of win is true then
                do script "cd $PATHDIR; clear" in (selected tab of win)
            end if
        end try
    end repeat
end tell
EOF

Command line interpreter and REPL for JSCocoa

A few months ago I started working on a JavaScript to Objective-C bridge. We had already implemented Objective-C in JavaScript, so I figured “why not?”

Well, I never got very far, but thankfully Patrick Geiller apparently had the same idea and actually executed it: He announced JSCocoa today. It looks like it’s a solid bridge, about up to par with PyObjC and RubyCocoa.

While the included GUI interface for trying out JSCocoa is nice, I prefer command line interfaces for my languages, so I ripped out the few lines of code from my original bridge and plugged in JSCocoa.

Code and build instructions on GitHub.

It’s very bare bones at the moment: it will either read one or more file names from the command line arguments, or if no arguments are supplied it will present a no-frills REPL. Obviously line-editing, etc would be one of the next steps, but for now it works nicely with rlwrap.

#import <Foundation/Foundation.h>
#import "JSCocoaController.h"

void JSValuePrint(JSContextRef, JSValueRef, JSValueRef *);

int main (int argc, const char * argv[])
{
    [[NSAutoreleasePool alloc] init];
    id c = [JSCocoaController sharedController];
    JSGlobalContextRef ctx = [c ctx];
    
    if (argc > 1)
    {
        for (int i = 1; i < argc; i++)
            [c evalJSFile:[NSString stringWithFormat:@"%s", argv[i]]];
    }
    else
    {
        while (1)
        {
            char buffer[1024];
            
            printf("js> ");
            
            if (fgets(buffer, 1024, stdin) == NULL)
                exit(0);
            
            JSStringRef script = JSStringCreateWithUTF8CString(buffer);
            JSValueRef exception = NULL;
            
            if (JSCheckScriptSyntax(ctx, script, 0, 0, &exception) && !exception)
            {
                JSValueRef value = JSEvaluateScript(ctx, script, 0, 0, 0, &exception);
                
                if (exception)
                    JSValuePrint(ctx, exception, NULL);
                
                if (value && !JSValueIsUndefined(ctx, value))
                    JSValuePrint(ctx, value, &exception);
            }
            else
            {
                printf("Syntax error\n");
            }
            
            JSStringRelease(script);
        }   
        
    }
}

void JSValuePrint(
                  JSContextRef ctx,
                  JSValueRef value,
                  JSValueRef *exception)
{
    JSStringRef string = JSValueToStringCopy(ctx, value, exception);
    size_t length = JSStringGetLength(string);
    
    char *buffer = malloc(length+1);
    JSStringGetUTF8CString(string, buffer, length+1);
    JSStringRelease(string);
    
    puts(buffer);
    
    free(buffer);
}

Useful Mac OS X-specific command line utilities

One of the greatest strengths of Mac OS X, for developers in particular, is that it has a very elegant and consistent graphical user interface as well as an excellent command line interface. I’m not going to cover the basics like “ls” and “cd”, but rather point out some Mac OS X specific tools that are less well known than they should be.

Many of these are the command line equivalents for the GUI versions available in OS X. Combining them with other command line tools can be very powerful and huge time savers. See the “man” pages for more details of each.

open – Opens a file, application, or directory in GUI-land. Very useful.

screencapture – Take a screenshot. Exactly the same as 3 or 4 (or even 4 then ) plus more.

say – Text to speech. Give it a file name, string of text, or pipe the output of another program to it. Options for difference voices,
saving the result to a file, etc. Fun hobby: ssh into a computer being used by someone else and start speaking to them using “say”.

pbcopy and pbpaste – Copy and paste to/from the OS X pasteboard.

srm – Secure “rm”. Like “rm” but overwrites deleted data. “-m” gives you DoD compliant erasing!

osascript – Run AppleScripts (or other OSA languages) from the command line. (I use this in the “term” script)

hdiutil – create and open disk images (.dmg)

defaults – view and set various hidden settings. Check out macosxhints.com for some of these.

These are just a few Mac OS X specific commands I found most useful. Amit Singh has a fairly comprehensive list over at his excellent kernelthread.com website.

ANSI escape sequences in JavaScript (color on the command line!)

Recently I’ve been doing a lot of work in Rhino, Mozilla’s JavaScript interpreter written in Java. While it’s a bit slower than some of the other JavaScript interpreters out there it has the advantage of being able to call Java libraries. This is immensely useful since JavaScript is pretty bare-bones in terms of built-in libraries.

Anyway, I had a simple logging system with the usual “error”, “warn”, “info”, etc levels, which was great except the important errors and warnings would get lost in the hundreds of info and debug messages. I didn’t know much about ANSI escape codes except that it allows for neat things like coloring text in your terminal, so I wrote a little code to help with this:

ANSI_ESC            = String.fromCharCode(0x1B);
ANSI_CSI            = ANSI_ESC + ‘[‘;
ANSI_TEXT_PROP      = ‘m’;
ANSI_RESET          = ‘0’;
ANSI_BOLD           = ‘1’;
ANSI_FAINT          = ‘2’; // unsupported?
ANSI_NORMAL         = ’22’;
ANSI_ITALIC         = ‘3’; // unsupported?
ANSI_UNDER          = ‘4’;
ANSI_UNDER_DBL      = ’21’; // unsupported?
ANSI_UNDER_OFF      = ’24’;
ANSI_BLINK          = ‘5’;
ANSI_BLINK_FAST     = ‘6’; // unsupported?
ANSI_BLINK_OFF      = ’25’;
ANSI_REVERSE        = ‘7’;
ANSI_POSITIVE       = ’27’;
ANSI_CONCEAL        = ‘8’;
ANSI_REVEAL         = ’28’;
ANSI_FG             = ‘3’;
ANSI_BG             = ‘4’;
ANSI_FG_INTENSE     = ‘9’;
ANSI_BG_INTENSE     = ’10’;
ANSI_BLACK          = ‘0’;
ANSI_RED            = ‘1’;
ANSI_GREEN          = ‘2’;
ANSI_YELLOW         = ‘3’;
ANSI_BLUE           = ‘4’;
ANSI_MAGENTA        = ‘5’;
ANSI_CYAN           = ‘6’;
ANSI_WHITE          = ‘7’;

ANSIControlCode = function(code, parameters)
{
    if (parameters == undefined)
        parameters = "";
    else if (typeof(parameters) == ‘object’ && (parameters instanceof Array))
        parameters = parameters.join(‘;’);
    return ANSI_CSI + String(parameters) + String(code);
}

// simple text helpers:

ANSITextApplyProperties = function(string, properties)
{
    return ANSIControlCode(ANSI_TEXT_PROP, properties) + String(string) + ANSIControlCode(ANSI_TEXT_PROP);
}

var colorCodeMap = {
    "black"   : ANSI_BLACK,
    "red"     : ANSI_RED,
    "green"   : ANSI_GREEN,
    "yellow"  : ANSI_YELLOW,
    "blue"    : ANSI_BLUE,
    "magenta" : ANSI_MAGENTA,
    "cyan"    : ANSI_CYAN,
    "white"   : ANSI_WHITE
}

ANSITextColorize = function(string, color)
{
    if (colorCodeMap[color] == undefined)
        return string;
    return ANSITextApplyProperties(string, ANSI_FG + colorCodeMap[color]);
}

Download here: ansi_sequences.js

ANSIControlCode(code, parameters) takes in a control code as a string (a single alphabetic character, see above Wikipedia article), and zero or more parameters, either as a single string/number or array of strings/numbers (again, see above Wikipedia article). It returns a string with the control sequence.

There are also two helpers functions for setting text properties:

ANSITextApplyProperties(string, properties) wraps “string” in a pair of control sequences. First the sequence to set text properties (“m” along with the given parameters), second the reset text properties sequence (“m” with no parameters). It returns the wrapped string. This lets you easily apply a number of properties to a string.

ANSITextColorize(string, color) simply sets the foreground color of a string, and returns the wrapped string.

Here are examples of the helper functions:

Note that color control codes must be combined with either ANSI_FG or ANSI_BG.

Also note that this is completely useless in the browser. Only shells can understand these control codes, so use it in Rhino and other command line JavaScript interpreters (while this code is written in JavaScript, the same concepts be applied to any other programming language that can run in a shell: C, Python, Ruby, etc).

Using command line tools to detect the most frequent words in a file

Antonio Cangiano wrote a post about “[Using Python to detect the most frequent words in a file](http://antoniocangiano.com/2008/03/18/use-python-to-detect-the-most-frequent-words-in-a-file/)”. It’s a nice summary of how to do it in Python, but (nearly) the same thing can be accomplished by stringing together a few standard command line tools.

I’m no command line ninja, but I’d like to think I have basic command of most of the standard filters. Here’s my solution:

cat test.txt | tr -s ‘[:space:]’ ‘\n’ | tr ‘[:upper:]’ ‘[:lower:]’ | sort | uniq -c | sort -n | tail -10

I’ll explain it blow-by-blow:

cat test.txt

If you don’t know what this does you’ve got a lot to learn. “cat” simply reads files and prints them to standard output (concatenates), for use by subsequent filters.

tr -s ‘[:space:]’ ‘\n’

“tr” is a handy tool that simply translates matching characters from the first set to the corresponding character of the second set. The first instance turns all whitespace characters (spaces, tabs, newlines) into newlines (“\n”) so that each word is on a separate line (the -s option “squeezes” multiple runs of newlines into a single newline).

tr ‘[:upper:]’ ‘[:lower:]’

The second instance translates all uppercase characters into lowercase (note: the two “tr”s are separate for clarity, but they could be combined into a single one).

sort | uniq -c

“sort” and “uniq” do exactly as their names imply, but “uniq” only removes adjacent duplicates, so you often want to sort the input first. The “-c” option for “uniq” prepends each line with the number of occurrences.

sort -n

We sort the result of “uniq”, this time by numerical order (“-n”) to get the list of words in order of the number of occurrences.

tail -10

Finally, we get the 10 most frequently occurring words by using “tail” to take only the last 10 lines (since the “sort -n” puts the list in ascending order)

It’s not perfect, especially since punctuation is included in the words, but the “tr” commands can be tweaked as needed.

Presenting GCCalc: a horrible abuse of GCC

Following an [interesting discussion on Reddit](http://programming.reddit.com/info/62v70/comments) about [first class functions](http://en.wikipedia.org/wiki/First-class_function) in C, I was inspired to see what I could do with this new-found knowledge. The result is what I affectionately call “GCCalc”, for reasons that will become clear below.

GCCalc is a simple command line calculator, much like the common [bc](http://en.wikipedia.org/wiki/Bc_programming_language) calculator on many Unix systems. It’s implementation, however, is *very* different than most calculators. While bc is said to have “C-like syntax”, GCCalc’s syntax *is* C. Whatever you enter on the command line automatically gets compiled, loaded, and executed, and the result is returned (as a double) and printed to the screen.

You can either enter expressions like:

round(46.95886*sqrt(1+2/9.99*sin((21%5)*pow(2,8))))

or you can enter whole C statements (as long as they’re on one line, for now) like:

int i; for (i=0;i<10;i++) { printf("hello world!\n"); } printf("goodbye\n"); Unfortunately variables are scoped to the function that wraps them, so they don't persist across multiple entries. However, you can access the last result using the "last" variable (a double). [Here's the source file](http://tlrobinson.net/projects/gccalc/gccalc.c), and here's a syntax highlighted version: It's been tested on Mac OS X (Leopard) and Linux (Ubuntu Gutsy), with GCC 4. Compile with "gcc -o gccalc gccalc.c" on OS X, or "gcc -o gccalc gccalc.c -ldl" on Linux.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <dlfcn.h>
#include <unistd.h>

#ifdef __ELF__
#define GCC_FLAGS "-fPIC -shared"
#define EXTENSION "so"
#else
#define GCC_FLAGS "-dynamiclib"
#define EXTENSION "dylib"
#endif

#define HEADERS "#include <stdio.h>\n#include<math.h>"

typedef double(func_return_double)(double);

unsigned count = 0;
char *cwd;
char tmp_path[1024] = {‘\0’};

void *lib = NULL;

int main(int argc, char **argv)
{
    double result = 0.0;
    char input_buffer[1024], code_buffer[2048], function_name[32], command_buffer[1024];
    
    // get out current directory, which we’ll use for tmp files (dlopen seems to need absolute paths)
    cwd = getcwd(NULL, 0);
    
    while (1)
    {
        // for unique function and file names (needed for dlopen/dlsym to work correctly)
        count++;
        
        // read in the next line
        printf(">> ");
        fgets(input_buffer, sizeof(input_buffer), stdin);
    
        // format the function name
        sprintf(function_name, "f%d", count);
        
        // format the code string: if it doesn’t contain a semicolon, assume it is just an expression
        if (strchr(input_buffer, ‘;’))
            sprintf(code_buffer, "%s\ndouble %s(double last) { %s\nreturn 0; }", HEADERS, function_name, input_buffer);
        else
            sprintf(code_buffer, "%s\ndouble %s(double last) { return (%s); }", HEADERS, function_name, input_buffer);
            
        // format the filename string, delete the file if it exists
        sprintf(tmp_path, "%s/libtmp%d.%s", cwd, count, EXTENSION);
        unlink(tmp_path);
        
        // format the gcc command string
        sprintf(command_buffer, "gcc -Wall %s -x c – -o %s", GCC_FLAGS, tmp_path);
        
        // execute gcc command, write out the code
        FILE *fp = popen(command_buffer, "w");
        fwrite(code_buffer, 1, strlen(code_buffer), fp);
        fprintf(fp, "\n");
    
        // pclose waits for gcc to terminate (fclose/close do NOT thus compilation will sometimes not finish prior to the dlopen)
        pclose(fp);

        void *ptr = NULL;
        
        // open the just-compiled dynamic library
        if ((lib = dlopen(tmp_path, RTLD_NOW|RTLD_LOCAL)) == NULL) {
            puts(dlerror());
        }
        // get the function pointer
        else if ((ptr = dlsym(lib, function_name)) == NULL) {
            puts(dlerror());
        }
        
        // execute it
        if (ptr != NULL)
        {
            func_return_double *func = (func_return_double*)ptr;
            result = (*func)(result);
            // print the result
            printf("=> %.*lf\n", (result/((int)result)>1.0)?5:0, result);
        }

        // clean up: close the library, delete the temp file
        dlclose(lib);
        unlink(tmp_path);
    }

    return 0;
}

Thanks to jbert on Reddit for the initial code and inspiration.

If only I had known about this back when The Daily WTF has having their [OMG WTF](http://omg.thedailywtf.com/) crazy calculator programming contest…

Open new Terminal window in current (or other specified) directory

A lot of times I find myself wanting to open another (Mac OS X) Terminal window in the same directory as my current one. This little shell script, which executes a little AppleScript, makes that trivial:

#!/bin/sh

if [ $# -ne 1 ]; then
    PATHDIR=`pwd`
else
    PATHDIR=$1
fi

osascript -e "tell application \"Terminal\"" -e "do script \"cd $PATHDIR\"" -e "end tell"

Save this script somewhere in your $PATH with executable permissions. Now instead of hitting Command-N then typing “cd really/long/path/to/your/current/directory”, you can simply type the name of the script (I used “term”):

term

Chain them together to open multiple windows. The following would open three new windows with the same current directory:

term; term; term

It can also take an optional directory path argument to override the current directory:

term /System/Library/

multiwhich

The “which” Unix command lists the location of the first matching executable in your PATH. The GNU version of “which” has several extra features including the ability to display all matching executables in your PATH, not just the first. This is useful for finding duplicates, etc. Unfortunately, whatever version of “which” is included in Mac OS X (and MacPorts) doesn’t have these extra features.

A quick Google search didn’t turn up anything, and I was in a shell scripting mood when I needed it, so rather than downloading and compiling GNU which I whipped up my own, “multiwhich”:

#!/bin/sh

for PATHDIR in `echo $PATH | tr ":" " "`
do
    sh -c "ls -1 $PATHDIR/$1" 2> /dev/null
done

Simply put this somewhere in your PATH with execute permissions, and type “which command“.

One other accidental “feature” of this script is the ability to list every executable in your PATH. This is great for finding duplicates:

multiwhich | sort | uniq -c | sort -n

It’s probably not the most elegant way to do it, but it serves it’s purpose. Perhaps someone will find it useful…

Update: I modified the multiwhich script slightly to support wildcards like “*” and “?”. You can now do things like “multiwhich x*” to get all binaries beginning with “x”, etc.