When I recently decided to sit down and properly learn the language, I was pleasantly surprised that underneath the line noise was an interesting and unique language. R is a descendant of LISP and, deep down, maintains some of the beauty its ancestor. It also borrows some unique and interesting features from other functional and dynamic languages.

Code is Data

R is true to its LISP roots in that you can create, modify, and evaluate parse trees from the code itself. One way to do so is with the quote() special-function, which returns its argument, unevaluated, as an expression object that can be traversed, modified and evaluated.

A fun (though not especially useful) consequence of this is that you can write an expression which returns itself as a quote:

> (function(x) substitute((x)(x)))(function(x) substitute((x)(x)))
(function(x) substitute((x)(x)))(function(x) substitute((x)(x)))
> expression <- (function(x) substitute((x)(x)))(function(x) substitute((x)(x)))
> expression == eval(expression)
[1] TRUE


Optional Laziness

By default, R uses eager evaluation, so expressions are evaluated as soon as they are assigned. However, R takes after functional languages like Haskell and O’Caml in that it allows lazy evaluation, where expressions are only evaluated at the time they are first used.

For example, consider the Haskell code:

m = sum [1..]


Where sum returns the sum of a list and [1..] is the (infinite) list of all natural numbers. In most languages, the assignment would cause the program to loop forever trying to sum all the natural numbers so it can assign that value to m. In Haskell, the assignment does complete; it simply assigns the expression sum [1..] to m so that it can be evaluated when the value of m is first used.

In R we can accomplish something similar with the delayedAssign() function:

delayedAssign("m", sum(1:Inf))


Note that in R, unlike O’Caml, the variables may be explicitly made lazy with delayedAssign, but are evaluated automatically when they are used.

Unfortunately, R evaluates lazy variables when they are pointed to by a data structure, even if their value is not needed at the time. This means that infinite data structures, one common application of laziness in Haskell, are not possible in R.

Operators are functions

When using higher-order functions, it’s sometimes useful to be able to treat operators as functions. Python accomplishes this in a clunky way: there is an operator module which redefines the built-in operators as functions. R takes a more functional approach. As in Haskell and O’Caml, operators are just syntactic sugar for ordinary functions. Enclosing any operator in backticks lets you use it as if it were an ordinary function. For example, calling `+`(2, 3) returns 5.

In fact, both the infix and prefix form are indistinguishable once they are parsed.

> quote(3 + 4) == quote(`+`(3, 4))
[1] TRUE


One surprising fact in R is that the assignment operators (<-, <<- and =) are functions like any other. As a result, they can be overwritten or passed around as desired, though neither strikes me as a particularly good idea.

Continuations

Continuations in R are a way of “breaking out” of a computation and jumping down the call stack to return early. The R function callCC() (call with current continuation) takes one argument, a function. It then evaluates that function, passing in a special function as an argument. callCC() then returns the first value that the special function is called with, or the return value of evaluating its argument if the special function is not called before the function returns.

To give you a better idea of what that looks like, consider this example:

> callCC(function(m) {return(4)})
[1] 4
> callCC(function(m) {m(2); return(4)})
[1] 2


Calling the function m(2) essentially cuts the computation short, drops down in the call stack to callCC, and returns 2.

If you’ve used continuations in another language, note that in R the exit function can only be called before callCC() returns. This makes R’s continuation semantics less powerful than those of languages like Scheme, Smalltalk, and Ruby.

R is not without its flaws and legacy baggage (you can trace its roots back to the S programming language 35 years ago), but once you learn to use it right, it’s a very powerful and indispensable language.

Here is the function:

# Number of times to indent output
# A list is used to force access by reference
__report_indent = [0]

def report(fn):
    """Decorator to print information about a function
    call for use while debugging.
    Prints function name, arguments, and call number
    when the function is called. Prints this information
    again along with the return value when the function
    returns.
    """

    def wrap(*params,**kwargs):
        call = wrap.callcount = wrap.callcount + 1

        indent = ' ' * __report_indent[0]
        fc = "%s(%s)" % (fn.__name__, ', '.join(
            [a.__repr__() for a in params] +
            ["%s = %s" % (a, repr(b)) for a,b in kwargs.items()]
        ))

        print "%s%s called [#%s]"
            % (indent, fc, call)
        __report_indent[0] += 1
        ret = fn(*params,**kwargs)
        __report_indent[0] -= 1
        print "%s%s returned %s [#%s]"
            % (indent, fc, repr(ret), call)

        return ret
    wrap.callcount = 0
    return wrap

The function can be used as a decorator. For example, in this simple (and inefficient) recursive Fibonacci sequence function:

@report
def fibonacci(n):
    if n in [0,1]:
        return n
    else:
        return fibonacci(n - 1) + fibonacci(n - 2)

The result:

>>> fibonacci(4)
fibonacci(4) called [#1]
 fibonacci(3) called [#2]
  fibonacci(2) called [#3]
   fibonacci(1) called [#4]
   fibonacci(1) returned 1 [#4]
   fibonacci(0) called [#5]
   fibonacci(0) returned 0 [#5]
  fibonacci(2) returned 1 [#3]
  fibonacci(1) called [#6]
  fibonacci(1) returned 1 [#6]
 fibonacci(3) returned 2 [#2]
 fibonacci(2) called [#7]
  fibonacci(1) called [#8]
  fibonacci(1) returned 1 [#8]
  fibonacci(0) called [#9]
  fibonacci(0) returned 0 [#9]
 fibonacci(2) returned 1 [#7]
fibonacci(4) returned 3 [#1]
3

The level of indent reflects the level of recursion, and the [#...] at the end of each line is the number of times the function has been called.

The level of indent is independent of the function being called, so it is helpful with mutual recursion as well. For example, when used with the functions even and odd from my earlier post on tail recursion, the result looks like this:

>>> even(5)
even(5) called [#1]
 odd(4) called [#1]
  even(3) called [#2]
   odd(2) called [#2]
    even(1) called [#3]
     odd(0) called [#3]
     odd(0) returned False [#3]
    even(1) returned False [#3]
   odd(2) returned False [#2]
  even(3) returned False [#2]
 odd(4) returned False [#1]
even(5) returned False [#1]
False

I find it useful to stick @report before the function I am having trouble with, and use comments to turn it on and off while I’m debugging that function. It can also be used at times other than function declaration, for example: report(base64.encodestring)(‘test’).

Update (July 6, 2008): Fixed so that keyword arguments are printed as well.

Update (August 16, 2008): Changed .__repr__() to the more proper repr().

There are a few performance improvements as well. The PHP version creates an array in memory proportional to the square of the size of the input, while the Python version’s array is directly proportional to the size of the input. I also sped up how the algorithm finds the indexes of the “new” elements in the “old” array.

Download simplediff.py

¹ It is probably the same algorithm that others use, but I haven’t gotten around to getting an ACM membership to access the related papers

For example, consider the mutually recursive functions even and odd. You know a number, n, is even if it is 0, or if n – 1 is odd. Similarly, you know a number is not odd if it is 0, and that it is odd if n – 1 is even. This translates to the python code:

def even(x):
  if x == 0:
    return True
  else:
    return odd(x - 1)

def odd(x):
  if x == 0:
    return False
  else:
    return even(x - 1)

This code works, but only for x < 1000, because Python limits the recursion depth to 1000. As it turns out, it is easy to get around this limitation. Included below is a generic tail_rec function that could be used for most cases where you need tail recursion, and an example of it used for the odd/even problem.

def tail_rec(fun):
   def tail(fun):
      a = fun
      while callable(a):
         a = a()
      return a
   return (lambda x: tail(fun(x)))

def tail_even(x):
  if x == 0:
    return True
  else:
    return (lambda: tail_odd(x - 1))

def tail_odd(x):
  if x == 0:
    return False
  else:
    return (lambda: tail_even(x - 1))

even = tail_rec(tail_even)
odd = tail_rec(tail_odd)

It’s not as pretty as the Scheme version, but it does the trick. Of course, the odd/even functions are just for the sake of a simple example and have no real-world use, but the tail_rec function could be used in practice.

April 2009 Update: this article has recently had some popularity. One of the more common comments is that tail_rec could be used as a decorator. In fact, this isn’t true, because even and odd need access to the raw, undecorated versions of each other in the creation of the lambda.

The complex houses married and single students and their families

In this case, most readers would probably think complex was an adjective that modified the plural noun houses. The post ended with a challenge – how easy would it be to create a program to automatically generate these sentences. Since school is out and I have some free time, I tried it myself. I found a decent free xml dictionary, and wrote a Ruby script to parse the important bits (the type of word and alternate forms) into an SQL database. I cross-checked all the words against a word frequency table to make sure there were no obscure words. I then wrote a Python script to put the words together into a (hopefully meaningful, but not often) sentence. I put the Python script onto my server so you can play with it here April 2009 Update: I removed the live demo as part of a server move.

As you can see, the sentences that it comes up with are far from meaningful. However, in most cases you can at least see how a reader could be taken down the wrong path (at least in the cases where there is a right path). In the above example, concrete could be an adjective or a noun, and spheres could be a noun or a verb (to form a sphere). Foster could be an adjective or a noun depending on the context, but I couldn’t see the reader seeing it as an adjective here. Certainly the sentence generator leaves a lot to be desired (especially considering that this was one of the better sentences), but I got about as far with it as I expected to. I think it could be improved further with a few modifications:

• Words in the database are already cross-checked to make sure they aren’t obscure, but often a word will be common as a noun and uncommon as a verb, or vice versa. I didn’t have a dataset that allowed me to determine if this was the case for a particular word.
• The valency of verbs is ignored. All verbs are assumed to be transitive, even though valency information is available in the database.
• I underestimated the difficulty of having a computer generate a meaningful sentence. It is difficult to determine what verbs are compatible with what nouns, I guess you would need to parse a large amount of English text (perhaps some of Project Gutenberg – I think Wikipedia would not be varied enough but I could be wrong).

I noticed later that Ero Carrera had taken a similar approach to what I did, but with his linguistics experience he better anticipated the problems I ran into. He has some good ideas, and his post is an interesting read.