I must give the HBR staff a lot of credit for the final product — they made a visualization designed to be interactive work in print. I also created an interactive version for their website.

The WSJ article came just as I was finishing up a visualization of some data I had collected on Groupon deals, which gives perspective on that massive number in terms of the individual deals.

Each box is a deal. I used height to represent number sold, and width to represent the price. Area is therefore gross revenue, and colour is city for the top 20 cities.

(Click for a larger, interactive version. Only works in browsers that support SVG, i.e. not IE)

The 2D-bin-packing was implemented in R and C++, based on code by mackstann. Thanks to my friends Eric and Lisa for feedback on a draft of the visualization.

I’ve received a lot comments about the image, many asking for more details on how I created it. When I tell people I used R, the reaction I get is roughly what I would expect if I told them I made it with a Microsoft Paint and a bottle of Jägermeister. Some people even questioned whether it was actually done in R. The truth is, aside from the addition of the logo and date text, the image was produced entirely with about 150 lines of R code with no external dependencies. In the process I learned a few things about creating nice-looking graphs in R.

Transparency and Faking It

My first attempt at plotting the data involved plotting very transparent lines. Unfortunately there was just too much data to get a meaningful plot — even at very low opacity, there were enough lines to make the entire image just a bright blob. When I increased the transparency more, the opacity was rounded down to zero by my graphics device and the result was that nothing was drawn.

The solution was to manipulate the drawing order of the lines. I used a simple loop over my data to draw the lines, so it was easy to control which lines are drawn first using order(). I created an ordering based on the length of the lines, so that longer lines were drawn “behind” the shorter, more local lines. Then I used colorRampPalette() to generate a color palette from black to blue to white, and colored the lines according to order they were drawn.

Great Circles

I wrote my own code to draw the great circle arcs, although I later found a CRAN package called geosphere that would have done it for me (albeit with rougher lines near the poles). I drew the great circle arcs in a way that was easy to derive but slow to compute. I bisected the lines recursively, finding their great circle midpoint, until they were short enough to resemble an arc. To find the great circle midpoint, I converted from spherical coordinates to Cartesian, found the midpoint, then converted back to spherical coordinates and extended the radius.

Euclidean Distance

Several observent commenters called me out on using Euclidean distance on the projection for the ordering function. Having the ordering function depend on the distance on the projection seems counterintuitive, as Eucliden distance is wildly distorted near the poles. I accepted this drawback because the exact drawing order wasn’t important, as long as very long lines were drawn below very short ones.

Due to the nature of the conference, I did my best to make things as accessible to someone with a general mathematics background rather than assuming familiarity with databases or order notation.

I’ve put the slides (pdf link, embedded below also) online. They may be hard to follow as slides, but I hope they pique your interest enough to check out the papers referenced at the end if that’s the sort of thing that interests you. I may turn them into a blog post at some point. The presentation begins with tabular data and shows some of the insights that led to the Dynamic Data Cube, which is a clever data structure for answering range-sum queries.

I’ll admit it: my résumé doesn’t stand out. I’ve had some great internships, but also a tendency to work for companies that aren’t (yet!) household names. And though I’m doing fine academically, it’s not well enough to stand out on my marks alone.

On the other hand, my blog lets me stand out. I’ve had a few opportunities to meet and interview with some great people and companies because they read my blog. Naturally, then, the primary goal of my résumé is to get people to visit my blog. Since I don’t quite have the audacity to make my résumé a note telling people to visit my blog, I’m faced with the problem of how to optimize my résumé to ensure people see my blog. That’s where this experiment comes in.

Methodology

I started thinking about variables in my résumé that could affect the rate at which people viewed my blog. I narrowed it down to three that I could easily test.

The first is the length of the résumé. My friends Rajesh and Shams are adamant about keeping their résumés down to a single page. Their arguments are sound, but I wanted to see if the data would back up their beliefs. I created a “short” version of my résumé which I squeezed into one page by omitting the Awards section and removing some skills.

Second, I wanted to know how my grades affected the résumé. Obviously I couldn’t start making things up, but since my major average differs from my overall average by a good margin, I had two numbers that I could use truthfully with a subtle change in wording.

Résumé variations with different grades

Résumé variation with a link to GitHub

I wanted to track three things: how many people downloaded the résumé, how many people scrolled to the bottom of the landing page, and how many people visited my blog. The first I accomplished by logging downloads. The second I accomplished with jQuery and an Ajax callback. The third I accomplished with a tracking image, just like hit counters in the 90s. I used IP address and cookies to match up actions with the associated résumé.

The only remaining problem was how to get hundreds of people to see my résumé in a short period of time. Fortunately Google offered me $110 in AdWords credits as a Google Analytics user, so I took advantage of that and ran ads on Google searches. Here is one of the half-dozen variations I ran:

One of the Google ads I ran

Results

After less than a week, I managed to exhaust my AdWords budget and gather a fair bit of data. I wrote a few hadoop jobs with my new toy and then brought the data into R for analysis and visualization.

Length

As you might expect, the people who encountered the short resume were much more likely to scroll to the bottom. Just over half did, versus just under a third of those presented with the long resume. This makes sense because there is less to scroll through, but it was nice to have the data confirm my suspicions. Note that in the following graph, and all others in this post, the grey lines indicate the 90% confidence interval.

The short résumé also resulted in more downloads and blog views, but not enough to be statistically significant with the amount of data I collected.

Grades

The grades shown on the résumé didn’t affect any of the metrics I was measuring in a statistically significant way.

Links

I was surprised to find that the non-blog link shown on my résumé affected the frequency of click-throughs to my blog. Even adding a link to my GitHub profile more than halved the frequency of a clickthrough to my blog. LinkedIn and twitter were even worse.

I created a heatmap-like visualization from the relative significances of each link to each other. For example, the upper leftmost cell means that it is 97.2% likely that if a sufficiently large group of people were exposed to each of the LinkedIn and blog-link-only versions of my résumé, the group that saw the blog-link-only version would visit my blog more. Jesse E. Farmer has written more about the details of how this is calculated.

Oddly, the effect was reversed when you consider downloads rather than blog views. The résumés without any social media links were far less likely to be downloaded than those with. Even a résumé with a twitter profile did better than one without, though not by enough to be statistically significant.

The additional links also reduced the frequency of readers scrolling to the bottom of the page.

Conclusion

There are two main things I learned from this experiment. First, I’m going to keep social network links off of my résumé. Although they increased the download rate, they decreased visits to my blog. Since the latter is my priority, I’m not going to start adding social networks to my résumé.

Second, the short résumé did better in every way. However, the improvement in blog views was not statistically significant. For now, I’m keeping my online résumé at two pages, but I will use the one-page version in print.

There’s a number of disclaimers I should make here. For one, even if my findings are true of my résumé, they might not be true of other résumés. Maybe a change in layout would diminish the effect of linking to social media profiles, or make the longer résumé convert better. I should also point out that I have no way of knowing who my audience was. They probably weren’t all in a position to hire a programmer or data scientist, so the factors that make them visit my blog may or may not have the same effect on those who are.

And finally, a shameless plug. I’m looking for an interesting data science internship this fall (September to December 2010). If you’re doing cool things with data, I’d be glad to hear from you. My contact information is in the sidebar.

When I recently decided to sit down and properly learn the language, I was pleasantly surprised that underneath the line noise was an interesting and unique language. R is a descendant of LISP and, deep down, maintains some of the beauty its ancestor. It also borrows some unique and interesting features from other functional and dynamic languages.

Code is Data

R is true to its LISP roots in that you can create, modify, and evaluate parse trees from the code itself. One way to do so is with the quote() special-function, which returns its argument, unevaluated, as an expression object that can be traversed, modified and evaluated.

A fun (though not especially useful) consequence of this is that you can write an expression which returns itself as a quote:

> (function(x) substitute((x)(x)))(function(x) substitute((x)(x)))
(function(x) substitute((x)(x)))(function(x) substitute((x)(x)))
> expression <- (function(x) substitute((x)(x)))(function(x) substitute((x)(x)))
> expression == eval(expression)
[1] TRUE


Optional Laziness

By default, R uses eager evaluation, so expressions are evaluated as soon as they are assigned. However, R takes after functional languages like Haskell and O’Caml in that it allows lazy evaluation, where expressions are only evaluated at the time they are first used.

For example, consider the Haskell code:

m = sum [1..]


Where sum returns the sum of a list and [1..] is the (infinite) list of all natural numbers. In most languages, the assignment would cause the program to loop forever trying to sum all the natural numbers so it can assign that value to m. In Haskell, the assignment does complete; it simply assigns the expression sum [1..] to m so that it can be evaluated when the value of m is first used.

In R we can accomplish something similar with the delayedAssign() function:

delayedAssign("m", sum(1:Inf))


Note that in R, unlike O’Caml, the variables may be explicitly made lazy with delayedAssign, but are evaluated automatically when they are used.

Unfortunately, R evaluates lazy variables when they are pointed to by a data structure, even if their value is not needed at the time. This means that infinite data structures, one common application of laziness in Haskell, are not possible in R.

Operators are functions

When using higher-order functions, it’s sometimes useful to be able to treat operators as functions. Python accomplishes this in a clunky way: there is an operator module which redefines the built-in operators as functions. R takes a more functional approach. As in Haskell and O’Caml, operators are just syntactic sugar for ordinary functions. Enclosing any operator in backticks lets you use it as if it were an ordinary function. For example, calling `+`(2, 3) returns 5.

In fact, both the infix and prefix form are indistinguishable once they are parsed.

> quote(3 + 4) == quote(`+`(3, 4))
[1] TRUE


One surprising fact in R is that the assignment operators (<-, <<- and =) are functions like any other. As a result, they can be overwritten or passed around as desired, though neither strikes me as a particularly good idea.

Continuations

Continuations in R are a way of “breaking out” of a computation and jumping down the call stack to return early. The R function callCC() (call with current continuation) takes one argument, a function. It then evaluates that function, passing in a special function as an argument. callCC() then returns the first value that the special function is called with, or the return value of evaluating its argument if the special function is not called before the function returns.

To give you a better idea of what that looks like, consider this example:

> callCC(function(m) {return(4)})
[1] 4
> callCC(function(m) {m(2); return(4)})
[1] 2


Calling the function m(2) essentially cuts the computation short, drops down in the call stack to callCC, and returns 2.

If you’ve used continuations in another language, note that in R the exit function can only be called before callCC() returns. This makes R’s continuation semantics less powerful than those of languages like Scheme, Smalltalk, and Ruby.

R is not without its flaws and legacy baggage (you can trace its roots back to the S programming language 35 years ago), but once you learn to use it right, it’s a very powerful and indispensable language.

According to TechCrunch, the company is in the process of raising money at a $1.2 billion dollar valuation. There was lots of speculation about the future worth of the company, but little information about current revenue, even though there is a lot of raw data readily available in the site’s archives. I put together a scraper (in just a few lines of Python, thanks to BeautifulSoup) and gathered a total of 1065 past Groupons.

It isn’t clear how Groupon decides which Groupons to display in its archives. Presumably they are the better selling ones, so my sample is not a random sample, which would affect the numbers. Everything that follows should be taken with a grain of salt, but they should be reasonable as ballpark figures.

According to the data I collected, the average Groupon costs $30 and entitles the buyer to 57% off. On average 1155 people purchase it, resulting in $28,130 of revenue to Groupon ($28,130 is less than 1155 * $30 = $34,650 because, apparently, people are more willing to buy the cheaper Groupons.)

Averages are nice, but what I really wanted was totals. I was able to approximate what fraction of the data I had because Groupon advertises the “Total dollars saved” and “Total Groupons bought” on every page. By dividing my numbers by those, I determined that I had a little over a third of the data. Specifically, my data covered 31.2% of Groupons sold, and 37.4% of total savings.

Extrapolating the data I had (again, with the disclaimer that my sample may not be random), I calculated the total revenue since the beginning to be $80,188,176. If Groupon takes a 35% cut (to take a wild guess), $28 million of that is left after Groupon pays the company offering the deal. According to CrunchBase Groupon employs 90 people. I won’t speculate as to the operating costs of Groupon over the last year and a bit of operation, but once you subtract that number the rest is profit to date.

Looking on a monthly basis, the recent growth of the company is clear. A third of the total savings — in over a year of business — happened last month. This works out to $26,706,059 in revenue last month alone, or about $9.3 million (less the operating costs) profit if you assume they take a 35% cut. The below graph shows the growth by month.

Whether or not it’s a $1.2 billion company (BusinessInsider says that’s actually low, though without any quantitative justification), they’re clearly doing well for a company just over a year after launch.

Here are a couple more graphs constructed from the data (click to enlarge).

(Graphs were created with Google Fusion Tables.)

Update May 26: a few more graphs with more recent data follow.

Unfortunately, I do not have access to a powerful Tomcat server with a lot of bandwidth, so I can’t host an online demo. If anyone has the resources and is interested in hosting it, please let me know.

Here are some screenshots of the application in action:

Since it is a web-based application, any supported web browser can be the client (see the documentation for a list of supported browsers; any modern Gecko-based browser is supported as well as IE and Opera.) The client interface is loosely based on Google Maps. The server is a Java Servlet run through Tomcat. You can read more about how it works in the documentation.

Downloads:

• webFractal 1.0 (zip file)
• webFractal 1.0 Documentation (pdf)
• webFractal 1.0 Documentation (text)