https://paulbutler.org/posts/ Recent content in Posts on Hugo -- gohugo.io en-us Sat, 08 Feb 2025 00:00:00 +0000 https://paulbutler.org/2025/smuggling-arbitrary-data-through-an-emoji/ Sat, 08 Feb 2025 00:00:00 +0000 https://paulbutler.org/2025/smuggling-arbitrary-data-through-an-emoji/ This Hacker News comment by GuB-42 intrigued me: With ZWJ (Zero Width Joiner) sequences you could in theory encode an unlimited amount of data in a single emoji. Is it really possible to encode arbitrary data in a single emoji? tl;dr: yes, although I found an approach without ZWJ. In fact, you can encode data in any unicode character. This sentence has a hidden message󠅟󠅘󠄐󠅝󠅩󠄜󠄐󠅩󠅟󠅥󠄐󠅖󠅟󠅥󠅞󠅔󠄐󠅤󠅘󠅕󠄐󠅘󠅙󠅔󠅔󠅕󠅞󠄐󠅝󠅕󠅣󠅣󠅑󠅗󠅕󠄐󠅙󠅞󠄐󠅤󠅘󠅕󠄐󠅤󠅕󠅨󠅤󠄑. (Try pasting it into this decoder) https://paulbutler.org/2024/the-haters-guide-to-kubernetes/ Sun, 03 Mar 2024 00:00:00 +0000 https://paulbutler.org/2024/the-haters-guide-to-kubernetes/ Among a certain tech set, Kubernetes has earned a reputation as an unnecessarily complicated time-sink that startups should avoid. Using Kubernetes with a small team is seen as a sign of over-engineering. I’m guilty of taking pithy jabs myself. I might gripe about Kubernetes sometimes, but it really is a great piece of technology. I highly recommend it to all my competitors. — Paul Butler (@paulgb) September 9, 2022 Despite my snark, “great piece of technology” truly is sincere; at the time of that post I had recently written about how much of Kubernetes’ complexity is necessary for what it does. https://paulbutler.org/2022/generalizing-secret-santa-with-integer-programming-in-julia/ Sat, 10 Dec 2022 00:00:00 +0000 https://paulbutler.org/2022/generalizing-secret-santa-with-integer-programming-in-julia/ You might know Secret Santa as the gifting game where each person in a group is assigned another person in the group to give a gift to. For three years in a row, I’ve organized a post card exchange for plotter enthusiasts called #ptpx, which I think of as a sort of “generalized Secret Santa”. Instead of just sending and receiving one gift, participants choose how many postcards they’d like to make and send. https://paulbutler.org/2022/why-is-it-so-hard-to-give-google-money/ Tue, 26 Jul 2022 00:00:00 +0000 https://paulbutler.org/2022/why-is-it-so-hard-to-give-google-money/ I’ve been trying to run search ads on Google for over a month. Due to two unrelated snags, I’ve only had ~12 hours of uptime with those ads in that time. Based on conversations with other startups that run Google Ads, my experience is neither unique nor surprising. Apropos of Google’s earnings day, I wanted to vent a bit about how hard Google makes it to use the product that drives so much of their revenue. https://paulbutler.org/2022/the-problem-with-bitcoin-miners/ Sun, 15 May 2022 00:00:00 +0000 https://paulbutler.org/2022/the-problem-with-bitcoin-miners/ Disclosure: I hold short positions in companies mentioned in this article. This is not financial advice. Six months ago, I wrote: [Bitcoin] miners burn an enormous amount of capital just to capture the same slice of a diminishing pie, but they take advantage of investor confusion over how mining works to frame it as investing in growth. I want to go deeper on that idea. But first, some accounting. https://paulbutler.org/2022/rust-for-web-devs-destructuring/ Sat, 19 Feb 2022 00:00:00 +0000 https://paulbutler.org/2022/rust-for-web-devs-destructuring/ This is the first of an unbounded and weakly-ordered series of posts aimed at developers coming to Rust from a JavaScript/TypeScript background. It’s “for web devs” because my imagined reader is a web developer, but it’s not about web development. Nor is it meant to be a comprehensive introduction to Rust, just a series of essays on the parts I find interesting. If you’re brand new to Rust, you might want to start with The Book or Rust in Action. https://paulbutler.org/2022/what-does-it-mean-to-listen-on-a-port/ Sun, 13 Feb 2022 00:00:00 +0000 https://paulbutler.org/2022/what-does-it-mean-to-listen-on-a-port/ This story explores some concepts in computer networking, inspired by Michael Nielsen’s idea of discovery fiction. Code samples can also be found in this repo. Excerpts use openbsd-flavoured netcat on Debian Linux; behviour and IPv6 support may vary by version. PS: I’m currently hiring Rust and TypeScript developers for a YC-backed startup. In the corner of the student union building there is a coffee shop, and in the corner of the coffee shop are two students. https://paulbutler.org/2021/play-to-earn-and-bullshit-jobs/ Tue, 28 Dec 2021 00:00:00 +0000 https://paulbutler.org/2021/play-to-earn-and-bullshit-jobs/ In Bullshit Jobs: A Theory, David Graeber makes the case that a sizable chunk of the labour economy is essentially people performing useless work, as a sort of subconscious self-preservation instinct of the economic status quo. The book cites ample anecdotal evidence that people perceive their own jobs as completely disconnected from any sort of value creation, and makes the case that the ruling class stands to lose from the proletariat having extra free time on their hands. https://paulbutler.org/2021/betting-against-bitcoin/ Tue, 26 Oct 2021 00:00:00 +0000 https://paulbutler.org/2021/betting-against-bitcoin/ I hold what amounts to a bet against bitcoin, expressed in terms of options and shorts on public market bitcoin-related companies. Given that crypto discourse tends to be dominated by bulls and boosters because that’s who shows up, I wanted to share my bear case. This is, emphatically, not investing advice. The position represents only a small part of my portfolio, and is something of a hobby project. The market can remain irrational, and all that. https://paulbutler.org/2021/calling-webassembly-from-rust/ Sat, 14 Aug 2021 00:00:00 +0000 https://paulbutler.org/2021/calling-webassembly-from-rust/ WebAssembly originated as a lightweight, portable, sandboxed code execution model for the web. Since its early days, people have realized that these qualities were useful outside of the web as well. As the standard has evolved, a wealth of tooling and standards for running WebAssembly ex-browser have appeared. As I write this in August 2021, embedding WebAssembly involves understanding a handful of moving parts: language proposals at various stages of implementation; consensus emerging around standards like WASI; and maturing toolchains for various languages. https://paulbutler.org/2021/ranked-choice-voting-for-data-scientists/ Sat, 12 Jun 2021 00:00:00 +0000 https://paulbutler.org/2021/ranked-choice-voting-for-data-scientists/ In a few weeks, New York City will become the largest US jurisdiction to use instant-runoff voting when it is used in the mayoral primary elections. Ranked-choice elections generate a lot of data. It’s not clear yet how much of this data the New York City Board of Elections will release immediately following the election — a bill that would mandate a broad data release did not pass before the assembly adjourned on Friday. https://paulbutler.org/2020/the-webassembly-app-gap/ Sun, 17 May 2020 00:00:00 +0000 https://paulbutler.org/2020/the-webassembly-app-gap/ To users of Figma, it’s no mystery why the browser-based design tool is popular. What has been a mystery to me is why it’s so hard to point to Figma equivalents in other categories. Specifically, I mean browser-based apps that embrace modern browser tech (WebAssembly, WebGL) to compete head-to-head with desktop apps without having to compromise features for performance. The answer, I believe, is simply that nobody’s built them yet. This essay states the case for the modern browser as a platform, and explores some components that might fill the gaps in a modern stack. https://paulbutler.org/2019/comments-on-a-false-majority/ Sun, 15 Sep 2019 00:00:00 +0000 https://paulbutler.org/2019/comments-on-a-false-majority/ Over the past year, I have undertaken a project to analyze instant runoff elections at the ballot-level across multiple cities and states. To date, I have processed 31 instant-runoff elections, which has involved implementing in code the ballot counting and tabulation methods of five localities. In the process, I’ve gained some insight into how the instant runoff process works (and when it doesn’t), obscure trivia of state-to-state implementation differences, and tools for analyzing election results. https://paulbutler.org/2019/the-kelly-criterion-how-to-size-bets/ Sun, 27 Jan 2019 00:00:00 +0000 https://paulbutler.org/2019/the-kelly-criterion-how-to-size-bets/ https://paulbutler.org/2018/did-the-white-house-doctor-a-video/ Fri, 09 Nov 2018 00:00:00 +0000 https://paulbutler.org/2018/did-the-white-house-doctor-a-video/ https://paulbutler.org/2018/the-misunderstood-electoral-college/ Tue, 06 Nov 2018 00:00:00 +0000 https://paulbutler.org/2018/the-misunderstood-electoral-college/ https://paulbutler.org/2018/understanding-the-mandelbrot-fractal/ Wed, 15 Aug 2018 00:00:00 +0000 https://paulbutler.org/2018/understanding-the-mandelbrot-fractal/ https://paulbutler.org/2018/optimizing-plots-with-a-tsp-solver/ Sun, 25 Mar 2018 00:00:00 +0000 https://paulbutler.org/2018/optimizing-plots-with-a-tsp-solver/ https://paulbutler.org/2018/tuning-style-transfer-with-a-control-surface/ Sun, 18 Feb 2018 00:00:00 +0000 https://paulbutler.org/2018/tuning-style-transfer-with-a-control-surface/ There’s a fun technique introduced by Gatys, Ecker, and Bethge that lets you reduce an image into separate “content” and “style” representations. Then, by taking a content representation from one image and the style representation from another, you can create a new image that matches the content of the first image in the style of the second. The gist of style transfer is that you take a pre-trained image classification network (VGG-16 trained on ImageNet), and then run an image through it. https://paulbutler.org/2017/surface-projection/ Sat, 16 Dec 2017 00:00:00 +0000 https://paulbutler.org/2017/surface-projection/ https://paulbutler.org/2017/stop-dragging-hayek-into-bitcoin/ Sun, 12 Nov 2017 00:00:00 +0000 https://paulbutler.org/2017/stop-dragging-hayek-into-bitcoin/ Note: This post discusses the relationship between Hayek’s work and cryptocurrencies, and is not financial advice. Opinions are my own. I do not hold a position in Bitcoin or other cryptocurrencies. When monetary theory comes up in reference to Bitcoin, the work of Friedrich von Hayek is often invoked. I have read a number of claims that the Nobel laureate’s 1976 book, Denationalisation of Money,1 lays the theoretical foundation for the cryptocurrencies we see today. https://paulbutler.org/2017/the-surprising-persistence-of-bad-signals/ Sun, 24 Sep 2017 00:00:00 +0000 https://paulbutler.org/2017/the-surprising-persistence-of-bad-signals/ How likely is a debtor to default on a loan? How likely is a student to succeed in college? Both of these questions demand a probability that is impossible to measure directly. Instead, we find signals that proxy for them. SAT scores are meant to predict college success. FICO scores proxy for creditworthiness. We accept that there is a certain amount of measurement error in these scores. If the market for these signals operated like markets for other things (say, lawnmowers or laundry detergent), we would expect that over time better signals would emerge and replace the existing ones. https://paulbutler.org/2017/fractal-generation-with-l-systems/ Sun, 09 Jul 2017 00:00:00 +0000 https://paulbutler.org/2017/fractal-generation-with-l-systems/ https://paulbutler.org/2013/visualizing-flight-options/ Sun, 13 Jan 2013 00:00:00 +0000 https://paulbutler.org/2013/visualizing-flight-options/ Few routine consumer purchases involve as great a volume of data as booking a flight. For an international trip, there can be dozens of options differing by price, duration, time-of day, connections, and carrier alliances. ITA Software, now part of Google, quietly introduced a Gantt chart-style visualization. The use of Gantt-charts for flights has since been perfected and popularized by Hipmunk. Hipmunk set the bar high, especially for domestic flights, by polishing the design and distilling the data to the minimum needed to book a flight. https://paulbutler.org/2012/make-for-data-scientists/ Mon, 15 Oct 2012 00:00:00 +0000 https://paulbutler.org/2012/make-for-data-scientists/ This is a follow-up to Chris Clark’s post, Engineering Practices in Data Science, and is cross-posted to the Kaggle blog. Any reasonably complicated data analysis or visualization project will involve a number of stages. Typically, the data starts in some raw form and must be extracted and cleaned. Then there are a few transformation stages to get the data in the right shape, merge it with secondary data sources, or run it against a model. https://paulbutler.org/2011/groupon-visualization-in-hbr/ Mon, 25 Jul 2011 00:00:00 +0000 https://paulbutler.org/2011/groupon-visualization-in-hbr/ A version of my Groupon visualization appears in the latest Harvard Business Review. At the suggestion of their editorial and design staff, I re-did the visualization to focus on a smaller subset of the data: deals in San Francisco. I must give the HBR staff a lot of credit for the final product — they made a visualization designed to be interactive work in print. https://paulbutler.org/2011/what-480-of-gross-revenue-looks-like-to-groupon/ Mon, 28 Feb 2011 00:00:00 +0000 https://paulbutler.org/2011/what-480-of-gross-revenue-looks-like-to-groupon/ On Saturday, the Wall St. Journal posted details of an internal Groupon memo that reported $760 million in revenue last year. The WSJ article came just as I was finishing up a visualization of some data I had collected on Groupon deals, which gives perspective on that massive number in terms of the individual deals. Each box is a deal. I used height to represent number sold, and width to represent the price. https://paulbutler.org/2010/visualizing-facebook-friends/ Sat, 18 Dec 2010 00:00:00 +0000 https://paulbutler.org/2010/visualizing-facebook-friends/ Earlier this week I published a data visualization on the Facebook Engineering blog which, to my surprise, has received a lot of media coverage I’ve received a lot comments about the image, many asking for more details on how I created it. When I tell people I used R, the reaction I get is roughly what I would expect if I told them I made it with Microsoft Paint and a bottle of Jägermeister. https://paulbutler.org/2010/data-structures-for-range-sum-queries/ Sat, 10 Jul 2010 00:00:00 +0000 https://paulbutler.org/2010/data-structures-for-range-sum-queries/ This week I attended the Canadian Undergraduate Mathematics Conference. I enjoyed talks from a number of branches of mathematics, and gave a talk of my own on range-sum queries. Essentially, range-aggregate queries are a class of database queries which involve taking an aggregate (in SQL terms, SUM, AVG, COUNT, MIN, etc.) over a set of data where the elements are filtered by simple inequality operators (in SQL terms, WHERE colname {<, <=, =, >=, >} value AND …). https://paulbutler.org/2010/an-experiment-in-a/b-testing-my-r%C3%A9sum%C3%A9/ Thu, 01 Jul 2010 00:00:00 +0000 https://paulbutler.org/2010/an-experiment-in-a/b-testing-my-r%C3%A9sum%C3%A9/ Objective I’ll admit it: my résumé doesn’t stand out. I’ve had some great internships, but also a tendency to work for companies that aren’t (yet!) household names. And though I’m doing fine academically, it’s not well enough to stand out on my marks alone. On the other hand, my blog lets me stand out. I’ve had a few opportunities to meet and interview with some great people and companies because they read my blog. https://paulbutler.org/2010/why-r-doesnt-suck/ Sat, 19 Jun 2010 00:00:00 +0000 https://paulbutler.org/2010/why-r-doesnt-suck/ I first encountered the R programming language a few years ago when I needed to make some plots. Although I’ve used it occasionally since, I always considered it a sort of “Perl for statisticians” — a useful swiss-army knife with ugly syntax and inconsistent semantics. My workflow generally involved manipulating the data in Python and using R to make a simple plot, minimizing the amount of R code I wrote as much as possible. https://paulbutler.org/2010/groupon-math-data-scraping-to-estimate-revenue/ Thu, 15 Apr 2010 00:00:00 +0000 https://paulbutler.org/2010/groupon-math-data-scraping-to-estimate-revenue/ There’s been a lot of talk recently about the Chicago startup Groupon. Groupon brands itself as a group-buying site, but it’s really more of a localized version of what woot.com does. They post a new deal (which they call a Groupon) every day, available only on that day. If enough people want to buy it, everyone gets it for a substantial discount. Otherwise, nobody gets anything, but this rarely happens from what I can tell. https://paulbutler.org/2009/do-we-really-need-another-programming-language/ Wed, 11 Nov 2009 00:00:00 +0000 https://paulbutler.org/2009/do-we-really-need-another-programming-language/ Yesterday, a group inside Google released a new programming language called go. Among the many comments on the new language, I noticed a number of people rhetorically asking if there is really room for more programming languages. I’ve seen the same sentiment expressed before, seemingly whenever someone releases a pet language (Arc, Factor) or rediscovers an older programming language (Smalltalk, Erlang, LISP). I’m convinced that there is, in fact, room for more programming languages. https://paulbutler.org/2009/n-queens-in-a-tweet/ Wed, 24 Jun 2009 00:00:00 +0000 https://paulbutler.org/2009/n-queens-in-a-tweet/ Most people who like puzzles or study computer science have probably encountered the famous N-Queens problem. If you haven’t, before reading any further, try this online version of the most popular form, the 8-Queens problem. The 8-Queens problem is to find positions on a chess board for eight queen chess pieces so that none of them “threaten” the others. Since a queen in chess can move horizontally, vertically, and diagonally, this means placing each queen on her own horizontal, vertical, and diagonal lines. https://paulbutler.org/2008/python-debugging-with-decorators/ Sun, 22 Jun 2008 00:00:00 +0000 https://paulbutler.org/2008/python-debugging-with-decorators/ I’ve written a little python function which I have found to be very helpful for debugging. It takes a function, and returns a function which is identical to the original except that it prints a message to the console with useful information every time the function is called or returns. Here is the function: # Number of times to indent output # A list is used to force access by reference __report_indent = [0] def report(fn): """Decorator to print information about a function call for use while debugging. https://paulbutler.org/2008/simplediff-in-python/ Wed, 13 Feb 2008 00:00:00 +0000 https://paulbutler.org/2008/simplediff-in-python/ A while ago I posted a PHP implementation of a diff algorithm I came up with. Since it was well received, and it’s a useful little algorithm to have, I created a Python version as well. There are a few performance improvements as well. The PHP version creates an array in memory proportional to the square of the size of the input, while the Python version’s array is directly proportional to the size of the input. https://paulbutler.org/2007/tail-recursion-in-python/ Thu, 13 Dec 2007 00:00:00 +0000 https://paulbutler.org/2007/tail-recursion-in-python/ After spending a lot of time in Scheme, it’s hard not to think in recursion from time to time. When I recently started to improve my Python skills, I missed having Scheme optimize my tail recursive calls. For example, consider the mutually recursive functions even and odd. You know a number, n, is even if it is 0, or if n – 1 is odd. Similarly, you know a number is not odd if it is 0, and that it is odd if n – 1 is even. https://paulbutler.org/2007/garden-path-sentences/ Wed, 27 Jun 2007 00:00:00 +0000 https://paulbutler.org/2007/garden-path-sentences/ I recently came across an interesting post on the Powerset Blog recently about garden path sentences. Garden path sentences are sentences that lead you down the wrong path through a string of words with multiple meanings. For example, The complex houses married and single students and their families In this case, most readers would probably think complex was an adjective that modified the plural noun houses. The post ended with a challenge – how easy would it be to create a program to automatically generate these sentences. https://paulbutler.org/2007/a-simple-diff-algorithm-in-php/ Tue, 15 May 2007 00:00:00 +0000 https://paulbutler.org/2007/a-simple-diff-algorithm-in-php/ A diff algorithm in its most basic form takes two strings, and returns the changes needed to make the old string into the new one. They are useful in comparing different versions of a document or file, to see at a glance what the differences are between the two. Wikipedia, for example, uses diffs to compare the changes between two revisions of the same article. Solving the problem is not as simple as it seems, and the problem bothered me for about a year before I figured it out. https://paulbutler.org/2007/webfractal-web-based-fractal-explorer/ Sat, 17 Feb 2007 00:00:00 +0000 https://paulbutler.org/2007/webfractal-web-based-fractal-explorer/ Last weekend, I won a nice new Toshiba laptop in a local software competition. My entry was a web-based fractal explorer. I had a lot of fun making it, and it is fun to play with as well. I have decided to release it under an open-source license so that other people can play around with it (see the download link at the bottom of this post). Unfortunately, I do not have access to a powerful Tomcat server with a lot of bandwidth, so I can’t host an online demo.