Using Deep Learning to Train a World-Class Candyland Player

A child playing Candyland
Photo by Amboo Who?

I’ve been spending the last year working on machine learning methods to train an algorithm that could be competitive with professional-level Candyland players. I’m happy to report that as of today, I attained my goal.

For the first pass of my automated Candyland player, I built a very simple rules-based engine. The engine ran efficiently on a Mac Book Pro. The rules-based engine did ok in lab tests with strong amateur players. I knew it couldn’t stand up to the rigor of professional Candyland tournament play, however.

For the second pass of my player, I created a more sophisticated look-ahead engine. Before taking its next card on its turn, it would simulate every possible permutation of the rest of the deck. While this was very processor intensive, by moving it into AWS and distributing it across 100 xLarge instances, it ran near real-time, although at some expense. The results of this player weren’t significantly better than the first engine.

In my third pass, I trained a shallow Convolutional Neural Network with 100,000 Candyland games. The results of this were disappointing. Playing against my look-ahead engine, over 500,000 games, it did no better than winning 50.05% of the time. Adding more layers didn’t improve the score significantly.

The fourth permutation of my Candyland player was a fully recurrent neural network running across a few dozen AWS GPU instances. This player played itself for several weeks improving the weights in its network using a genetic algorithm. The results from this were fascinating. It was able to win against itself 100% of the time.

This morning, April 1st, 2018, I played three games against the RNN player. While I am not a professional level Candyland player, I am rated 1200 on the amateur circuit, and it was my favorite game in 1st grade. This last version of my player beat me all three times! I think it is finally ready.

I’m going to be setting up some exhibition games with my Candyland player as soon as I can track down someone on the professional Candyland circuit. Not only will this gain some positive publicity for machine learning, but I’m hoping that it will help me recoup the $350,000 in AWS bills that I racked up building, testing, and training my players.

Talks: 1st half 2017

I’ve been remiss in posting since I’ve been back in Seattle. Readjusting to life in the states, and adjusting to the new role has kept me busy. I hope to rectify that in the future. I have been lucky enough to be invited to speak several times this winter and spring in the US and Europe. If you’ll be there or live nearby and want to meet up, drop me a line on twitter and let me know!

Articles I’ve liked recently (stuff I’ve been reading lately #3)

Why Apple Music Is So Bad When the iPhone Is So Good
On April 28, 2003, Apple launched the iTunes Music Store, saving the music industry from the scourge of piracy while creating a large and steady source of…

Scaling Lean — Lean Startup Co. Blog — Medium
Scaling Lean Written by Jennifer Maerz of Lean Startup Co. How can you really measure the effectiveness of your company’s business model? The process goes much…

Silicon Valley’s Scapegoat Complex — Thinkpiece Dot Club
Tech’s Scapegoat Complex T he recent revelation that tech investor Peter Thiel provided the funding for a series of lawsuits against Gawker Media is one of…

Keep an eye on Norway: Its startup scene is about to go huge
I spent a lot of my formative years in Norway, and have been periodically checking in on the Norwegian startup scene. The first time I went and had a look…

‘The Life of Pablo’ and the Death of the Traditional Album
As a genuine certified old person, one who generally doesn’t stream music, who remembers having racks and racks of CDs, and who keeps much of his vinyl sorted…

If Your CFO Hasn’t Already Told You to Control AWS Costs, He’s About To
You made the obvious move and migrated to Amazon Web Services (AWS). Months later, the attractive glow of the move from CapEx to OpEx spend has been dimmed by…

Can Netflix Survive in the New World It Created? –
One night in early January, a little after 9 o’clock, a dozen Netflix employees gathered in the cavernous Palazzo ballroom of the Venetian in Las Vegas. They…

Stuff I’ve been reading lately #2

12 Powerful habits I have stolen from ultra-successful people (tip to kate matsudaira)

12 Lessons of Waking Up at 4:30 a.m. for 21 Days — Life Hacks for Business
12 Lessons of Waking Up at 4:30 a.m. for 21 Days If it works for me, maybe it works for you!

Practical Networking Tips for Introverts
Networking was such a foreign concept to me. And I really do mean foreign — we don’t do the organized networking thing in El Salvador. I knew that building…

Curing Our Slack Addiction
I love Slack. I really really do. So much so I would call it an addiction at this point. Slowly but surely this addiction has been killing my sanity and sapping…

Official Google Blog – This year’s Founders’ Letter
Every year, Larry and Sergey write a Founders’ Letter to our stockholders updating them with some of our recent highlights and sharing our vision for the…

The Choice Explosion
Lansing, W.Va. — A few years ago, the social psychologist Sheena Iyengar asked 100 American and Japanese college students to take a piece of paper. On one side,…

Doing a TED Talk: The Full Story – Wait But Why
You’ve probably heard this Seinfeld joke: According to most studies, people’s number one fear is public speaking. Number two is death. Death is number two. Does…

Dropbox cut a bunch of perks and told employees to save more as Silicon Valley startups brace for the cold
You have successfully emailed the post. 20h 8,000 REUTERS/Stringer Winter’s coming in Silicon Valley. When Dropbox employees walked into their new office on…

Thoughts on Take Home Interviews
There is a movement now in tech to really think about what it would take to improve our interview process. This is a movement a long time coming. White board…


Stuff I’ve been reading lately

I’m playing with an idea here. One of the things I really like on Instapaper is the browsing of what your friends are reading. My pal Kevin Stewart, has particularly awesome tastes on technical articles and I find myself saving almost everything he reads to my own queue. 

I won’t vouch that my own tastes are quite as good, but maybe the articles that I’ve found valuable might be valuable to others. So, I created an action to save any article that I’ve liked to a google doc. Periodically, I will post the most recent articles to this blog. At some point in the future, I might autopublish them. For now, I like the idea that I can edit them before they go live.

Collaborative Overload
Over the past two decades, the amount of time managers and employees spend on collaborative work has ballooned. At many companies…

Two Items That Aren’t On Your Meeting Agenda, But Should Be
In this era of frequent testing, schools get a dizzying quantity of data about their students. Figuring out what to do with it, though, is more complicated than…

Good Bosses Create More Wellness than Wellness Plans Do
In the name of employee wellness, and in response to insurance company demands, corporations are offering well-being initiatives with financial incentives.…

Billing by Millionths of Pennies, Cloud Computing’s Giants Take In Billions
Andrew R. Jassy, the senior vice president of Amazon Web Services. The per-millionth pricing began last November, in the A.W.S. product Lambda. Amazon Web…

Congratulations! You’ve Been Fired
AT HubSpot, the software company where I worked for almost two years, when you got fired, it was called “graduation.” We all would get a…

Medium raises another $50M
Medium , the online publishing platform led by Twitter co-founder Ev Williams, just announced that it has raised $50 million in Series C funding. It’s been less…

Why Uber Won — Greylock Perspectives
Why Uber Won The Startup Steroid Era and the Use of Capital as a Performance Enhancing Drug The start of 2016 marked the end of the steroid era of startups —…

Hands-on: $290 Kindle Oasis starts a new chapter for Amazon e-readers
The new Kindle Oasis comes with a leather battery cover included in the price, available in three colors, black, walnut and merlot. (GeekWire Photo) The word…

Nice deck from Dan McKinley of Stripe: Choose Boring Technology

As you grow as a developer (and development leader) and you work with more and more technologies over time in different projects, you start to realize how easy it is for the team to get more focused on the challenging technical problems than the actual product issues. Ignoring the product issues will kill the product (and possibly the company). With limited attention (he calls them innovation credits), it is best to put your effort into innovations that can differentiate your product. All too often, teams get more focused on the next cool technologies, turning everything into a nail as the old saying goes.

Dan McKinley does a great job explaining this in his talk below.

Fun with the Apple predictive keyboard

I’m getting used to the predictive keyboards in iOS8. I found myself wondering today what would happen if I just hit a single key and then just picked the suggested word over and over. So I typed “a” and the following resulted. It looked like it was going to end up in an infinite loop, but while it got very repetitive, that never happened.

So, cheap fun (if you don’t include the cost of the iPhone)…

And the rest of the day after a long day at the same thing to say it was the best of the year and the rest of the day after a while ago but the fact I can get it right away with the best of the best way of the best thing to say it was the best thing to say it is not the same thing to say that I have a great way of the day I have a great way of life and the rest of my favorite part of the best of luck with the best thing ever I go back and the rest of my favorite part of the day I have to go back and the rest of my life and the rest of my favorite part of a sudden it was the first half of the best of the year and the rest of my favorite part of a sudden it was the first half of the day after a while ago but the rest of the best…



Your Software Will Fail You

Dead formats

I was reading about Jason Kincaid’s issues with Evernote, a piece of software that I am also dependent on (but luckily haven’t had any issues with yet). It reminded me of other software that I depend on that has or is currently failing me: iTunes, and 1Password.

The software I use every day today is different from the software I used every day a few years ago. The software I use in a few years will be different than the software I use today. Through decades of computer usage, I’ve realized that I can’t depend on my software, and that relying on it to exist and to work is folly. As we move towards subscription models for software, this will be ever more the case.

Let me talk a bit about how I work with my software now, in the form of a suggestion list. Some of my friends in the industry think it is surprisingly luddite, but it has reduced my pain over the years significantly. It means less work in some areas (like fixing busted software updates), and more work in other areas (maintaining plan-text backups), but as I moved to having everything digital, it at least lets me feel like my files are somewhat future proof.

  • When you buy digital media, only buy DRM-free. I learned this lesson many years ago. In the early days of digital music there were many competing, protected, file formats. All of them died. If you bought Liquid Audio files, or one of the variants of Microsoft protected audio files, you were screwed. I buy DRM-free eBooks whenever I can (I love that O’Reilly sells their books DRM-free always). When I buy music, I only buy MP3 or Ogg files (this doesn’t seem to be a problem to find any more). TV or Movies are more problematic. I basically only buy those when I absolutely can’t get them any other way, and I assume that at some point in the future, I will have a way to remove the DRM or I will just lose them. If I can’t find a reasonably priced DRM-free alternative, I will sometimes buy the physical copy and rip or scan it instead of paying for DRM-hobbled versions.
  • Only use software that lets you output archival file formats. Evernote let you output all your notes as HTML or XML. I backup my Evernote to these files on a monthly basis. I backup all my Outlook e-mail as mbox (plaintext) files (you can do this by dragging individual folders to disk). 1Password also lets you output as an XML file (although you want to encrypt this somehow in your backups). For the web services I use, I use ifttt to archive them either to Evernote, or Dropbox, or both.

    Creative application files are difficult, but I have also learned this the tough way. I recently had to track down a copy of Adobe Illustrator CS5 because it was the last version of the software that worked with Freehand files. I was a big Freehand user and I never got around to putting my Freehand files into some application-neutral format. Now I make sure that I save a copy of everything in an uncompressed, full resolution, non-proprietary format so that I can get to it again if I need it. This takes up a lot of space, but space gets cheaper. Losing access to something that you spend hours, days or weeks on is worth the cost of a few extra GB.

    I learned in my experience moving from iPhoto to Lightroom how painful it is when you use an application with virtual edits. Periodically, in LR, I output the edited versions of my images at full resolution so I won’t lose those if I can’t use LR anymore. The metadata is another problem, but LR at least save these in the sidecar or image files, so I can reconstruct them if I have to.

    For audio apps, I output the dry and wet stems from each channel when I finish a track so that I can remix later and maybe use them as a guide if I want to try to reconstruct a track from the original files.

  • Back up everything, multiple ways. What is the point of making sure your digital life is future proof if you can still lose it with a hard disk failure? In my case, I use CrashPlan for cloud backup. I also have a set of drives at work that are backups of my home drives. I bring each one home for one night just to back up its pair and the rest of the time it stays at work. In this way, I have three copies of every file. I was also contemplating another NAS backup within my house, but with over 6TB of data, that is a bit expensive for a fourth, redundant copy. I will probably do it anyway at some point.
  • Save copies of the software that you buy/download. Unfortunately, everything these days is still authorized over the internet. This means that if you need to install an old piece of software to read a file that is no longer readable some other way, you might not be able to get it fully enabled, but you still might be able to use it in trial mode for a short period of time, long enough to recover that old file. Having the DMG for CS5 saved my bacon.
  • Only update when you need/have to. This is the most controversial thing, especially because I have always made my living by selling (or renting) software (and upgrades) to people. This has also gotten a lot harder since the rise of the desktop app store and subscription revenue model. For my personal machine that I rely on and have no help or support on, I am very, very careful about when (or if) I update critical software. Before I apply a minor OS update, I always check the support boards to see if there are any issues. I almost never apply a major OS update to my personal computer. I actually can’t think of the last time I did this. The same goes for my critical (as opposed to fun) software. If everything is working on my machine and I’m able to get everything done, I prefer to leave it in that state rather than messing with my machine, possibly screwing myself up. There are a few exceptions. I will always apply security updates, for example. This doesn’t mean that I am always several years behind on software though. I update my hardware on a pretty regular basis, and usually when I do, I update all the software that I am currently using as well. I will still keep my old hardware around for a while, in case I need an old application for something.
  • Keep your files organized. Having everything means you need to be able to find anything. The good part of keeping things in standard file formats means you can take advantage of your OS’s search capabilities, but you’ll still want a reasonable directory structure.


For software developers
It’s easy to ignore old operating systems and backwards compatibility. You can look at your analytics and say “no one uses that feature anymore.” I’ve made that calculation myself many times as an engineering leader. Still, it is worth making sure that your users have an exit or even a staying-put strategy. Especially, if you are building a service or subscription instead of an application. I used to use Gowalla. I put a lot of data into that service. When it went out of business, they put up a page promising a tool to download your data. I thought that was a classy way to go. That tool never appeared, and all that data was lost.

If you want to treat your users right, make them never regret using your software. If you are lucky enough to have your software last for a while, remember all the people who paid you along the way. Treat them with respect, and they will keep paying you into the future.