This is the second post on my experience running the MX ruby apprenticeship program. The Previous Post focused on our goals and plans for the apprenticeship.

With dates planned it was time to get some applications for the apprenticeship. How should someone apply to the apprenticeship? How would we know which candidates best fit our criteria of being self-motivated to learn new things and tackle hard problems? Who would bother applying in the first place?

The Application

Since we had a pretty ambiguous selection criteria, we decided to use a similarly ambiguous application medium.

Make us a page, anywhere on the internet, that provides evidence that you are self-motivated, learn new things easily and like to solve complex problems.

The requirement of making a page helps to enforce our base criteria for what skills an apprentice should have coming into the program. But it also lets the apprentices have a lot of freedom in choosing how to present themselves. They could use videos, animations, gifs, etc.

We ended up receiving 55 applications and they were as different from each other as we were hoping. Here are a few that caught our attention:

  • Danielle Hunter created a page that was compelling both visually and as a narrative (protip: read the recommendation at the bottom).
  • Aristides Perez came out of the DevBootcamp program and had a good overview of some of this projects.
  • Jean Simon is an excellent of example of keeping it very simple. There is very little visually happening, but the writing was compelling to us.
  • Cam Kidman is one of the only examples I’ve seen where calling out your strengths and weaknesses came off really well.

Evaluation

Given such a diverse set of applications we needed a way to evaluate them to pick a set of people to bring back for interviews. I made a github repo called apprenticeship-applications and created an issue for each applicant that contained a link to their application. The entire MX engineering team pitched in to spend a few minutes evaluating different candidates. Team members would notice different things and add comments about the applicants’ github history, application page and even twitter accounts.

During this phase of evaluation there were a lot of disagreements between the reviewers. People would comment about a given applicant’s skill level, and another engineer would comment that we aren’t selecting for skills, we are selecting for potential. There were a lot of discussions in the github issues and in person about which applicants looked the strongest.

Eventually one person needed to see them all so that the judging could be done on a single scale. So after all the other comments I went through and scored each of the 55 applications and picked 15 people to bring in for interviews. Reading 55 applications takes a long time and I noticed a few trends for which ones stood out to me.

  • Any kind of public programming history was much better than none. When reviewing an application it was really hard for me to feel confident in our ability to get someone up to speed if there was no visible history of their work (ie. github, bitbucket, googlecode, etc)
  • Working applications are less convincing than a story. This one surprised me. Lots of people applied with a link to a site that they had built, But this made it really hard to evaluate anything about the person other than the fact that they can build an app. Understanding why they built it, who helped them or how they overcame problems along the way was very helpful.
  • Avoid phrases like “fast learner” or “hard worker”. Telling a story about what you learned and how you used it is much more compelling than designating yourself as a fast learner.

Interviewing

In order for the apprenticeship to work, we really needed the apprentices to teach each other. So we organized the interview process as small groups (2 to 4 people) and we would give them an example problem to work on as a team.

The example problem was to take a monthly budget and a list of goals and predict when each goal would complete. We gave applicants the option to document their solution as a verbal description, a flowchart, pseudocode or actual code. Most applicants chose to discuss the problem by writing ruby or javascript code back and forth as they were talking. We were evaluating their ability to verbally discuss a technical problem, to collaborate as a team and to solve the problem.

After a lot of arguing in the application process, this stage of evaluation was extremely smooth. After each group interview I would ask each engineer who had been there to rank the applicants and the rankings were nearly identical. The biggest disparity that existed was whether to rank an applicant as 2nd or 3rd. These small disparities did not end up facotring into the final hiring decisions.

The Outcome

Although we originally intended to hire 3 apprentices, we ended up offering a position to 5 and all of them accepted. We are thrilled that Jean Simon, Cam Kidman, Aristides Perez, Andrew Lewin and Toby Redd have joined the team as apprentices. Along the way we learned that there are a lot more talented candidates than we had thought.