We've just launched plato for individuals



Google Sign inLinkedIn Sign in

Don't have an account? 

Failed Attempt at Job Simulation


27 August, 2019

Ryan Garagay shares why job simulation failed at his company and was consequently phased out of the interview process.


In the past I had experience with a company that used job simulation and homework as interview strategies. These tasks are normally designed to give an accurate preview of what the role would entail. Furthermore, they help weed out candidates that aren't a fit for your organization. Unfortunately, these two methods didn't work for us, here's why.

Actions taken

Poor Planning The simulations and homework were taking up too much of the candidates' time. At a certain point, people lost interest because they were too long. More so, it was assumed that candidates were interviewing with other companies at the same time, companies who had an easier interviewing process. So we were at a disadvantage because of the poorly planned length of the tests. Validation on-site It became apparent that the interview process on-site was more efficient. We had enough successful hires who didn't go through the job simulation and homework stage that we eventually just phased those stages out. If the on-site process wasn't working so well, maybe we would have put more effort into developing and growing the job simulation and homework areas.

Lessons learned

  • We found that lengthier simulation and homework worked really well for higher demanding roles. So positions that are highly coveted - such as data science or even product management - allowed us to be more selective by purposefully planning for longer tests.
  • I think there is a fine balance between hypothetical testing versus actual work. Even within our organization I have challenged hypothetical questions that has nothing to do with our work. We have been doing this for quite some time, and have never encountered that type of problem nor have any reason to believe it would occur in the future, so why ask it. Don't waste valuable time by asking about things that you will not encounter in your day-to-day work. There are core competencies like data structures and algorithms that should be evaluated but in our case instead of asking tricky/puzzle question we relate it to a task where that skill was useful and have the candidate go through that problem by whiteboarding or actual coding session.
  • For roles which might not require a lot of actual experience we do ask hypothetical questions that relate to your work, even if you don't make the candidate actually follow through to a clear solution. These types of questions give you an idea of what the candidate knows, what they would do, and how they're going to get there versus having them give you an accurate answer or a perfect line of code.

Related stories

Hiring En Masse
31 August

Mason Mclead, CTO at Software.com, highlights all the advantages of hiring at once a group of people who have already worked together.

Mason Mclead

Mason Mclead

CTO at Software.com

Building a Team in the Midwest
24 August

Brian Hough, CTO at Beam Dental, recalls his company’s massive scale from only 12 to over 100 employees including engineers, product managers, designers, and leaders in under 12 months and highlights how hiring in the Midwest differs from hiring on the coasts.

Brian Hough

Brian Hough

CTO at Beam Dental

Hiring the Right Product Person for Your Team
18 August

Alessandro Pintaudi, Product Management Director at Payfit, dissects all the aspects of the hiring process that will enable you to hire a product person who will take your product to the next level.

Product Team
Alessandro Pintaudi

Alessandro Pintaudi

Product Management Director at PayFit

Growing Engineering Organization: One Step At a Time
2 August

Peter Fedorocko, Director of Engineering at Workday, recalls his early efforts to build a large, multilevel engineering organization in a short period of time -- one step at a time.

Scaling Team
Building a Team
Large Number of Reports
Peter Fedoročko

Peter Fedoročko

Director of Engineering at Workday

Recruitment and Interview Rotas: the Engineers’ Way
30 June

Jeff Foster, Head of Product Engineering, explains how engineers at his organization self-managed their taking part in the interviewing process.

Jeff Foster

Jeff Foster

Head of Product Engineering at Redgate

You're a great engineer.
Become a great engineering leader.

Plato (platohq.com) is the world's biggest mentorship platform for engineering managers & product managers. We've curated a community of mentors who are the tech industry's best engineering & product leaders from companies like Facebook, Lyft, Slack, Airbnb, Gusto, and more.