In the past, hiring developers has had some terrible practices. A few that come to mind are whiteboard tests, complex take home exercises, and more recently, timed programming exercises in an unfamiliar environment.
Now that AI is here and finally doing some of the things we feared or anticipated it might, we can start to see what this might do to the hiring process.
I have to admit. When I set out to write this, I thought it would all be positive. But I’m not sure that’s the case as I start to think through it.
First the good.
AI will all but eliminate take home programming exercises. It’s way too easy to tell ChatGPT to build you an app that you describe to it. Then just update parts to make it unique. I wouldn’t do this. But many developers will. I guess it depends on how desperate you are. The work outlook isn’t great, and if you need a gig, you do what you have to do to find one.
So any type of this should take a couple of hours, but in reality takes two to three times that, will probably die.
This opens the door for my preferred type of interview. An actual conversation. An interview where the interviewer and the candidate just talk about software. The candidate gets hypothetical (or real) problems and has the opportunity to talk through how they would solve it, similar things they have build in the past, and ask some clarifying questions.
The most recent gigs that I enjoyed the most contained zero coding exercises. With one, the director of engineering simply said that he’s seen what he needed to. He could tell by talking to me that I could do the work. And that’s true. I’ve also done my share of hiring. And I know in the conversation if they know what they are talking about.
An example: Ask a candidate about some project on their resume. Ask them what issues they ran into and how they solved it. When they state the problem they had, start digging into how they approached a solution and different things that they tried.
If that candidate had a core part of the project and was actively making decision to push the project forward, they will be able to talk through it with little effort.
On the other hand, if they weren’t deeply involved in the process, they will have a really hard time talking through the solutions. And the reason is that they were not a core part of that project. It’s on their resume, but not real experience.
When a developer leans too heavily on AI, they don’t understand the problem/solution. Therefore they can’t speak in depth on the solution.
This all assumes you are looking for a senior. If not, then other questions like what did you learn from the more senior developers on this project would get you what you need. If a less experienced developer can’t tell you what they learned on a project, pass.