As someone who's recently been through the ringer, including one six hour take-home, all I want is a clear demonstration of respect and rationality.

Bring me into your professional office. Let's talk like professionals. Allow me to demonstrate my professional skills. Call me back with a professional yes or a professional no, all within a professional time frame. That's it.

Remote or otherwise less traditional assignments / roles / jobs will deserve and benefit from their own process. But how we strayed from the straight forward formula is beyond me, my guess it was an initiative started by a handful of companies who had trouble hiring.

I don't think hiring devs is the systematic issue everyone makes it out to be. Assessing talent is always hard, be it an artist's, an athlete's or a programmers. I think rather then assume the cost of the investment in hiring, companies chose to blame the system and that's where this absurd roller coaster started.

What if your professional skills are such that it's not possible to demonstrate them in an hour or two at someone else's office--either because of the nature of the skills, or because you're one of the many excellent programmers whose work suffers when there's someone actively staring at you, like the guy in the article?

Let's be honest. Programming as a discipline attracts a disproportionate number of people for whom social skills do not come easily. Granted, you don't want to hire a grumbling misanthrope who refuses to take direction, but you also don't want to turn away a perfectly good team player who lacks the largely irrelevant skill of gladhanding under pressure.

I think a lot of the comments in discussions like this come from programmers who do have solid social skills, and there's certainly nothing wrong with that, but an interview process that gives you personally a fair evaluation is not necessarily the most reliable one for programmers in general.

It isn't but it strikes me we are looking very hard to find a new way to do things, when the old way was pretty damn good.

Sit me down and talk about technology for ~thirty minutes. If I don't have the social skills to successfully do this (minority issue) I likely would not be able to communicate well with a team and thus should not be a candidate anyways. You will then know immediately whether you want to hire me (or progress me to another round) or not.

I then get a call two days later and can progress with my life.

People pretend like all this hiring strategy is for the good of the candidate. It's not. As with everything else its for the good of the company and its investors.

Here's a thought:

Invest in a competent hiring manager who can see through applicant bull shit and identify talent within a reasonable range. Includes basic negotiation skills. And assume the rightful risk that is employing another human being.

This is an efficient way to hire a team of good bullshitters. I've interviewed people who did extremely well when we were "talking like professionals" but were unable to do even very simple coding problems.

My bar for coding is really not that high. I don't expect perfect syntax. I don't pick the language. I don't expect "the one answer". I expect people to write code that could work after syntax and small bugs are fixed, and most critically, I expect people to be able to discuss their code meaningfully.

Unfortunately, "ability to write basic code" and "ability to talk like professionals" are not tightly correlated. And I expect both of these things from dev candidates.

Coding is a skill largely separate from other critical dev skills, e.g. design. Someone can legitimately have deep knowledge of, e.g., how to build a 3-tier service architecture without actually being able to code. This isn't "faking it". It's a different skill. You can study database partitioning strategies and learn about the latest and greatest frameworks for writing web pages and APIs without ever writing a single line of code. And sure, you can try to probe deeply enough to expose where their knowledge stops, but you're still not probing for coding skill. There's actually a decent chance that you'll end up probing for trivia which is not a very useful filter. "Oh, you don't don't know the auto-generated C++ class members? You clearly haven't actually worked in C++!" Yeah, right. Anything known broadly-enough known that it's reasonable to assume all coders would know it is broadly-enough known that non-coders could learn it, too.

Even if you could probe deeply enough in conversation to expose that they can't code, you could do the same with a few minutes of actual coding. When someone can't code, it becomes clear pretty quickly when you ask them to code.

I do not believe coding is largely separate from engineering. There are, of course, other skills that are necessary or desirable.

> Someone can legitimately have deep knowledge of, e.g., how to build a 3-tier service architecture without actually being able to code. This isn't "faking it". It's a different skill.

Absolutely. It is also a level of skill I would only expect or look for in a position that was not going to be doing daily coding, such as a systems engineer.

> You can ... learn about the latest and greatest frameworks for writing web pages and APIs without ever writing a single line of code.

Here I disagree. If you have never used a framework or API, you don't know it. I don't care that you can regurgitate the Javadocs; I care that you know things like its pitfalls, quirks, and runtime oddities.

> There's actually a decent chance that you'll end up probing for trivia which is not a very useful filter.

To be clear, I abhor trivia-based interviewing. What I am talking about is practical engineering tradeoffs between one course of action versus another, preferably supported by direct personal experience in the past.

> Even if you could probe deeply enough in conversation to expose that they can't code, you could do the same with a few minutes of actual coding. When someone can't code, it becomes clear pretty quickly when you ask them to code.

Exposing the level of competence a five-minute coding problem can is trivial to do in parallel with a deep engineering discussion. In fact, I think some sort of code should be part of that discussion. I just don't think it should be the "implement this" CLRS problem. It should be something two-way, more representative of what the job is like on a daily basis.

I could pick up a cookbook and study how to cook a souffle. I could learn enough about this academically that I could answer basically any questions you might ask. Having this academic knowledge is a good thing, but it doesn't mean I've ever even separated an egg. If you really want to know if I can make a souffle, your best bet is to hand me the stuff and ask me to do it.

> Here I disagree. If you have never used a framework or API, you don't know it. I don't care that you can regurgitate the Javadocs; I care that you know things like its pitfalls, quirks, and runtime oddities.

And here I assert that you're either probing on trivia or probing on things that can be learned without using the framework (actually, probably both). The pitfalls, quirks, and oddities are well documented in thousands of blogs.

> To be clear, I abhor trivia-based interviewing. What I am talking about is practical engineering tradeoffs between one course of action versus another, preferably supported by direct personal experience in the past.*

So this is an entirely different thing than pitfalls. Now you're talking about designing systems and dealing with tradeoffs. This isn't coding, and I don't believe you can always discover coding gaps by probing on this.

> Exposing the level of competence a five-minute coding problem can is trivial to do in parallel with a deep engineering discussion. In fact, I think some sort of code should be part of that discussion.

I guess I'm confused about what you're asking in an interview, then. Are your candidates coding or not?

> It should be something two-way, more representative of what the job is like on a daily basis.

I think it's unrealistic to try to get the candidate to solve real-world, day-to-day problems in an hour. This isn't what the job is like. "Design and implement this feature" is not a 45-minute task typically. It's typically days or weeks, so asking a "representative" problem in an interview is infeasible unless you're just doing high-level design, in which case it's not predictive for coding ability.

When I was writing signal processing code, we had four levels of engineering.

- The first was an Algorithm Description Document. This document laid out, in mathematical terms, the algorithms used for various signal processing functions in the system. It was purely conceptual.

- The second was an Algorithm Implentation Document. This document mapped the algorithms to specific parts of the hardware and software system, laying out the logical module structure and data flow. It also specified how the algorithms would be realized in code, since parts of a particular algorithm might need to run in different parts of the system. The AID was still primarily mathematical.

- The third was a series of software design documents. These documents specified the details of module interfaces, code and file layouts, and specifics about what algorithms would be implemented in what functions.

- The fourth was the actual translation of the mathematical algorithms in the AID into actual C code. Most of the engineering at this level dealt with function-level optimization and some platform-specific stuff.

Of those four levels, only the last is what I would consider "coding". It is also the least important. Anyone who can really understand the first three levels and is willing to learn can handle the fourth. As I mentioned previously, there will be differences in efficiency, but not in capability. It would take willful ignorance for this not to be the case. I think this is also what you are considering "coding", but I'm not sure; I also think you might be merging #3 and #4.

Now, most commercial projects do not run this way and, as far as I know, few have dedicated systems engineers to manage the system architecture and APIs. Thus, the software engineers building the system generally perform the work in all four of these levels simultaneously as the system builds out. Despite this, I still only consider the work that fits level 4 to be "coding", and still consider it to be the least important.

What I look for when I interview are engineers who operate very well in levels 1 - 3. If I find that, level 4 is pretty much a given absent the rare pathological case or someone with a very "academic" attitude. The questions I personally ask deal with levels 2 and 3. Those questions cover some of what might be considered coding, like API design and class layout. I do not have candidates actually generate code, though. Some of the other interviewers have candidates write small functions in pseudocode or explain existing, uncommented Java code, though.

I also need to note that we have a take-home test that candidates need to pass before they get an on-site.We have also recently added an open coding challenge which may eventually replace the take-home test. This probably covers your requirement for candidates to code, even though it isn't done in front of one of us.

* People who read a lot of blogs so know the lingo, but never code

* People who get the theory but can't actually implement something that works without bugs

* People that over-engineer simple things

I think this is where my definition of "talk like a professional" may be more strict. I'm interested in understanding, in opinions, in the actual engineering.

> People who get the theory but can't actually implement something that works without bugs

Having spent most of my career working with PhDs trying to design and implement signal processing and NLP algorithms, I have experienced a lot of that. Given the proper attitude, it is fairly easy to correct coding deficiencies if someone really understands what they are supposed to be doing.

> People that over-engineer simple things

The greater offenders are fairly easy to detect in a directed technical discussion. The lesser offenders are fairly easy to correct in code review, again given the proper attitude, as they are unlikely to be doing things that require major refactoring.

In short, I think I place much more emphasis on engineering as opposed to coding. My experience has been that any coding issues people with good engineering skills and an attitude worth hiring have are quickly and easily corrected. You cannot really understand the engineering of a system without being able to code it. You might not be as efficient as someone else, but you are capable.

* People who have been on a team long enough to learn how things should work but lack the skills to really do the coding.

I phone screened a guy fairly recent who had a very long background (~20 years) in a pretty security-conscious field. He did very well in open-ended design discussions and was able to explain his prior work very well. He couldn't code trivial binary tree operations, though. I can only assume that someone else on his team is carrying most of the coding burden. This guy might be great at a PM-type job (or not, I don't interview for those jobs), but not as a dev.

Problem with that is it's hard to find one. It's gotta be someone with pretty good coding skills. This person would then have to share their time between either coding head-down or managing coders, either of which are time-consuming and intolerant of disruption.

One idea I had was you could have mutual feedback. A bunch of people would after a while have anonymous input on each other, giving you say 10 coder's impressions of a fellow coder. This wouldn't have to be quizzes, half hour coffees could do.

Haha, exactly when was this 'old way' ever reality?

- never hear back

- automated email 3 weeks later

The first one, they asked me to solve an incredibly complex math problem that I had no idea about so I struggled with it for 5 or so hours before giving up. Probably good that they didn't give me the job; if they were expecting me to know the level of math they were requesting then I would have had a pretty bad time. So I guess the test worked in that regard.

The second one interview at a different company, they gave me more moderate questions after a phone interview- mostly questions testing knowledge of a specific language. When I asked how long it would take, I was told "you have a deadline of 8 hours from when you download the zip file to when you turn in your code, expect it to take the entire 8 hours". That was probably the most stressful 8 hours of my life, way worse than any whtieboard interview would have been. I didn't look away from my computer once the whole time, did not let myself get up to eat, go to the bathroom, or anything. My girlfriend was in the room and tried to console me while I was doing it and stressing out and I ended up yelling at her to be quiet because I was trying to hard to concentrate. I finished the last problem about 10 minutes before the deadline, zipped the files up, and sent it to them. I never heard back.

Favorite interview I've ever had was a combo of a live coding session and a phone interview. They used an online tool that they gave me a link to that had syntax highlighting. The interviewer on the phone could see where I was typing and could highlight stuff etc. Think google docs, but for coding. They asked me to write some algorithms, then asked me to re-write them recursively, then asked a bunch of questions about the limitations of the algorithm, cases when it would be used, etc. It was fun, not stressful in any way, and the interviewer even taught me some things along the way! I feel like they got a really good picture of my thought processes and my personality so they could accurately judge if I would be a good fit on the team.

Paying somebody to do the take-home coding assignment and paying them would definitely help alleviate some of the stress, but I'm still not a fan of the format. You don't know how long it took the person to make their solutions, and you don't get a good idea for how they work, only what they can do (both are equally important, in my opinion).

1. A very loose time limit. Maybe, as a rule of thumb, at least 10x the time you expect the challenge to take (so, a 1-hour challenge is given a 10-hour limit to turn it in, and a 5-hour challenge is given two days). If you expect it to be done in real time, why they heck don't you just bring the candidate in for a normal interview?

2. If the challenge is expected to take more than, say, 5 hours, the candidate should be paid for their time. Long challenges are going to disincentive candidates with lots of good alternatives to your company, so compensating them as in your own interest if you want good engineers to make it through your process. Plus, it's a strong signal that you value their time.

3. The challenge should be as close as possible to the sort of thing you do day-to-day. This should be obvious, but don't ask math questions or graph traversal questions unless this sort of thing comes up regularly in the job you're hiring for.

4. The candidate should be given an opportunity to discuss their solution. In real-life code, a seemingly stupid decision is may well be totally justifiable, but not obviously so from the code itself.

5. The candidate always gets an explanation of why they were rejected. At one company I worked at, I was in the position of reviewing takehome tests (which were one component of a longer process which involved some on-site coding). I had to write those explanations, and I know it's hard. It's easy to say "your skills are not a good fit at this time", but it's hard to tell someone you don't know what's wrong with something they put a lot of work into. It's still worth doing.

Please name these idiots.

1) I didn't score well on the test. It's within reason they read this post and release my scores with a 'it was because he scored low'. Future employers then come across my scores and deny me.

2) Don't bite the hand that's like the hand that MIGHT feed you. You don't yell and storm out of investor meetings. Because then they tell other investors. Similar logic here.

3) This place is very optimistic. Pessimistic people aren't particularly desirable. Whistle blowers are included in the latter group especially by corporate entities. Who knows what it could cause.

It really boils down to the fact that it's not a safe environment to do so. Or at least it doesn't feel so.

I think you're playing the game of life with your scared setting turned up too high. You might get better results to turn the bold up a bit over time as you gain confidence.

My take-homes made me fundamentally question my ability as a programmer. And I'm a damn good programmer. That is wrong on so many levels.

I can only pledge that if I'm ever in a position to hire developers (at any level) that I'll never force that kind of experience on a potential applicant.

I feel their is a major gap between junior and senior developers. And in this environment a junior developer has zero leverage with the companies he/she applies to. So they treat us like cattle.

It's like with every generation gap, their isn't much point blaming those who came before us. We can only do the best we can and make an effort not to replicate their mistakes. The next class(es) of software engineers have to improve and be better then the ones before them.

I'm preaching, but I hope you get what I mean.

There's a gap between knowing concepts and knowing how to apply those concepts. There's actually a certain (limited) amount of artistry to writing code. Hiring a programmer without seeing code is like hiring an Architect without ever having seen a house he's designed.

You don't need an 8-hour take-home coding exercise to determine whether or not someone is capable of writing working code.

Like FizzBuzz I think the take home worked well as another data point. It easily filtered out people who couldn't follow basic instructions, people who were clueless about Java even though their resume said different, and people who simply didn't care. The problem also gave us something to talk about when the candidate came in for the in person interview.

Overall the problem was dead simple for anyone we would actually want to hire.

I like this, not for the money, but because it guarantees that the company is not giving this assignment to 100 different candidates ($500 x 100 is getting prohibitive for a hiring budget). I'm more willing to put the effort if I know I'm one of the "finalists" for the position.

Downside is that, as a company paying someone to accomplish a task, I'd want to give a different task to each applicant, so that if they do a good job, it provides value to the company.

And this is a problem because rarely are two programming problems exactly as difficult as each other, so the test isn't normalized. Interviewing is hard enough when you keep all the variables that you can control the same.

That's less than you're already spending on salary for the interview slate, so you're at most doubling your already small costs for a critically important function for your company.

I don't think I would give out the same work to multiple people, let them put it into production on their first day and get the pride going ;)

before i produce work for free its good to know they are actually considering me and not just throwing me work as another 'step' in their process.

Just as an aside, I would actually be more willing to undertake this type of interview if you weren't paying me. I don't have any ethical qualms about interviewing while working a full-time engineering job, but I'm not entirely comfortable taking a contracting job under the table. It may be something of a symbolic gesture given that it's such a small time investment and small amount of money, but it feels fundamentally dishonest, especially if you're a current or plausible future competitor to the company I currently work at.

The official policy where I currently work is that moonlighting is allowed but must be approved by our legal team, and while I could probably get away with not asking for permission, I'd be pretty uncomfortable in both cases (asking, or not asking).

Sure, and I don't think I would force pay you. Would offer you the in person option, or the homework option without pay. Could maybe work it into a signing bonus if you took the job or something? Definitely not looking to put someone into an ethical dilemma, but not wanting to be one of those companies who makes candidates do really long take home interviews and then makes them mad when they don't extend an offer.

Yes, and if I copy that resume and put my 6 year old's name on it, then her resume clearly shows she is a skilled programmer.

I'm at a point in my career where I mostly refuse take home tests because I'm busy and I value my free time. Getting paid would make me much more willing to jump through hoops. Particularly when that day of vacation cost me $350 after taxes, so it's not like I don't have skin in the game already.

So, if you are interviewing at 20 companies, you need to expend 100+ hours or almost 3 full work weeks?

It should take 1 hour or less. Period. Probably less that 1/2 hour. I doubt I get much more information asking you for a 5 hour assignment than a 1/2 hour assignment.

If I'm really that interested in your programming on the fly, I should do it at the interview where I'm paying for your lodging, food, etc. I should tell you what you are going to be doing, and to bring your laptop set up to do that.

That's a lot of companies to entertain! When I was interviewing, I could barely maintain the composure to interview with 3 companies! (Only one of which had a take-home assignment.)

I have never interviewed at 20 companies when I needed a new position. If you do, cool - but not sure I want to simplify my hiring process to help someone mass interviewing.

If we have a conversation and we both like each other, let's see if we can come to terms we are both happy with. If we are playing other companies off each other for 1k raises, it seems not really worth the hassle.

I share this opinion, but only for certain ways of doing it. If it's done in a fully automated way, before even having a phone screen, then it's too easy to waste my time. As an alternative to the whiteboarding interview, however, it could be great.

You get paid that much as a freelancer when you produce value for the company. So by doing an extra assignment, especially one that is a useless piece of code, no - $200/h is ridiculously high price.

Agreed.

Annual billable hours for consultants, be they developers, management, lawyers, et al, are considered huge if near 1500. That roughly works out to just under 30 billable hours per work week when vacations/holidays and non-billable time are taken into account.

As you surmise, just because someone _can_ bill X/hour doesn't mean that each hour they breathe commands the same remuneration.

I'd much rather freelance for that type of stress & when having to compare the economics time-wise, which a take home test simply does not match - that is the whole point.

If a company does not want to pay that much, maybe they should respect the prospective employee's time, especially since it is a job seeker's market.

If you get to the paid homework step, we are ready to hire you, pending successful homework.