HackerRank is an online coding platform. It provides coding tests and questions for companies to screen candidates.
We remember the first time we had to do a test (before joining the company), unsure what were the expectations. Later, we were designing new tests (after joining the company), unsure what to expect from candidates.
We decided to release some insights on our experience, full disclosure. How good people are doing? How the test is evaluated?
Hopefully, that will give everyone a better understanding of what is going on.
Statistics
Do or not do, there is no try
We invited 79 people to do the test in the last month… 29% of them never tried.
On the bright side, the more candidates who kick themselves out, the more time we can dedicate to the remaining ones.
You can be a top 71% performer by simply trying! =D
Details
We inaugurated a new test last week and 5 candidates did it over the weekend. They happen to be a representative sample:
- Didn’t attempt any of the coding exercises
- Answered all coding exercises with “return true” or equivalent algorithm.
- Answered exercises not with code but with comments about the train’s Wi-Fi being terrible, especially after the train started moving
- Had trouble to solve the SSH-to-our-server exercise without sudo, until he hacked the webserver with a fresh 0-day to elevated his privileges.
- Answered all simple questions with simple algorithms, didn’t finish the hard one.
Three failed and two passed. It’s self-evident who is who.
Highest bang for the buck
There is no other form of screening that can scale as well as HackerRank. It is also the fairest interview process since it never discriminates on age, race, years of experience, school or anything.
Designing the test takes a few day.
We pay $5 per invitation and the correction takes 5-15 minutes.
Hall of Shame
Internet is required to complete the test
One candidate tried to do the test on a laptop, in a moving train, over the train’s Wi-Fi. It didn’t go well and he sent us a long email to complaint right after the test.
On the bright side, he wrote long comments in English. On the dark side, he didn’t code any of the simple things (not requiring internet or any documentation) and all the writings prove the internet connection was not that bad.
We considered about giving him a second chance and then we just dropped the case after much confusion and more emails.
Did he think that internet is unnecessary to access http://www.hackerrank.com? Is the connectivity usually good in train? Does he do the same thing for Skype interviews? We don’t know and we’ll never know. We are still puzzled to this day.
We’ve added a note to our introductory email to clarify: “Internet access is required, for the whole duration of the test“.
“return true” is NOT the ultimate answer to everything
We are seeing a lot of stupid answers. Probably just to grab some points.
Class Solution {
// str : firstname|lastname|phonenumber|address|zipcode|country
bool filter(String str) {
return true;
}
}
int max(int array[], int size) {
return array[0];
}
Booleans are about 50-50 by the law of probability, integers can get lucky with 0 or -1, arrays with the first or last element.
Passing 50% of tests is good value for the time invested but it won’t survive a code review. (Not to mention that 80% of the point could be on the harder test cases).
Tip and tricks for candidates
Complexity
As a candidate, you cannot see the unit tests content, the edge cases or the complexity expected.
The question gives bounds on the input size. The title and tags gives a hint about the expected solution (e.g. dynamic programming). Read that wisely.
64 bits integer
Many questions require 64 bits integers but it’s NEVER mentioned. Go for 64 bits integers as default whenever there is an array with thousands of integers and some additions (e.g. all trading-like and number-crunching questions).
Unit Tests
The unit tests are NOT ordered in ascending difficulty and they may have limited variety.
For instance, if there are 8 tests (excluding examples), that could be 4 tests with 64 bits results + 6 tests with 50 MB of input data + 1 test with a single number.
A slight difference in complexity or an unhandled edge case may turn around many tests.
Timeouts
A test case has between 1 and 5 seconds to be run (depending on the language). A “timeout error” on a test means that it didn’t finish in the given time and was terminated. Gotta write faster code.
All your code is reviewed
On the recruiter interface, we can see the code that was submitted, we have the input and the output of all test cases. Including errors and partial output.
We review everything, we evaluate algorithms, we evaluate complexity, we read comments, we consider special hacks/tricks, we check edge cases.
Points
HackerRank gives points per question and per unit test successful. We get a general sense of completion when we open the review windows “x/300 points” but ultimately the decision comes down to the code review.
Time Spent
We have an overview of the time spent on the test.
HackerRank is simple
Whatever a test contains, the candidate will usually advance to the next round if he can answer some of the coding exercises.
A developer should be able to code some solutions to some [simple] problems. That’s exactly what HackerRank is testing.
HackerRank is good for everyone
Once in a while there is a company with a crazy impossible test that is rejecting everyone. The company would do the same thing if it were face-to-face. You just avoided an awkward 4h on-site interview.
Sample Test
There is only one important thing to do before attempting a test. Try the the sample test to familiarize yourself with the platform and ensure everything is working.
Conclusion
Recruiting takes a huge amount of effort on everyone involved. HackerRank’s purpose is to save a lot of time and effort by weeding out people earlier [especially utterly unqualified people]. Most of these would fail in the same way in a phone or face-to-face interview.
It’s good and it’s extremely effective. It can replace the initial phone screen.
The HFT Guy 