Hurray! Our paper “Grading uncompilable programs” has been accepted at IAAI 2019! This is the first attempt to provide semantic feedback on uncompilable codes. This is our fourth paper on the topic as we have improved technology to grade and provide feedback on programming skills.
Automata, the world’s only AI-based evaluator of programming skills, is being used across the world to hire software engineers. In our earlier post, we discussed how a US-based company used the platform to improve hiring efficiency.
In our IAAI paper, we mention a study with a Chinese company. We find that the system helped in selecting around 26% more candidates for the interviews. These candidates had written a logically meaningful code, but the code did not compile. Out of these candidates, around 19% of candidates were hired. This is a big win as many worthy candidates who would have got missed out by traditional program grading systems were hired. (See details below)
|1||Code unrelated to given task||3361||1979|
|2||Appropriate keywords and tokens are present||3264||2125|
|3||Right control structure exists with missing data dependency||2547||1440|
|4||Correct with inadvertent errors||2955||1017|
|≥ 3||Selected for interview||9330||2457|
Recently, the system has been used by a large IT company in India for large scale hiring of entry-level software engineers. The company hires 1000’s of software engineers every year and struggles with filling all open positions. By way of grading non-compilable codes, they have been able to improve hiring throughput by 30-40%.
Stay tuned, more to come… Programs + AI opens up a million opportunities!!!
- Rohit and Varun