AM Research is a division of Aspiring Minds. Aspiring Minds aspires to build an assessment-driven job marketplace (a SAT/GRE for jobs) to drive accountability in higher education and meritocracy in labor markets. The products developed based on our research has impacted more than two million lives and the resulting data is a source of continuous new research.


A cocktail of assessment, HR, machine learning, data science, education, social impact with two teaspoons of common sense stirred in it.

An Automated Test of Motor Skills for Job Prediction and Feedback

We’re pleased to announce that our recent work on designing automated assessments to test motor skills (skills like finger dexterity and wrist dexterity) has been accepted for publication at the 9th International Conference on Educational Data Mining (EDM 2016).
Here are some highlights of our work –

  • The need: Motor skills are required in a large number of blue collar jobs today. However, no automated means exist to test and provide feedback on these skills. We explore the use of touch-screen surfaces and tablet-apps to measure these skills.
  • Gamified apps: We design novel app-based gamified tests to measure one’s motor skills. We’ve designed apps to specifically check finger dexterity, manual dexterity and multilimb co-ordination.
    amultifingermanual

 

 

 

 

 

 

 

 

  • Validation on three jobs: We validated the scores from the apps on three different job roles – tailoring, plumbing and carpentry. The results we present make a strong case for using such automated, touch-screen based tests in job selection and to provide automatic feedback for test-takers to improve their skills!

If you’re interested in the work and would like to learn more, please feel free to write to research@aspiringminds.com

Data Science For Kids Goes International

We successfully organised our first international data science workshop for kids at the University of Illinois as a part of SAIL, a one-day event to learn more about life on campus by attending classes taught by current students.
The workshop aimed towards introducing the idea of machine learning and data-driven techniques to middle-to-high-school kids. Participants went through a fun exercise to understand the complete data science pipeline starting from problem formulation to prediction and analysis.cssail
Special mention and thanks to the mentors, Narender Gupta, Colin Graber and Raghav Batta, students at the university who helped us execute the academic and peripheral logistics of the workshop efficiently and making the experience engaging and interesting for the attendees.

naren

colinraghav

 

 

 

 

 

 

Narender Gupta                     Colin Graber                          Raghav Batta

To read the mentor experiences click here.
Visit
sail.cs.illinois.edu for more information on the event or workshop.

What AM Research told you in 2015 – the data science way?

As the year came to an end, we looked back on what we shared with the world in 2015. As data nerds, we pushed all our blog articles in to an NLP engine to cluster them to identify key themes. Given the small sample size and challenges to find semantic similarity in our specialized area, we waded through millions of unsupervised samples through deep learning with a Bayesian framework, ran it on a cluster of GPUs for a month…yada yada. Well, for some problems it is just that humans can do things easily and efficiently; so that is what we actually did.

The key themes were:

Grading of programs – 4 posts

We need to grade programs better to be able to give automated feedback to learners and help companies hire more efficiently and expand the pool considered for hiring. We at AM dream to have an automated teaching assistant – we think it is possible and will be disruptive. Thus we dedicated 4 of our posts on telling you about automatically grading programs and its impact.

The tree of program difficulty – We found that we could determine the empirical difficulty of a programming problem based on the data structures it uses, the control structures used and its return type, among other parameters. We used these features in a nice decision tree to predict how many test takers would answer the question correctly, and we predicted with a correlation of 0.81! This tells us about human cognition, helps improve pedagogy and also helps generate the right questions to have a balanced test. And this is just the tip of the iceberg. Second, we approached the same by looking at the difficulty of test-cases and their inter correlation. We understood what conceptual mistakes people make and also got a recipe to make better test cases for programs and had insights on how to score them. For instance, we found that a trailing comma in a test case can make it unnecessarily difficult!

Finding super good programmers – Given these thoughts on how to construct a programming test and score it, we showed you how all this intelligence put together with our super semantic machine learning algorithm, we can spot 16% good programmers missed by test case based measures. Additionally, we also found automatically the super good ones writing efficient and maintainable code. So please say a BIG NO to test case based programming assessment tools!

venn

Reproduced from “AI can help you spot the right programmers”. It shows a test case metric misses 16% good programmers. Furthermore AI can help spot 20% super good coders

Pre-reqs to learn programming - Stepping back, we tried determining who could learn programming through a short duration course. We found that it was a function of a person’s logical ability and English but not did not depend on her/his quantitative skills. Interestingly, we found that a basic exposure to programming language could compensate for lower logical ability in predicting a successful student who could learn programming. A data way to find course prerequisites!

Building a machine learning ecosystem – 3 posts

Catching them young! We designed a cognitively manageable hands-on supervised learning exercise for 5th-9th graders. We helped kids, in three workshops spread across different cities, make fairly accurate friend predictors with great success! We think data science is going to become a horizontal skill across job roles and want to find ways to get it into schools, universities and informal education.

“Exams. I would take my exam results, from the report card of every year. And then I will make it on excel and then I will remember the grades and the one I get more grades I will take a gift” [sic.]

flashcard

Reproduced from datasciencekids.org. Whom will you befriend? Can machine learning models devised by high school kids predict this?

The ML India ecosystem – Our next victims were those in universities. We launched ml-india.org to catalyse the Indian machine learning ecosystem. Given India’s very low research output in machine learning, we have put together a resource center and a mail list to promote machine learning. We also declared ourselves as self-styled evaluators of machine learning research in India and promise to share monthly updates.

Employment outcome data release – We recently launched AMEO, our employability outcome data set at CODS. This unique data set has assessment details, education and demographic details of close to 6000 students together with their employment outcomes – first job designation and salary. This can tell us so much about the labor market to guide students and also identify gaps – to guide policy makers. We are keenly looking forward to what wonderful insights we get from the crowd! Come, contribute!

Pat our back! – 3 posts 

blog4-image

Reproduced from “Work on spoken English grading gets accepted at ACL, AM-R&D going to Beijing!”. We describe our system that mixes machine learning with crowdsourcing to do spontaneous speech evaluation

We told you about our KDD and ACL papers on automatic spoken English evaluation – the first semi-automated automated grading of free speech. We loved mixing crowdsourcing with machine learning – a cross between peer and machine grading – to do super reliable automated evaluation.

And then our ICML workshop paper talked about how to build models of ‘employability’ – interpretable, theoretically plausible yet non-linear models which could predict outcome based on grades. More than 200 organizations have benefited by using this model in recruiting talent and they do way better than linear models!

Other posts

On the posts off these three clusters, we told you about –
Why we exist – why we need data science to promote labor market meritoracy

– The state of the art and goals for assessment research for the next decade (See ASSESS 2015)

Our work on classifying with 80-80 accuracy for 1500+ classes

It has been an interesting year at AM, learning from all our peers and contributing our bit to research, while using it to build super products. We promise to treat you with a lot more interesting stuff in open-response grading, labor market standardizing and understanding next year. Stay tuned to this space!

Varun

Aspiring Minds releases AMCAT employment data at CODS 2016!

Aspiring Minds Research is pleased to announce that it will be co-organizing this year’s data challenge at CODS 2016, the annual top-tier conference on machine learning and data science organized by the Indian chapter of KDD.

ameo_2015

Undergraduates – performance and salaries
This year, we wanted data science enthusiasts to get a flavor of the kind of data we have and work on. We have released AMEO 2015 – a dataset on Aspiring Minds’ Employability Outcomes. which captures the academic and demographic information of engineering undergraduates giving AMCAT, Aspiring Minds’ battery of standardized assessments. What makes this dataset unique and rich is that it also has employment outcomes (annual salaries of students’ first jobs) along with standardized test scores.

Interesting questions
The answers to a lot of interesting questions possible lie in this dataset –

  • Can we predict the salaries a particular undergraduate would get on graduating?
  • Is the recruitment industry meritocratic – Do people with higher skills get paid higher? Or are there biases which don’t allow for these?
  • How important are English skills in getting a job?

and many more!

Participate and spread the word – 1000 USD cash prizes!
Interested in finding out the answers to these questions?
Take a stab at the data right away by downloading it from the contest website (mentioned below).

Get started right away and help spread the word and!
1000 USD cash prizes to those with the best submissions!

Contest website
http://ikdd.acm.org/Site/CoDS2016/datachallenge.html

As machines become intelligent, where does India stand?

Machine learning is the science of learning to do tasks by observing examples. It is transforming the world by enabling machines do all sorts of ‘intelligent’ tasks such as understanding images, human speech, predicting preferences, diseases and many others. With tremendous amount of data, interconnectedness, sophisticated algorithms and huge processing power in small devices, machines do things which were beyond their reach until recently. On the other hand, machines are still unable to do many tasks which humans do effortlessly, say understanding a story – this constitutes the next big challenge for machines, well, the humans that build these machines!

In some way, it has never been so exciting! Where should India be, as machines are becoming more intelligent? It is simple – it should be making the most of the opportunity. We need to participate and contribute in high quality research, innovation and also convert new results into effective business models.  The opportunity is global – the location of a digital business doesn’t constrain its market – a company in a Bangalore or a Gurgaon could serve the US market, the Europe market or even the whole world. Machine learning is not just a scientific or an academic pursuit. The economy and society can get great returns by the research and innovation in the area.

But are we there yet? Where are we placed in the global scene in both, academic and industrial research?

Read the full article here – http://ml-india.org/where-does-india-stand-machine-learning/

Tweets

Join us for Aspiring Minds new product launch event. Date:05 December 2017 | Place - The Grand Hyatt, Mumbai. Block… twitter.com/i/web/status/9…

Team AM & #ESSCI exhibit vocational & motor #skills assessments at @MSDESkillIndia National Entrepreneurship Awards… twitter.com/i/web/status/9…

Demand of Spoken #English #skill across job roles. Philippines National Employability Report 2017 Read-… twitter.com/i/web/status/9…

"Migration musts" - Robust #skills #assessment AND International Skill Certificates says @himanshu0820 @skillbharat financialexpress.com/india-news/why…

#AMNERPhili, the first #employability study on grads of #Philippines. How job-ready are the youth of Phili? Read on… twitter.com/i/web/status/9…

"Tamil Nadu ranks 4th in job openings in India" reveals AM's #skillmapindia Read report by @timesofindia #skills bit.ly/2z1y3h9

Get privy to amazing facts on employability & #skills of graduates in Philippines Read full report here #AMNERPhili aspiringminds.com/philippines-na…

Bots and humans will need to work as team as their #skills complement each other~@varaggarwal #automationtwitter.com/i/web/status/9…

#AMNERPhili launches at Contact Islands 2017 @ccapph today! Watch this space for more updates. #ContactCenter #BPOtwitter.com/i/web/status/9…

#Software developers and sales ppl most-in-demand by Ind employers Skill Map - India, 2017 Read on… twitter.com/i/web/status/9…

65% Philippines grads lack right job skills- AM's Philippines National Employability on 12 Oct@#ccapph #AMNERPhili news.abs-cbn.com/business/10/05…

Catch @varaggarwal share a sneak preview of 1st ever '#Philippines National Employability Report' @ #ccapph Stay tu… twitter.com/i/web/status/9…

#skillmapindia 2017 is a #BigData view of the top #jobs and #skills preferred by employers across the country. pic.twitter.com/inRoR0TUbz

Twitter Media

AM's #SkillmapIndia reveals #SoftwareDeveloper #Jobs top dd chart in Corp India. Eng more desired than Logic&Maths… twitter.com/i/web/status/9…

Aspiring Minds launches #SkillMapIndia 1st ever interactive platform showing #jobs & #skills dd in India #SkillIndia business-standard.com/article/pti-st…

Hear @varaggarwal CTO & Co-Founder Aspiring Minds speak at #DigitalHaryanaSummit #DigitalTransformation Haryana twitter.com/nasscom/status…

AspiringMinds #NER shows glaring reality of India's #employability quotient Triggers @NITIAayog 3Y action plan #jobs bit.ly/2xNLyBq

Serious #training intervention needed in #skills which #Nursing assistants lack. #healthcare #jobs #careertwitter.com/i/web/status/9…

Team @AspiringMindsAM thanks our honored guests for gracing #AMNEC2017, Pune Chapter. @sanjyotkhare @sonyadctwitter.com/i/web/status/9…

Using #datascience to assess employees and find the #right organizational fit: Sushant Dwivedy @AspiringMindsAM at… twitter.com/i/web/status/9…

Rajat Mathur-VP Client Solutions, Aspiring Minds addressing #career office heads at #AMNEC2017 #Pune @myamcat #jobstwitter.com/i/web/status/9…

The stage is set! Aspiring Minds kick starts National Employability Conclave 2017 | Pune Chapter #AMNEC2017twitter.com/i/web/status/9…

Sushant Dwivedy, Head, Enterprise Client Solutions @AspiringMindsAM addressing #IBPAP HR Summit #AMIBPAP pic.twitter.com/QBfxOd7gWo

Twitter Media

Very interesting discussions on Day 1 #IBPAP HR Summit 2017. #AMIBPAP #HRTech #talent #disruption pic.twitter.com/msBuVjR5yj

Twitter Media

Team AM at #IBPAP #HR Summit 2017 #Philippines. The stage is set for some interesting conversation around humanizin… twitter.com/i/web/status/9…

Just 1 day to go! Hear Sushant Dwivedy, Head Enterprise Client Solutions, Aspiring Minds speak at #IBPAP #HR Summit… twitter.com/i/web/status/8…

45% #nursingschool graduates lack basic medical knowledge | Skills Nursing Assistant -2017 by Aspiring Minds #skills indiatoday.intoday.in/story/45percen…

Consolidation of #edtech in #India will see a few select players emerging as leaders - @varaggarwal @livemint bit.ly/2wvQyMW

Aspiring Minds at #IBPAP #HR Summit 2017. Watch this space for interesting conversations around #talent quality,… twitter.com/i/web/status/8…

80% employee turnover is due to bad hiring decisions @HarvardBiz Soak up sum really gr8 hiring & #InterviewTipstwitter.com/i/web/status/8…

Aspiring Minds' predictive #assessment algorithms improve recruit efficiency by over 35% Read on @livemint @myamcattwitter.com/i/web/status/8…

95% engineers are unable to code @EconomicTimes basis Aspiring Minds' National Employability Report #Engineering bit.ly/2hfV4ZH

#AspiringMinds Employability Report of 2016 - 80% engineers unemployable. Read more in #financialexpress here bit.ly/2vWLNsM

#AMCAT partners with @BrighterMonKE for skill #assessment #AFRAMCAT @AfricaHRLeaders @India_HR_Forum twitter.com/BrighterMonKE/…

Looking for a candidate with exceptional lang skills? Find easily with #AMCAT English test #AFRAMCATtwitter.com/i/web/status/8…

Clues for interviewers to gauge candidate's commitment #AFRAMCAT #Commsweek @247ureports skill #assessment #HR bit.ly/2v3lZOv

Get the 'right mix' of logic, reasoning and attention to detail with #AMCAT #AFRAMCAT #Commsweek @247ureports skill… twitter.com/i/web/status/8…

Do you need people with strong logical ability? Try #AMCAT #AFRAMCAT #Commsweek @247ureports @channelstv skill… twitter.com/i/web/status/8…

Mohinder #jimmy Amarnath sharing some awesome perspectives from his golden cricket days at #nasscomHRSummit #teamAMtwitter.com/i/web/status/8…

Day2 at #nasscomHRSummit . Looking forward to some wonderful conversations #teamAM #innovation #hrtech #Leadershiptwitter.com/i/web/status/8…