Our dataset is gathered by using a new representation language to annotate over the AQuA-RAT dataset. AQuA-RAT has provided the questions, options, rationale, and the correct options.
• Question: A train running at the speed of 48 km / hr crosses a pole in 9 seconds . what is the length of the train ?
• Rationale: Speed = ( 48 x 5 / 18 ) m / sec = ( 40 / 3 ) m / sec . length of the train = ( speed x time ) . length of the train = ( 40 / 3 x 9 ) m = 120 m . answer is c .
• Options: a ) 140 , b ) 130 , c ) 120 , d ) 170 , e ) 160
• Correct Option is: C
The rationales are noisy, incomplete and sometimes incorrect. We correct these rationales and provide stepwise solutions for a portion of AQuA-RAT.
• Our Annotated Formula: multiply(divide(multiply(48, const_1000), const_3600), 9)
We have designed an annotation platform through the Figure Eight platform. We have defined a representation language that can capture the steps that are necessary for finding the solution to the problem. Each step in our final program is the predefined operation and its arguments, which can be chosen from the problems, a list of constants or previous calculations. Here is the overview of our job:
Figure eight provides unique features that help us achieve data that is clean, reliable and accurate. The task of annotating math word problem is especially complicated and time consuming task. The points that should considered are :
The results of the annotation task are not all correct. There are cases that a logically wrong program would get us to the final answer. Therefore we designed a validation task. Here are the steps for that task:
The final answer is acceptable when the currect calculation result is the same as the correct option and at least one number from problem is used. The Result would automatically copied to the requiredd result field which let the contributors to submit their results.Close