Iceland took the lead in the 18th minute as Jon Dadi Bodvarsson finished with a close-range shot.
Bodvarsson sent the ball to the left bottom corner of the net, giving Austria goalkeeper Robert Almer no chance.
Austria was awarded a 37th-minute penalty as Bayern Munich midfielder David Alaba was pulled down in the box but defender Aleksandar Dragovic missed the equalizing chance.
Austria equalized in the 60th minute, as Schalke midfielder Alessandro Schopf beat Hannes Halldorsson to make it 1-1.
But Iceland had the last word, claiming victory in injury time as defender Arnor Traustason slid the ball into the Austrian net to leave it 2-1.
Iceland booked their ticket for the knockout stages, alongside group leaders Hungary. The tournament newcomers will face England.