The 17-year-old, who was born on 6 January 2000, reduced the arrears in Hamburg's defeat in the capital to become the northern Germany club's youngest ever goalscorer.
"He's becoming more and more of an option, each week he spends with us," said Hamburg coach Markus Gisdol prior to his side's trip to the capital. His team had scored just two goals in their last seven games, until Arp came along.
The precocious forward scored five goals in as many games for Germany's Under-17s at the recent FIFA Under-17 World Cup in India, after scoring 26 goals in 21 games for Hamburg's similar-aged selection last season.
He became the first player born in a year starting with the number two to appear in a Bundesliga game when he featured in the Nordderby against Werder Bremen last month, and he has now etched his name in the history books again with his first goal.
"The lad's got everything, really everything," said former Hamburg legend Horst Hrubesch last month. "I've rarely seen a player who can score a goal out of nothing like he can."
Viewers of Hamburg's trip to Hamburg on Matchday 10 will be able to vouch for that after Arp opened his Bundesliga account, becoming the first millenial to do so.