The 23-year-old, who helped Stefan Kuntz's side win the UEFA European Under-21 Championship earlier this summer, has signed a five-year contract with Dortmund.
Toljan made 20 league appearances for Hoffenheim last season as Julian Nagelsmann's charges achieved an historic fourth-place finish.
"He can play on the right and the left of defence and was a star of the German U21 team's success in the summer," said Dortmund sporting director Michael Zorc.
"He is a player with huge potential to develop."
Toljan joined Hoffenheim's youth set-up from Stuttgart as a 16-year-old in 2011. He made his first-team debut for the club in 2013, going on to make 60 appearances in all competitions for the Sinsheim-based club.
The second part of the deal sees Passlack join Hoffenheim on a two-year loan, as well as the 19-year-old Dortmund talent signing a contract with the club until 2021.
The former winner of the Fritz-Waler medal has made 14 Bundesliga appearances for Dortmund.
"We hope Felix is able to have as much playing time as possible and believe he is in good hands with Julian Nagelsmann," Zorc concluded.