Stewart's spot-kick winner seals FA Cup progress.
Sunderland came from a goal behind to defeat Crystal Palace at the Stadium of Light in the FA Cup third round back in 2005.
The Black Cats struck through Andy Welsh and Marcus Stewart after a Neill Collins own goal had given the Eagles the advantage.
Mick McCarthy’s side dominated their top-flight opposition from the off as Welsh clattered the bar from close range and George McCartney headed just wide in the early stages.
Palace grew into the game and went close through Vassilis Lakis before they eventually took the lead just before half time.
Collins left Thomas Myhre in the Black Cats goal with no chance as he poked the ball into his own net from Wayne Routledge's cross with Andy Johnson looming.
Welsh replied just minutes later on his home debut as his hopeful free-kick curled past everyone and nestled in the net.
Stewart grabbed the winner on the hour after he was hauled down by Fitz Hall in the penalty area and he cheekily dinked his spot-kick home.
Myhre then denied Johnson with a good save but it was Sunderland who finished the stronger despite their advantage with Sean Thornton twice hitting the bar in the final ten minutes.
Sunderland: Myhre, Wright, Caldwell, Collins, McCartney, Thornton, Whitehead, Robinson, Welsh (Piper, 75), Elliott (Bridges, 83), Stewart (Brown, 69).
Subs not used: Lynch, Alnwick.