Bielsa's first win in five meetings against Pep Guardiola propels Leeds up to ninth in their first season back in the English top flight for 16 years.
Guardiola is a very public admirer of Bielsa but it was the 65-year-old Argentine's turn to praise his opposite number on Thursday.
Victory takes Leeds United above Arsenal in the table and to within five points of sixth-placed Liverpool as they push for a European place in their first season back in the top flight for 16 years
Leeds, who returned to the Premier League after a 16-year absence, are 12th in the table after a disappointing run of three defeats in their past four matches.
Since the Football League's fourth tier was added in 1958, this was only the second time a top-flight side had lost by three or more goals to opposition from that level in the FA Cup.
Leeds approach the busy festive period beginning with Sunday's home game with Burnley in 14th place, seven points above the relegation zone.
Premier League: Manchester United manager Ole Gunnar Solskjaer relishing renewal of rivalry against Leeds United
The Norwegian came on as a substitute in the last Premier League meeting between the two sides in February 2004, months before Leeds tumbled out of the English top-flight.
Premier League: Leeds defender Robin Koch to undergo knee surgery after aggravating injury in Chelsea defeat
The 24-year-old initially sustained the injury on his debut in the opening game of the season against Liverpool.
Premier League: Chelsea's Frank Lampard dismisses talk of 'spygate' in build up to clash with Marcelo Bielsa's Leeds
Lampard and Bielsa will face off for the first time since the 2019 play-offs in the second-tier Championship, when Derby, then managed by Lampard, beat Leeds at the end of a turbulent campaign.
Hostilities between the clubs' fans stretch back more than 50 years but it will be the first Premier League meeting between Leeds United, Chelsea for 16 years.
Arsenal have now not scored from open play in five Premier League games and even before being reduced to 10 men, offered precious little going forward.
Premier League: Brendan Rodgers jokes he 'got lucky' after Leicester City beat Marcelo Bielsa's Leeds United
Brendan Rodgers' Leicester have made their joint best-ever start to a top-flight campaign, even better than when they won the title in the 2015/16 campaign.
Premier League: Patrick Bamford hat-trick fires Leeds United to third place as Aston Villa suffer first defeat of season
Bamford had one goal in 27 Premier League appearances prior to this season, but he already has six in six games this term.
Everton will head into the Merseyside derby against Liverpool after the international break with maximum points from their first four games.
Matchday previews: Marcelo Bielsa's Leeds host Manchester City; Roma seek first win of season against Udinese
Roma seek their first win of the season when they visit Udinese, who lost their opening two matches.
Premier League: Patrick Bamford scores late as Leeds United edge past Yorkshire rivals Sheffield United
Leeds look well-equipped to enjoy a strong showing in their first season back in the top-flight for 16 years.
Back in top-flight after 16 years, Marcelo Bielsa's Leeds put Liverpool under enough pressure by coming from behind thrice to expose flaws in a team that won the title by 18 points.
After finally leading Leeds back into the top-flight, Bielsa's side face the daunting task of beginning their Premier League campaign away to champions Liverpool on Saturday.
Premier League: Leeds United manager Marcelo Bielsa confirms he is staying at club for one more season
Speaking remotely via a new translator, Bielsa refused to elaborate on why talks over an extended deal had taken so long.
Premier League: Remind Marcelo Bielsa to try to relax a little bit more, says Leeds United owner Andrea Radrizzani
Bielsa is the longest serving Leeds manager since Simon Grayson's December 2008-February 2012 stint that saw the team promoted from the third tier.