Atletico's transfer ban ended last week and Costa was playing his first league game since rejoining the Spanish club from Chelsea.
Antoine Griezmann returned to form with a stunning overhead kick and assist for Kevin Gameiro to beat Roma 2-0 on Wednesday.
La Liga: Atletico Madrid boss Diego Simeone vows to defend under-fire Antoine Griezmann 'to the death'
Atletico Madrid coach Diego Simeone insisted he will defend off-colour star forward Antoine Griezmann "to the death", as long as he remains an Atletico player, after Saturday's 0-0 draw against local rivals Real Madrid.
Both Madrid sides suffered a familiar lack of cutting edge in front of goal this season
La Liga: Barcelona record best start to season with win over Athletic Bilbao; Valencia beat Alaves to stay 2nd
Barcelona recorded their best ever start to a La Liga season as coach Ernesto Valverde enjoyed a successful return to Athletic Bilbao with a 2-0 win on Saturday
Antoine Griezmann ensured Malaga didn't spoil Atletico Madrid's moving in party at their new 68,000 Wanda Metropolitano stadium with the only goal in a 1-0 win on Saturday.
Lionel Messi scored twice and saw a penalty saved as Barcelona maintained their perfect start to the season in La Liga with a 2-0 win at Alaves on Saturday.
Atletico's hopes seemed to have gone when Griezmann was controversially dismissed 23 minutes from time.
La Liga: Atletico Madrid close gap at the top with narrow win over Real Sociedad, Athletic Bilbao win
Atletico Madrid's fine form continued as Filipe Luis scored the winner in a 1-0 win over Real Sociedad to close to within seven points of La Liga leaders Real Madrid on Tuesday.
Luis Suarez struck four goals for the second time in four days as Barcelona moved back to the top of La Liga with the help of three penalties in a 6-0 win over Sporting Gijon on Saturday.