La Liga's reigning champions lost 2-1 to Alcoyano, who scored a remarkable winner in extra-time after having a man sent off.
Atletico's defeat makes it consecutive seasons they have lost to a third-tier side in the cup after being knocked out by Cultural Leonesa 12 months ago.
Ibiza made no mistake in dumping out top-flight opposition Celta Vigo, racing into a 3-0 half-time lead over Celta, helped by two goals from Sergio Castel.
European football matchday: Spurs host second-tier Brentford in League Cup; Sevilla in Copa del Rey second round
Brentford are in fourth place in the 24-team Championship and have beaten four Premier League teams to get to their first ever cup semifinal.
Holders Madrid and Athletic will meet on 14 January in Malaga a day after Barca's fixture with Sociedad in Cordoba, with the final in Sevilla on 17 January.
Bilbao proved to be tough opponent at Alfredo Di Stéfano stadium despite playing most of the match with only 10 men. But when the Basque Country team was threatening to at least salvage a point, Benzema scored his team-high ninth and 10th goals of the season.
Copa del Rey: Around 250 fans to be allowed to watch Atletico Madrid's game against fourth-tier side Cardassar
Fourth-division club Cardassar announced on Wednesday that their 16 December game against visiting Atlético will have a limited number of fans at the Es Moleter stadium in the Balearic Islands.
Copa del Rey: Spanish Football Federation heeds to Athletic Bilbao, Real Sociedad's request of playing final in front of fans
The RFEF said it agreed “wholeheartedly” with the two football clubs wishes for fans to attend the match.
The first final between Athletic Bilbao and Real Sociedad, two prestigious Basque clubs, was due to be held on 18 April in Seville until all organised football was indefinitely postponed in March as COVID-19 swept Spain.
LaLiga: After Barcelona's chaotic start to 2020 with on and off field setbacks, Lionel Messi calls for calm to think about football
Messi made the call for his club to put its house in order ahead of an important stretch of matches for the defending league champions
Copa del Rey: Martin Odegaard's winner helps Real Sociedad edge past second-division side Mirandes in semi-final tie
Real Sociedad beat second division Mirandes 2-1 at home in the Copa del Rey semi-final first leg on Thursday to stay on course to reach the final for the first time in 32 years.
Copa del Rey: Iker Muniain's goal gives Athletic Bilbao slight edge over Granada with narrow victory in semi-final first leg
Athletic spurned early chances before the electric Williams broke down the left flank and drove a low ball across for Muniain to convert from close range.
Atletico Madrid secured a much-needed 1-0 victory against Granada which was inspired by returning captain Koke.
Copa del Rey: Revamped tournament earns praise from fans, players and small clubs after upsets of heavyweights
The transformation of the Copa del Rey, however, has earned rave reviews from fans and players, above all from the mid-level and small clubs which now have a much better chance to pull off an upset.
The Copa del Rey's new single-elimination format claimed its biggest victims, with Real Madrid and Barcelona both being eliminated in the quarter-finals.
Copa del Rey: Mirandes pull off another upset with 4-2 win over Villarreal, reach semi-finals for second time
Small second-division club Mirandes pulled off another upset in the Copa del Rey on Wednesday, eliminating Villarreal 4-2 to reach the competition's semi-finals for the second time.
Former Valencia forward Roberto Soldado converted the penalty four minutes into added time after VAR awarded the spot-kick for a handball by a Valencia player after a corner
Copa del Rey: Lionel Messi inspires Barcelona to thumping victory over Leganes to seal quarter-final spot
A Lionel Messi-inspired Barcelona eased past Leganes 5-0 on Thursday to reach the Copa del Rey quarter-finals.
Copa del Rey: Raphael Varane, Vinicius Junior help Real Madrid outclass Zaragoza to seal quarter-final berth
Real Madrid thrashed a depleted Real Zaragoza side 4-0 on Wednesday to reach the Copa del Rey quarter-finals.
Copa del Rey: Fans 'don’t deserve this,' says Atletico Madrid midfielder Saul Niguez after elimination at hands of Cultural Leonesa
Atletico, who are third in La Liga and into the last 16 of the Champions League, beat Barcelona 3-2 in the Spanish Super Cup earlier this month and held Real Madrid to a draw in the competition’s final before losing on penalties