Neymar was sent off in disgrace after Brazil's first defeat to Colombia in more than two decades on Wednesday night.
The Barcelona striker was shown a red card for his role in a melee that marred the end of the Copa America Group C tie.
Neymar sparked the clash when he kicked a ball into a Colombia defender from close range, after the whistle had sounded at the end of the game. He then appeared to aim a headbutt at Colombia goalscorer Jeison Murillo.
Colombia striker Carlos Bacca was also shown red following the fracas at the end of the game, which leaves three teams - Brazil, Colombia and Venezuela - on three points in Group C.
"Referees have to stop thinking that they are the stars, the stars are not them -- they are there to control the game," Alves fumed. "We are used to this in South America - everyone here is against Brazil."
Alves also accused Colombia's players of seeking to provoke Neymar, 12 months after the striker suffered a broken vertebra during the two teams' stormy World Cup last eight battle.
"They know the personality of Neymar and they went in search of him," he said. "They tried to provoke him and make him nervous."