Blatter responded to last week's protest by AC Milan midfielder Kevin Prince-Boateng by saying players should not walk off the pitch, but that administrators should use harsh sanctions to deal with racism.
Piara Powar, executive director of the European anti-discrimination organisation FARE, said FIFA itself had failed to implement tough action at any time in the past.
Powar told the Press Association: "How many times has FIFA sanctioned any team with a points deduction for racism in its competitions such as World Cup qualifiers?
"When have you ever seen FIFA take tough action over racism in an international friendly?
"The answer is never. That's the frustration - these are people who have the ability to make those things happen but are not delivering this action.
"People will judge you on your actions not your words."
Blatter had appeared to suggest that players could use racism as an excuse to walk off during a game if they were losing.
Speaking at a conference in the Middle East, Blatter told United Arab Emirates newspaper The National: "Walk off? No. I don't think that is the solution.
"I don't think you can run away, because eventually you can run away if you lose a match.
"This issue is a very touchy subject, but I repeat there is zero tolerance of racism in the stadium; we have to go against that.
"The only solution is to be very harsh with the sanctions - and the sanctions must be a deduction of points or something similar."
Powar criticised Blatter for that stance and said the FIFA president appeared not to know what was in the rules.
He added: "The issue is that players should be allowed to walk off as a protest as a last resort. In the case of Boating and Samuel Eto'o with Barcelona they walked off after they had already brought it to the attention of the referee and nothing had happened.
"We disagree entirely with the idea that Kevin-Prince Boateng ran away from the Milan match in which he was subjected to racial abuse. It's a nonsensical suggestion.
"The fact that the president of FIFA himself doesn't seem to be aware of the rules which have taken a lot of time to get on to the statute books is really disappointing."
AC Milan president Silvio Berlusconi also rejected the FIFA president's position.
He told radio station RTL: "I am of the opposite view. In fact, I thanked and congratulated my players for their decision to leave the field during the friendly in Busto Arsizio.
"This is an uncivilised problem that needs to be stopped, people should not allow these things to happen.
"Teams out on the pitch should set an example of civility and the educational role of football should not be underestimated.
"It's not only about the behaviour of players in the game but of the public, and everyone needs to avoid giving Italy a negative image."
Italian state police confirmed that six suspects have been arrested over the incident, having been identified by footage captured on video cameras at the Stadio Carlo Speroni.