Rockland County District Attorney Thomas Zugibe said the actor owes more than 350,000 US dollars (£215,000) in taxes and penalties.
He said Stephen was arraigned on a charge of repeated failure to file tax returns.
The 46-year-old actor pleaded not guilty and was released without bail. He is due back in court on February 5.
The district attorney said Stephen could face up to four years in prison if convicted.
Stephen's lawyer, Russell Yankwitt, was not available for comment.
The star is the youngest of the acting Baldwin brothers. He starred in 1995's The Usual Suspects and appeared in 1989's Born On The Fourth Of July.