His court appearance was transmitted over a video, CNN reports.
Ullah (27) is being treated for lacerations and burns to his hand and abdomen at Bellevue Hospital after the bombing.
The police had accused him of detonating a homemade device on a busy transit hub in New York City, injuring five people and creating panic.
Ullah will be held without bail until his next preliminary court hearing on January 13, 2018.
He was charged with five federal terrorism-related charges and three state terrorism-related charges, according to CNN.
According to the criminal complaint, Ullah had posted on Facebook earlier in the day of the bombing attempt, saying, "Trump you failed to protect your nation."
Meanwhile, in a news conference on Wednesday, counter-terrorism chief Monirul Islam said that Ullah's attempted attack was inspired by the "writings and online sermons of a Bangladeshi cleric named Jasimuddin Rahmani, the spiritual leader of Ansarullah Bangla Team (ABT), a banned Bangladeshi terrorist organisation," The Washington Post reported.
Rahmani is said to be serving a five-year prison term after his conviction on charges of inciting the murder of a Bangladeshi blogger. (ANI)