There's a big place at 318 Lafayette (right near Lafayette + Houston),
the one in Chelsea Market (14th St. and 11th?) is also pretty good (plus, there's all the food goodness of Chelsea Market, but beware because it's very crowded during the week, especially lunchtime, when it's wall-to-wall people).
Broadway Panhandler (8th St. and Broadway) has some stuff, occasionally good sales, but I usually find them overpriced for what they have...
Sur la Table is all over the city (near Prince St. and Broadway), and they're fairly good, but it's not restaurant-supply priced.
Bowery Restaurant Supply is pretty good, from what I recall--183 Kenmare (Bowery and Kenmare).
Sandra knows of a great cake shop, but I forget the location! (ETA: I looked it up; it's New York Cakes and Baking Supply, 56 W. 22nd St. at 6th Ave. It's a great store, but don't expect kid-glove treatment! We saw them chew out people who came in and photographed a few things. A strictly-business kind of place, but a real treasure trove for all things baking and cakes.)
I'd agree with Cyn's recommendation for Kalustyan's for spice shopping--be sure to time it around lunch (or early lunch; there's a crowd) to take advantage of their second-floor cafe--good mujaddara and pickled vegetables! smileys/smile.gif
