The Stone Pony - Why go to bed when you can have another drink? At The Stone Pony, located at 913 Ocean Avenue, the party marches into the late, late hours, as they have one of the best all-night parties. Of course, this could mean some serious pain the next morning (or, more likely, the next afternoon.) Don’t say we didn’t warn you.