People in part of Norfolk say they fear for their lives trying to cross a road for a bus.