Google maps are programed into the system, so it can't be that difficult to program in where a stop sign or light is, or what the speed limit is. I can't really think of any signs the car would need to "read", as long as everything programmed within the system and the vehicle is linked to a larger network somehow. This type of vehicle would of course require good integration to function safely.
As for the jaywalking pedestrian, even a properly educated child can learn to look both ways before crossing the street. If a person steps in front of a moving vehicle while jaywalking that is their own fault. We have crosswalks for a reason.
The biker on the road and certain weather conditions could be tricky issues to solve.