Call back. In the name good customer service. Plus it will only take a min or two of your time. (Devils advocate) What if you were schedule to go out and take a look at the job, about an hours drive. You get on lacation, and the client informs you that he had hired someone. This is after your initial call to confirm your intended arrival time. A phone call, though not owed would have been great. In business and personal always "do unto others..."
I always call and let the client know that at this time we are not interested in the work for whatever reason. People are usually cool about it. Sometimes I will tell people we are too booked to fit it in the schedule at this time as well. But I think the answer is yes. Call them and let them know so they are not hanging around waiting on you.