Patients' confidence in vaccines has waned since the COVID pandemic, according to a national poll and doctors.