Objective To improve a newly developed patient-reported outcome measure for thyroid patients using cognitive interviewing. Methods Thirty-one interviews using immediate retrospective and expansive probing were conducted among patients with non-toxic goiter (n = 4), nodular toxic goiter (n = 5) Graves' disease (n = 6), thyroid eye-disease (n = 6), and primary hypothyroidism (n = 10). The questionnaire was revised successively. Six iterative rounds of interviews were conducted. Identified problems were categorized according to Tourangeau's four-stage model. Results Problems were identified 126 times in 43 of the 99 tested items, four of the 15 introductions, and four of the five response categories. Fifty-four problems involved comprehension, one retrieval, 23 judgment, 28 response, and 20 were not applicable to the four-stage model. Among all problems identified, 18 concerned attribution, i.e. whether or not to report only issues considered of thyroid causality. Within each round of interviews, the number of problems declined from an initial average of six per interview to two, mainly due to a reduction in the number of problems associated with comprehension. The least amount of reduction was within the set of problems involving attribution. Conclusion The cognitive interview methodology was effective in identifying and reducing problems within the questionnaire responding process. Patients tended to selectively report problems they considered to be caused by their thyroid disease even when specifically instructed not to.