I was reading through an article about research design. The authors gave an example on the relation between internal and external validity which I thought might be an important area of consideration when designing research proposals.
"A restricted study population (and exclusion criteria) may limit bias and increase the internal validity of the study; however, this approach will limit external validity of the study and, thus, the generalizability of the findings to the practical clinical setting. Conversely, a broadly defined study population and inclusion criteria may be representative of practical clinical practice but may increase bias and reduce the internal validity of the study."
Broadly speaking, what questions can be asked to help decide one should strike a balance between the internal and external validity in our research proposal designs? Is it necessary then to carry out a pilot study first to establish some level of internal validity before moving on to establish external validity in a much bigger and formal follow-up study?
Update: This post has been shifted to academia.stackexchange. All changes and edits will be made via the posting there henceforth.