For instance, does watching an artist draw improve your drawing skill, even if you're not drawing along with them?
Watching others perform a skill serves as both instruction and visualization practice. The instruction can provide the beginner with a wealth of information that they lack, when starting a skill. It provides the experienced practitioner with details of technique that informs their growth.
How productive watching can be sometimes depends upon previous experience. For example, in the medical profession there is a phrase, "Watch one. Do one. Teach one." Which indicates that the practitioner should observe a technique once, do the technique once, and then teach it to someone else, once. This seems frightening, at first glance, since I certainly do not want to be someone's "do one" when I am lying on a surgery table.
However, the actual context of this learning method matters. Generally, what the practitioner is learning is some small variant of a skill they have already mastered. For example, if they already know a wide array of stitches for closing wounds, they may be learning a variant of one of these stitches. In this case, seeing the technique performed slowly once is sufficient. Then they practice the stitch by "doing one", often while being observed. Finally, they teach the technique to someone. This serves to reinforce their new knowledge, because to teach someone else something requires that you think about the subject carefully, which in turn aids in mastery.
Such a system of learning by watching breaks down, if the context of previous knowledge is omitted.