The Milgram experiment is makes people think the voltage is dangerous, due to the labelling and acting by the confederate 'learner', who eventually goes silent signifying unconsciousness in the least, and possibly deadliness. Despite these cues over 50% of participants ('teachers') progress to the highest 'shock' level with simple promoting from an authority figure. We are, of course, see drop-offs as the perceived danger to the 'learner' (confederate) increases, and the benefit of obedience are outweighed by the signalled danger. Of course you would have to measure beliefs about how harmful each level of shock is to see how it ties with drop-off exactly. Unfortunately, it seems beliefs about harm are assumed, and have not been actively researched. As you can see from the drop-offs its clear in some participants believe the shocks increase harm, whether those who continued to press think so.
In general warning labels are enough to create a negative emotional response for people i.e. cigarette packaging, food and drink.
What the obedience experiment indicates to some researchers is that majority of people who will continue to cause, what researchers interpret as, great pain with simple prompting from authority, due to the deferred responsibility and accountability. Of course this is assumed, and not actively measured, it could well be that the participants did not believe it to be so harmful, or could even see through the ruse. Interestingly, some studies in the past showed that putting other 'teachers' (confederates) in the room with the participant, could impact the participants 'shock' progression, if the confederates dissented against the authority by leaving part way through or refusing to continue. However, recent studies show that this dissident social effect has not been replicated. Which leaves question marks over why.
While individuals often believe that in a similar situation to the they would perform better than average in their country, knowledge of the experiment does play a role in how far participants believe they would go.
In summary, we assume the participants believe increasing shocks cause more harm. We do not know if those who went all the way believe the shocks were as harmful as participants who dropped out.