Without going into the business of ion channels, sensory receptors connect to specific brain areas (although there are relays in-between). For vision it is V1 in the occipital cortex, for audition A1 in the temporal cortex, for somato-perception S1 in the parietal cortex. Olfaction, gustation and the vestibular systems don't seem to have a clear "primary" area like audition, vision and somatoperception. The orbito-frontal cortex seems to be quite involved in olfaction/gustation, but it's also involved in a lot of different stuff. For the vestibular system some areas of the parietal cortex are also quite involved, but also in other stuff.
Primary areas receive direct connections from the sensory organs (again, with relays in-between) and are organized topically. For example, the visual cortex has a retinotopic organization where adjacent cells in the cortex encode information from adjacent cells on the retina. A1 has a (somewhat looser) tonotopic organization, and S1 has a "body-topic" organization (called the homunculus). Those pathways a quite fixed and quite distinct. There is no way to have a "mix-up" between them, and you never see that. There can be some "mix-up" within one pathway. For example amblyopia in vision where the visual cortex does not develop as expected and results in some visual loss (less than 20/20 vision in one eye).
So yes, the sensory organs develop well (with some exceptions) and project to very distinct areas all over the brain (frontal, parietal, occipital and temporal lobes). What would happen if you rewired the brain is unclear. There was one famous experiment where optical nerves in ferrets were connected to the auditory cortex. Experimenters observed a cortical map similar to what they would have expected in visual areas. So it suggests that at least the visual and auditory cortices are "interchangeable". But then these areas have their own connections to other areas, so what actually happens is anyone guess (do you "hear" visual stimuli?). Those are deep philosophical questions, similar to "do we all see colors similarly?", that we are far to have a good handle on scientifically.