This question would require an experiment that cannot ethically be conducted, but it is interesting.
Wikipedia has an article on historical attempts at language deprivation experiments:
An experiment allegedly carried out by Holy Roman Emperor Frederick II
in the 13th century saw young infants raised without human interaction
in an attempt to determine if there was a natural language ...
"foster-mothers and nurses to suckle and bathe and wash the children,
but in no ways to prattle or speak with them ... But he laboured in
vain, for the children could not live without clappings of the hands,
and gestures, and gladness of countenance, and blandishments." ...
James IV of Scotland was said to have sent two children to be raised
by a mute woman isolated on the island of Inchkeith, to determine if
language was learned or innate. The children were reported to have
spoken good Hebrew ...
While these experiments had the advantage of a lax ethical environment, they also had a major disadvantage of poor methodology, and the accuracy of their documentation is very questionable.
There are quite a few documented cases of feral children, but none that I'm aware of who were raised together so as to have had an opportunity to develop language.
The most compelling case for the innateness of language development that I'm aware of is the Nicaraguan sign language (ISN):
... a sign language that was largely spontaneously developed by deaf
children in a number of schools in western Nicaragua in the 1970s and
1980s. It is of particular interest to the linguists who study it,
because it offers a unique opportunity to study what they believe to
be the birth of a new language.
These children were in fact raised as normal children, but were language isolated, with no contact with other deaf children until school age. Some variables not well controlled for here include that they were exposed to the concept of language, were aware that other people communicated with each other, and already had a proto-language (mimicas). Nonetheless, they appeared to develop a proper structured language fairly quickly upon being introduced to each other, without any apparent support or encouragement from adults, who were entirely unfamiliar with any sign language.
There are similar examples of isolated sign-language development, such as ABSL and MVSL, but the problem with all of them is that the actual level of language isolation is not well controlled for, and hence not clear.