This is a hot topic of debate, so my answer will be an incomplete one.
There are actually two separate questions here. One is on language and the other one is on environment.
Language:
My answer is no; different languages do not limit the conceptual repository of human mind. The current ongoing debate is partially on the Pirahã language. Everett studied this language and after seeing that it does not have syntactic recursion, he claimed that it demonstrates that Pirahã community somehow could not think recursively. Of course this is a very simplified view and may be a little misinterpreted and/or a provocative understanding of the Sapir-Whorf hypothesis. But I believe that the main problem is on the theoretical grounds of grammar. Steedman's CCG shows a very powerful alternative way of looking to grammars. It is computational and combinatorial. I don't know any particular studies on Pirahã with combinatorial grammars, but that would be an interesting issue to investigate. My personal opinion is that this thinking of "language effects the way of thinking" is also a bit dangerous and open to manipulations too much.
Environment:
Environment definitely affects the cognitive abilities. There are several studies on this. You can search the internet for papers such as Gathercole (1999) or Gleitman, Elissa, Newport (1995). These are the ones that came to my mind immediately. One of the facts is that late exposure to language entirely limits the grammatical abilities. Environmental feedback is also essential. It may be hard to observe physically impaired children but there are also studies that focus on deaf children in hearing families in the absence of sign language. These are interesting studies, but the bottom line is: yes, environment affects cognitive abilities.
