I would argue that gamification is essentially a form of applied motivational psychology. As such, all findings from research on motivation could potentially be important for gamification. In the other thread that you mention, Self-Determination Theory (Gagne & Deci, 2005) has already been brought up. I would like to make a couple of very general suggestions regarding your problem that stem from SDT.
Self-Determination Theory
In terms of SDT, motivation can be either externally controlled or autonomous. Someone seems to have suggested punishment, which would be a form of externally controlled (a.k.a. extrinsic) motivation. Another obvious way of extrinsically motivating people would be to reward them. Maybe this is an option for you, but since you already mention that reputation points, badges and the like don't work, this might not be a good route.
You are probably looking for a way to motivate people in such a way, that they participate in your games because they are autonomously motivated. In SDT there are 3 different forms of autonomous motivation: intrinsic, identified and integrated motivation. You don't have to worry about those that are intrinsically motivated, because they already like to participate in games and do so. Also, you can't control intrinsic motivation externally, it must come from the individual.
Internalization
However, integrated and identified motivation which are both forms of autonomous motivation (the latter is a weaker form) can be influenced. SDT states that it is possible that a process of internalization occurs, in which a certain behaviour becomes part of the identity of a person, which in turn fosters (autonomous) motivation for this behaviour. In other words, if you manage to somehow influence people's values and attitudes towards your game, they will be more likely to participate in it. As the other thread already mentioned, internalization occurs when the 3 basic psychological needs (need for competency, relatedness and autonomy) are met. SDT puts a special focus on the need for autonomy. Only when this need is met is the behaviour expected to become integrated. So as a very general answer to your question, you might want to look into ways to satisfy those needs. It also seems to work better to satisfy at least 2 of those needs instead of only one (Gagne & Deci, 2005).
Deci et al. (1994) performed a laboratoy experiment to explore ways, in which internalization can be promoted. They found three factors: giving a meaningful rationale, acknowledging that people might not find the task interesting and emphasizing choice rather than control. I must admit though, that I have not read this paper but only the very brief summary that Gagne & Deci (2005) give. It might be interesting for you to read it. Also, there is probably more research on SDT that has be done, so a literature search might yield interesting results.
References:
Deci, E. L., Eghrari, H., Patrick, B. C., & Leone, D. R. (1994). Facilitating internalization: the self-determination theory perspective. Journal of Personality, 62, 119–142.
Gagné, M., & Deci, E. L. (2005). Self‐determination theory and work motivation. Journal of Organizational behavior, 26(4), 331-362. PDF
