Is the depletion effect of self-control reliable?

Question

Although a lot of research (Muraven, 1998) has indicated the ego depletion of self-control, a recent paper (Xu, 2014) claims that they fail to replicate the depletion effect: impaired performance in subsequent tasks. So, is the depletion effect of self-control reliable?

References

• Muraven, M., Tice, D. M., & Baumeister, R. F. (1998). Self-control as a limited resource: Regulatory depletion patterns. Journal of personality and social psychology, 74(3), 774. http://dx.doi.org/10.1037/0022-3514.74.3.774
• Xu, X., Demos, K. E., Leahey, T. M., Hart, C. N., Trautvetter, J., Coward, P., ... & Wing, R. R. (2014). Failure to Replicate Depletion of Self-Control. PloS one, 9(10), e109950. http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0109950

Answer 1

Short answer: We don't really know yet, but we should know soon.

Long answer:

There has been much research about the ego-depletion effect. The effect has been found in many published studies. As cited in APS Observer (2014):

A recent meta-analysis revealed a medium effect size (d = 0.62) across 198 tests of the ego-depletion effect (Hagger, Wood, Stiff, & Chatzisarantis, 2010).

However, there have been some doubts whether these results reflect publication bias (only studies that found the effect may have been published, those that have found no effect or the reverse effect may have landed in the "file drawer") or weak methods (small samples).

APS Observer (2014):

Despite these findings, some researchers have questioned the strength of the ego-depletion effect. A recent analysis conducted by Carter and McCullough (2013, 2014) suggested that the effect may be quite small or entirely an artifact of publication bias. The researchers cite evidence that many tests of the effect were substantially underpowered suggesting that the likelihood of finding such a large number of significant effects in the literature is improbable. Although the interpretation of these analyses have been questioned (Hagger & Chatzisarantis, 2014), it is possible that publication bias may have inflated the size of the effect evident from the literature.

That is why the APS journal Perspectives on Psychological Science has accepted the proposal for a large scale replication project led by Alex O. Holcombe and Martin S. Hagger.

APS Observer (2014):

By using preregistered replications across multiple laboratories, this [project] will allow for a precise, objective estimate of the size of the ego depletion effect.

More details about the project can be accessed at its website on the Open Science Framework.

References

Carter, E. C., & McCullough, M. E. (2013). Is ego depletion too incredible? Evidence for the overestimation of the depletion effect. Behavioral and Brain Sciences, 36, 683-684. doi: 10.1017/S0140525X13000952

Carter, E. C., & McCullough, M. E. (2014). Publication bias and the limited strength model of self-control: Has the evidence for ego depletion been overestimated? Frontiers in Psychology, 5. doi: 10.3389/fpsyg.2014.00823

Hagger, M. S., & Chatzisarantis, N. L. D. (2014). It is premature to regard the ego-depletion effect as ‘too incredible’. Frontiers in Psychology, 5, 298. doi: 10.3389/fpsyg.2014.00298

Hagger, M. S., Wood, C., Stiff, C., & Chatzisarantis, N. L. D. (2010). Ego depletion and the strength model of self-control: A meta-analysis, Psychological Bulletin, 136, 495-525.