My initial thought when reading this question is that it would an extremely difficult thing to pin down whether or not depression would affect memory in the long term this is due to a variety of confounding factors but the one that strikes me as most prominent is that major depression rarely goes untreated for a long period of time, in a population of people suffering with depression most are likely to have sought treatment as some point. Meaning that a study into the effects of depression in the long term would be very difficult to facilitate as it would be hard to infer whether the changes in memory function would be due to depression or the effects of treatment for depression such as SSRIs, MAOIs, electroconvulsive therapy, and others. Although, SSRIs have actually been found to improve memory (Harmer et al, 2002), and effects of ECT have been found to be temporary with people recovering by 6 months (Calev et al, 1991).
A link between depression and memory impairment is supported
Burt, Zembar, & Niederehe conducted a meta analysis of 99 studies looking at depression and memory and found that:
A significant, stable association between depression and memory
impairment was revealed. Further analyses indicated, however, that it
is likely that depression is linked to particular aspects of memory,
the linkage is found in particular subsets of depressed individuals,
and memory impairment is not unique to depression.
There is some evidence that people suffering from major depressive disorder have slightly different brain structure to neurotypical individuals. Brenner et al found that Patients with depression had a statistically significant 19% smaller left hippocampal volume than comparison subjects, without smaller volumes of comparison regions (amygdala, caudate, frontal lobe, and temporal lobe) or whole brain volume. It was not clear if this could change when the persons symptoms subdued, however to me it seemed like a permanent difference (obviously difficult to infer causality tho).
In terms of memory being enhanced in a manic state it appears that memory is much the same in a manic state as it is in a normal state (Glahn et al; Weingartner et al)
References
Bremner, J. D., Narayan, M., Anderson, E. R., Staib, L. H., Miller, H. L., & Charney, D. S. (2000). Hippocampal volume reduction in major depression. American Journal of Psychiatry, 157(1), 115-118.
Burt, D. B., Zembar, M. J., & Niederehe, G. (1995). Depression and memory impairment: a meta-analysis of the association, its pattern, and specificity. Psychological bulletin, 117(2), 285.
Calev, A., Nigal, D., Shapira, B., Tubi, N., Chazan, S., Ben-Yehuda, Y., ... & Lerer, B. (1991). Early and long-term effects of electroconvulsive therapy and depression on memory and other cognitive functions. The Journal of nervous and mental disease, 179(9), 526-533.
Glahn, D. C., Bearden, C. E., Cakir, S., Barrett, J. A., Najt, P., Serap Monkul, E., ... & Soares, J. C. (2006). Differential working memory impairment in bipolar disorder and schizophrenia: effects of lifetime history of psychosis. Bipolar disorders, 8(2), 117-123.
Harmer, C. J., Bhagwagar, Z., Cowen, P. J., & Goodwin, G. M. (2002). Acute administration of citalopram facilitates memory consolidation in healthy volunteers. Psychopharmacology, 163(1), 106-110.
Weingartner, H., Miller, H., & Murphy, D. L. (1977). Mood-state-dependent retrieval of verbal associations. Journal of Abnormal Psychology, 86(3), 276.
