At a behavioral level, what you are referring to is called the Sunk Cost Fallacy. Here is an example that explains it from a business perspective -
In business, an example of sunk costs may be investment into a factory or research that now has a lower value or no value whatsoever. For example, \$20 million has been spent on building a power plant; the value at present is zero because it is incomplete (and no sale or recovery is feasible). The plant can be completed for an additional \$10 million, or abandoned and a different but equally valuable facility built for \$5 million. It should be obvious that abandonment and construction of the alternative facility is the more rational decision, even though it represents a total loss of the original expenditure—the original sum invested is a sunk cost. If decision-makers are irrational or have the wrong incentives, the completion of the project may be chosen. For example, politicians or managers may have more incentive to avoid the appearance of a total loss. (source)
On a more personal level, this can be generally explained as the Zeigarnik effect, which states that-
people remember uncompleted or interrupted tasks better than completed tasks
Although there are some issues with the reliability of the effect, it is generally extrapolated to say that -
the phenomenon compels humans to finish a task that they've already started, and that when we don't finish a task we experience discomfort and intrusive thoughts about it. (Youtube video source)