Although Cognitive Inertia (as mentioned by hexadecimal) is a nice, broad way to point to the phenomenon, people have also studied it from other points of view. Two of them are from economic and cognitive angles -.
From an economic level, what you are referring to is called the Sunk Cost FallacySunk Cost Fallacy. Here is an examplean example from Wikipedia that explains it from a business perspective -:
In business, an example of sunk costs may be investment into a factory or research that now has a lower value or no value whatsoever. For example, \$20 million has been spent on building a power plant; the value at present is zero because it is incomplete (and no sale or recovery is feasible). The plant can be completed for an additional \$10 million, or abandoned and a different but equally valuable facility built for \$5 million. It should be obvious that abandonment and construction of the alternative facility is the more rational decision, even though it represents a total loss of the original expenditure—the original sum invested is a sunk cost. If decision-makers are irrational or have the wrong incentives, the completion of the project may be chosen. For example, politicians or managers may have more incentive to avoid the appearance of a total loss. (source)
On a more personal level, this can be generally explained as the Zeigarnik effect, which states that- "people remember uncompleted or interrupted tasks better than completed tasks".
people remember uncompleted or interrupted tasks better than completed tasks
Although there are some issues with the reliability of the effect, it is generally extrapolated to say that the phenomenon -"compels humans to finish a task that they've already started, and that when we don't finish a task we experience discomfort and intrusive thoughts about it." (from an AsapSCIENCE video on "The science of Productivity")
the phenomenon compels humans to finish a task that they've already started, and that when we don't finish a task we experience discomfort and intrusive thoughts about it. (Youtube video source)