Implemented "robust optimization" as described in the paper. The optimization now solves for the "robust energy function" which calculates a weighted least squares. The goal is to have those pixels which are already very close to their source not be less affected by others during optimization.
I tested it against the method that I used before and it was only marginally noticeable. Here are the results: