I was trying to solve a partial differential equation (PDE) using a neural network. The solution to the PDE is not unique unless the boundary condition is determined.
In my case, the neural network takes $(x, y)\in [0, 10]\times[0, 10] = A$ and outputs a scalar. Suppose at the beginning of the training, I only sample from the middle of $A$, which will give us a neural network that satisfies the PDE except for the boundary condition. Then we sample from the boundary of $A$.
My question is, will it be costly to train using the algorithm above? My guess is yes since we have to drag that solution to somewhere else based on the boundary condition, causing large change to the output value. And intuitively, it's better to construct the solution on the boundary first to make the foundation of the solution correct.
I know that we can test it by experiments, but I'm more interested in a theoretical explanation, e.g., why / why not will it be costly to update parameters of the neural network after the boundary data get considered?