Neural network for PDE: Should we train the PDE using more initial and boundary data at the beginning?

Question

I was trying to solve a partial differential equation (PDE) using a neural network. The solution to the PDE is not unique unless the boundary condition is determined.

In my case, the neural network takes $(x, y)\in [0, 10]\times[0, 10] = A$ and outputs a scalar. Suppose at the beginning of the training, I only sample from the middle of $A$, which will give us a neural network that satisfies the PDE except for the boundary condition. Then we sample from the boundary of $A$.

My question is, will it be costly to train using the algorithm above? My guess is yes since we have to drag that solution to somewhere else based on the boundary condition, causing large change to the output value. And intuitively, it's better to construct the solution on the boundary first to make the foundation of the solution correct.

I know that we can test it by experiments, but I'm more interested in a theoretical explanation, e.g., why / why not will it be costly to update parameters of the neural network after the boundary data get considered?

Answer 1

There is sadly very little theory that is useful for predicting the behavior of neural networks. Also, their effectiveness tends to be dependent on the particular workload/task you are trying to solve it. While I know it's not what you were hoping to hear, I suggest that you try it in empirical experiments.