I have a reinforcement learning model I'm trying to create for a grid based game. One of the features of the game is that the game board can get bigger mid game, although I know this is a problem that machine learning isn't very good at. However, one idea that I had was to train a convolutional neural net model that doesn't have a fully connected layer at the end, so that the input and output size of the net can vary together, and the model will still work because it won't need any additional weights to produce an output. In addition, adding a fully connected layer or two at the end would be a very large amount of extra weights to train, since the action space of the problem is very large (15232 discrete actions if I remove the expanding grid portion of the game). If my calculations are correct, a single fully connected layer would become 464 million more weights. Which seems like a lot compared to my existing 223,350 trainable parameters.
Here is one iteration of this network type I've tested: For this model, the observation space is 92 layers deep, and 28 wide by 16 tall. And the action space is 34 layers deep, and 28 wide by 16 tall. I'm not getting amazing results from this network (although better than random choice), which could be due to a multitude of reasons. However, I don't know if this is bad idea as I'm having trouble reasoning about this design generally.
Is a fully connected layer at the end necessary for the network to learn global ideas about the observation space?