Human eye is very sensitive to luminance change and order of magnitude less sensitive to chrominance change.
MPEG under the hood is based on JPEG transform, so you have 8x8 blocks of DCT.
It blurs a bit the whole block approximating it. Colour space changes to YUV or YCbCr, to encode two channels of colour and one of luminance.
Luminance (grayscale if it was colour) is not compressed as much as new created two colours. These colours are fitted more loosely than luminance.
The most heavy tricks are in colour space and luminance is preserved with lower compression (and less stages).
If you have BW frame it cannot be dealt with high compression as this faster degrades quality. MPEG was not created to deal with BW data. Origins of this codec are from times when grayscale tv was appended colour on top of existing frame.
Here I mean origins, not the same scheme as bandwidth was smaller at this time, I am referring to colour space and compression over chrominance.
MPEG popular scheme is 4:2:2 which encodes four parts of luminance (BW) and two times two colour parts. Colour is encoded as differences, because there are two channels and three colours.
So simply 50% of data is well preserved luminance and 50% are three colours with higher compression ratio. Compressed better means greater ratio of original vs processed channel.
Luminance takes (taking it simple as luminance is part of these colours) roughly equal space as colour info.
Different codecs are using different strategies, I referred to MPEG, not some new mixed schemes.