The subject you're describing often goes under the names computer architecture, computer systems, computer organization and design, and the like. One example is Elements of computing systems, based on a course From Nand to Tetris originating in the Hebrew University. The Amazon page on the book links to other popular textbooks on the topic.
Generally speaking, this area is at the intersection of computer science and electrical engineering. Some aspects are much closer in spirit to electrical engineering, for example the physics behind memory and storage architectures. Other aspects, such as cache coherence algorithms and error-correcting codes, are more likely to be considered an integral part of both disciplines.