This is an interesting question that has a multidisciplinary application. Not only does this apply to computer science, but given the nature of preservation of that data over time also applies to archival science!
While DVDs and CD-ROMs generally have a high information density, they were never designed with longevity in mind. The average life-expectancy for either of the two is about 100-200 years, according to Council on Library and Information Resources. 
This may sound like a long time, but your question regarding the ability to retrieve the information 120 years from now comes into play. A major factor in deciding if a media is archival or not is the ability for that information to be interpreted years later. Assuming there is still a method for retrieving data from CDs and DVDs 120 year from now depends on several factors:
- How much information are you looking to store?
- What are the storage conditions?
--You said "time capsule," but does this mean the capsule will be hermetically sealed? Will it be purged of all moisture, oxygen, and other environmental contaminants that are known to degrade material integrity over time? Will the environment be replaced with an inert gas like nitrogen to provide a stable environment?
- What is the tolerance for data loss/corruption? Is this going to be something that needs to have archival quality, e.g. zero-tolerance for loss? CDs and DVDs can experience some loss given their storage and handling over time. 
- You have already asked one of the most important questions regarding the ability to retrieve the information 120 years from now.
The last question is something that archivists have been asking for some time. In today's world where we are generating more data on a daily basis than we have since the beginning of time, we need to develop a solution that addresses the questions of how much information needs to be stored and for how long does it need to last?
A question still remains unanswered: does your time capsule have to have media that already exists, or is it a quandary into the realm of storage technologies that could be developed to store information for a long period of time?
If it has to be of media that already exists, there are a few options out there:
- Microfilm: industry standard for data preservation. Film of particular types is guaranteed to last for ~500 years.  In the application of computer science, you could quite feasibly encode some arbitrary data in binary dots/dashes onto a piece of microfilm and interpret the data via computer. The advantage here is twofold: binary is a numbering system that isn't going anywhere and all you just need something to view the microfilm. Any intermediate technology can change between now and the next 120 years (think scanners, other ways of reading data into a computer), but all you need is a way to read visual data into a computer. This is unlike CDs or DVDs that can only be read by a very specific device.
- You already mentioned one of them: an LP. Specifically an LP made from copper, aluminum and gold like the ones on Voyager I and II.
- Least romantic? Straight up paper. As long as it is well-kept and information density is not a concern, acid-free paper will stick around for 120+ years (think The Declaration of Independence).
Things to stay away from:
- SSDs or anything based on flash memory. These all need a minimum trickle charge of voltage to hold onto all of their data. Over time, the electrons leak from the storage cells and the data becomes corrupted. 
Some things have already been proposed for future storage:
- Recording the information on a CD or DVD-like surface, or a small square, either of which would be made from glass. This would not involve the metallic substrate employed by laser-read media that is known to degrade over time. The method for recording the information would be etched in the media by laser in a similar bit pattern, while using etchings of varying depths could be used to increase the information density.
- Using DNA to encode data. DNA is known to last for millennia as long as it is kept from UV light. It would also provide a method of storing information in base-4 -- given the fact that DNA uses the four base pairs G, C, T and A to store information -- versus base-2, which uses 0 and 1. This would effectively increase the information density in two domains: the space in which the data could be stored given the relatively small size of DNA and the numbering system used to store that data.