I'm not sure I know of a simple, formal, abstract definition of "atomic" for a programming language. I know of many different ones at various levels of abstraction and with regards to various topics. I also know some that seem fairly abstract but have fairly sophisticated prerequisites (e.g. presheaf models of concurrency). Part of the problem is that there are a lot of different ways of saying two programs "behave" "equivalently" and some definitions may respect different notions of "equivalent".
A sketch (that I'm making up right now) of one approach with respect to an interleaving model of concurrency is: Given an operational semantics, a sequence of statements is executed atomically if all transitions (of the operational semantics) involving those statements happen in sequence. A sequence of statements is atomic if all possible sequences of transitions executes the sequence of statements atomically. We could loosen this to an "as if they were executed atomically" which we might start to formalize by saying that all possible sequences of transitions are equivalent (presumably observationally equivalent) to a sequence of transitions that executes the statements atomically.
Basically, executed atomically means that once we start executing some sequence of statements, we don't interleave the execution any other statement into the execution of the sequence. Implementation-wise, this might mean that we only execute one thread at a time as in a time-slicing implementation of threads, and a sequence of statements executed atomically means there were no context switches during their execution. The above definition has an issue in that it doesn't really handle coordination between threads well. It wouldn't really make sense to say that communication along a channel between two threads, for example, was ever atomic. (Often we talk about operations on a data structure as being atomic, and that would probably be more appropriate for this example. The definition would be specific to the data structure.)
For your example program and assuming an operational semantics where a print
statement could be handled in a single transition, then Program 2 would always be executed atomically and is (fairly trivially) atomic. Your second execution describes a scenario where Program 1 is executed atomically while your first does not. If there was some mechanism that guaranteed your second execution always happened, then you could say Program 1 (with that mechanism) was atomic, but otherwise it presumably would not be atomic on its own.
As mentioned in my comment, "atomic" is often used in an informal and/or implementation-oriented sense. Therefore what exactly is intended is often unclear, though the spirit of what is intended is usually reasonable clear.