While the line isn't sharp, I would distinguish somewhat between older BUGS like languages, where the underlying analogy is typically a directed graph, and more

While the line isn't sharp, I would distinguish somewhat between older BUGS like languages, where the underlying analogy is typically a directed graph, and more recent probabilistic programming languages like Church, where the underlying analogy is an execution trace. The excitement around this newer form of probabilistic programming comes from the fact that they really are programming languages. Unlike BUGS, they inherit all the flexibility, composability, and encapsulation of the underlying language. Inference is viewed as sampling execution traces, so the connection with a programming language is quite tight.