I think thats right, using the slang baby as a direct object is rare but possible. “I shot my baby” e.g., but using it with “another baby” really throws it off

I think thats right, using the slang baby as a direct object is rare but possible. “I shot my baby” e.g., but using it with “another baby” really throws it off because a lover is not something you usually have multiple of at the same time so there is not enough context to resolve the ambiguity and lots of natives default to the wrong meaning.