When is it safe to roughhouse with my baby?

Depends on "rough": Depends on your definition of "roughhousing". Babies need physical activity to develop nerve-muscle control/connections. Engaging with them is safe levels of play that challenges them is good. But, care should always be taken that you aren't exposing them to undue stresses or strains for which their immature systems aren't yet ready. If they fuss, back off immediately. Avoid jarring li'l brains!

Never: If you intend to be rough with a baby please don't have any. When a kid is old enough to tell you you are hurting them you might push the play a bit, but babies die every year because parents don't understand how frail their systems are.