I am four months pregnant and had a vegetable sushi roll. Should I be worried?

Not if just veggies. If it was just veggies then it shouldn't be a problem. You certainly want to avoid raw fish along with minimizing deli meats and unpasteurized milk and cheeses. The problem with raw fish is mercury, but more significantly parasites (like anisakis). The deli meats, milk and cheese can harbor harmful bacteria. These are no problem for healthy adults but can compromise an unborn baby.