Can celery actually make your penis bigger?

No it doesn't. Nor does any product or medication. Beware of any such claims.
The grocery store. Would never be able to keep celery in stock if that was true. Celery does not enlarge the penis. If does however provide fiber. So, eat celery to keep yourself regular but don't expect your penis to grow any larger because of it.