I bumped my elbow hard today on something. No bruising or swelling but really hurts! Should I ice it? Or what?

Good idea. Wrap it with a towel to protect the skin and use a ziplock with ice in it or a bag of frozen peas if there is no other ice pack available. Ice for 15 minutes every 1/2 to 1 hour while awake. Probably bumped your ulnar nerve running by the olecranon, commonly called the "funny bone".
Ice. I would definitely recommend icing your elbow for 20 minutes 3-4 times per day for a few days. Asking as it is relatively functional. If it continues hurting, gets worse or loses function, then have it seen by your family doctor or an urgent care doctor to be further evaluated and treated. Ibuprofen 200-600 mg three to four times a day should help as well.
Yes. Rest, ice and elevation. See how it feels tomorrow. Elbows can be particularly painful because of the "funny bone" actually a nerve that runs close to the surface (ulnar nerve). Also be sure to take some ibuprofen or naproxen as well.