I reached below the kitchen cabinet, i accidentally bumped the back part of my head on it when standing up. it didn't bleed and it was sorta not that hard of a bump. should i pay a visit to the er? it doesn't really ache much but i have an headache?

Minor trauma as describes can trigger headaches of variable intensity for up to a week after the event. When the injury is not accompanied by loss of consciousness, pre or post event memory loss, changes in balance or similar symptoms it is unlikely to yield evidence of significant injury on exam or testing. In the final analysis, you must go if the worry exceeds your comfort at not going.

Agree with Dr Ferguson, as it appears you experienced a minor closed head injury without intracranial consequences. Such events are common and usually accompanied by local pain and less commonly a mild headache. Medical attention might be necessary if nausea, vomiting, dizziness, imbalance, blurred or double vision, episodes of unconsciousness, confusion, etc.