Heat. Often kids have some relief if they put their hurt ear down on a heating pad or compress at a low heat level. The diffuse heat messages going to the brain seem to diminish the ear pain some. If over 6mo I prefer Ibuprofen for the discomfort. Sx lasting more than 3 days deserve a doc's review.
Irrigation. The oil may have solidified and is attached to either the ear canal skin or even the ear drum. Try 3% hydrogen peroxide, 4 times a day (dropper full) for a week. If that does not help see an ENT doctor for proper and safe removal by either microscopic removal or simple irrigation of warm water.