You can but...
In soaps, olive oil is used for it's moisturizing capabilities, and coconut oil for its lather and cleansing properties.
However, these oils have reacted and transformed during processing with lye to maximize these characteristics. Unprocessed oils may not be harmful to your skin
(unless you are acne
prone), but may not provide the desired results in unprocessed form.