Depends. If it is just fever and the baby is otherwise acting normally, without cough, fussiness, pulling at ear, not eating, crying uncontrollably or other symptoms present, you should wait it out. If you have specific questions, call and speak with a pediatrician, and if they suggest it, go and see them.
You decide. If your mom sense tingles that something terrible is going on you must.However, there are pro's & con's.Most baby fevers are transient self healing viruses.If the kid is playful,eating/drinking okay it is fine to monitor the situation at home & call your doc's service for advice.The ER is filled with car crashes/heart attacks/contageous disease/etc.You would be relegated to the minor problem group.