To Fine Tune or Not Fine Tune? That is the question