In idiomatic English, "actually" in the context you used it suggests a contrast to something else. "I'm considering asking Bob to do this job." "You should ask

In idiomatic English, "actually" in the context you used it suggests a contrast to something else. "I'm considering asking Bob to do this job." "You should ask Sally, she actually knows what she's doing."