“Follow your passion” is indeed too simplistic, or too general. It leaves a lot of natural steps or progressions of life out of the equation.

It should perhaps sound more like “find your passion” or “discover your passion”…which as you point out, derives from curiosity.

“Follow your passion” isn’t necessarily bad advice, but incomplete advice to give someone.