Just guessing here but...
The edge you get with a rough stone is not "sharp" in the sense of a smooth, continuous edge. Under magnification, it would look like a lot of jagged pointy teeth which can cut because their tips are sharp but which really cuts like fine saw. Finer stone = smaller pointy teeth.
If you're pushing the knife edge into the sandpaper, the paper - being flexible - is likely rising like cloth before a chair leg and cutting across your edge at a blunt angle, removing the sharp teeth and leaving a dull edge. If you're pulling the edge away from the sandpaper, then it is raising a burr on the edge (essentially bending the teeth and a ridge of metal upward) so that the resulting edge is spread out and not sharp.
If the latter case, steeling the edge might grind down the burr resulting in, again, a lot of little teeth more or less aligned, or a "sharp" blade. If you're pushing the blade forward on the sandpaper, I don't know what the steeling would do.
If the blade is "sharp" in the sense you want it, then you're fine to just keep doing whatever you're doing. Aficianados of sharp usually will apply a fine, hard stone to the edge to try to get rid of the teeth and replace it with an actual edge, hopefully no more than a micron or two across. That's...shaarrrp.
FYI, one test of sharpness which distinguishes between teeth type sharpness and edge type sharpness is to bend a piece of regular paper (or newspaper for a more challenging test) about 1/2" (..1 cm..) from the edge so it sticks straight up. Then take your knife and push it gently straight down against the raised edge of paper. A really sharp blade will cut it with just downward pressure, no sawing back and forth.
To get a really sharp edge, learn about edge profiles so you understand what you're grinding, and get a fine, hard stone to properly finish the edge.
Have fun...
[BTW - I'm just AMAZED at your (and many others out there) English skills, unless you happen to be a native English speaker living out their in Finland...]
