Writing Trials: How Long Is a Piece Of String?

I've been wondering lately how authors know how long the thing they are about to write will be. Or even if they try to. How do they set out to write a short story, or a novelette, or a full-blown novel?

This came up because of the story I'm plodding along with which has not turned out how I expected . It seemed to be a 5000 - 8000 word story in the initial idea stages, even when I started plotting it out. Now? It's looking more like a 20,000 word story, or even longer. I'm about three scenes in and I'm already at 6000 words with about ten more scenes to go. Assuming I don't come up with any more whizzy dizzy ideas for it...

This has told me one thing: I am rubbish at working out how long a story is likely to be before I start writing it. This shouldn't really be a problem, as a story needs to be what it will be and I fully embrace that. But I want to write more short stories (5000 - 8000 words), so I can send them out and get feedback (read: rejection slips) before I finally manage to sell one or two to short story markets. This is very difficult to do when you continually end up writing longer pieces. Not to mention frustrating.

I've found the planning tools in Scrivener has helped me see when a story is likely to be on the longer side, but it doesn't really cover the fact that I come up with most of my good ideas during the writing process itself. I pull a scene here, add another there and before you know it: *POOF* I have a novel on my screen. I've seen a lot of advice on various blogs and sites saying that new authors should write some short stories, and to keep novel length on the shorter side for the first go. But none that I've found have explained how you can predict how long your story is going to be or techniques to try to limit yourself to working on shorter stories.

I suspect I'm thinking the wrong way about this; if a story needs to be 5000, 10,000 or 50,000 words than it needs to be that length. The one thing I can say I have learned is not to give up - I've left the detritus of a couple of tales lying in my files because they were taking too long to finish and I ended up lost in a quagmire. But this story I'm working on now; nuh uh. Not giving up. I'm enjoying it too much and despite the surprise over the word count I still know where it's going and can't wait to get there.

Maybe that's all the trick is; keep writing, don't give up and eventually you'll have a whole bunch of stories of varying length to send out into the big ugly. Something is better than nothing after all.