Will it affect my Speaking score if I have a stammer / stutter?

There is a difference between: medical stammering/stuttering (a neurological condition) and stammering and stuttering because of nervous false starts (something many students do).

If you have the neurological condition of stammering/stuttering, you should inform your IELTS test centre about this as soon as possible. They may ask to see a note from your doctor or other documentation, but even if you don't have this, it is worth talking them about so that they can take it into account when scoring your test

If you make nervous false starts (where you hesitate and repeat yourself), this is because of unfamiliarity with speaking English out loud. You are given a score for Fluency & Coherence, and this will be reduced by a lot of hesitation, repetition and self-correction.

The solution is practice; the more practice you do, the more your confidence will grow, and this will reduce your nervousness. 

Practice includes speaking with someone else who speaks English or, as many of our students work alone, just recording your answers by yourself is a good activity. Memorising answers is a bad idea. You need to practice thinking of ideas on the spot and then expressing them as naturally as possible.