Annuals grow very quickly from seed. Sown now you could have a garden of flowers in six weeks' time.
However, what I would recommend would be to nip out to the garden centre and buy red valerian, lavender and the perennial wallflower "bowles mauve" for an instant effect that will flower for a long period this year and for years to follow. Each can be propagated extremely easily from cuttings too so you can have a whole garden full of plants for very little outlay in the long run.
If your buddleia doesn't put on a bumper display (as long as it is on decent soil and watered when necessary) I will eat my keyboard!