
Polished and tumbled stones are in abundance on-line. Credit card buying on-line however means you don't get to see it and handle it before purchasing. You pay your money, you take your chances. I have had very good luck on Amazon, and a couple stinkers because I didn't correctly read the descriptions. My bad in each case.
China is selling a lot of polished and shaped stones on Amazon and through US customs delivery time is 5-8 weeks. India also sells gems on Amazon through Royal Mail with about 4weeks delivery time, and I love their fragrant packaging. Brazil mines are also selling bulk and I just received 1/2 pound crystal points, each the size of my thumb, that I am happy with. Always check the reviews and shipping fees. There is a thread on EUTM about favorite suppliers online.
Local hobby shops, museum gift shops, lapidary "rock shops", rock and gem shows are some places to check. Tourist traps excel in pretty stones. Plan a trip if you have resources for it. Take a cave tour or visit the site's tourist gift shop.
You can find unpolished and semi-polished stones to buy in garden centers, landscape suppliers and gravel yards. Water polished stones can be found by rivers and the sea.
Don't overlook right under your feet. Cemeteries and construction sites...any place where digging is going on. They may not be glossy and pretty until you take time to scrub them and rub some olive oil on them.
As for which stones to include in your kit, a good place to start is with the chakra/rainbow colors (red, orange, yellow, green, blue, purple) and then a clear quartz all-purpose stone, and a black stone for grounding. Some places sell the whole rainbow in a pouch, and in these cases I think a thumb-stone size is good.
Oh gosh, I just love stones and can go on and on ad nauseum.