The main stitch you need to know to stitch a penny rug is called a blanket stitch. It looks like this:
There is a video here that will take you through the steps to make this stitch. It is a great stitch for a lot of other things besides penny rugs. http://www.needlenthread.com/2006/11/buttonhole-stitch-video-tutorial.html
You can find the basic instructions for making a penny rug here. http://pennyrugsandmore.blogspot.com/2009/05/makings-of-penny-rug-by-colleen.html
These penny rug coasters are an easy beginner project. http://weefolkart.com/?q=node/150
The blanket stitch is a versatile stitch and can be used as an edging on a lot of other projects. Use it to edge a sweatshirt jacket, a baby blanket, or to applique one fabric on to another.