Rivets are often used because they have a lot of holding power. Rivets are useful for lots of projects where you need to fasten thin things securely together that don't ever need to be separated again. The rivet head has a nice cosmetic finish, too, as compared to a screw.
Rivets have a mandrel (the thin stalk), a rivet body (the larger diameter part), a flange (the round frisbee-shaped disc), and a rivet head (the plug at the end of the rivet body which is attached to the mandrel).
To use it, you first make correctly-sized holes in the two things you want to rivet together (the "target" material). Then you stick the rivet's mandrel completely into the rivet gun. Then you stick the rivet body through the holes in the target until the flange rests flush against the target. Then you you squeeze the handles of the rivet gun, which causes it to grab onto the mandrel and pull it back hard. This pulls the rivet head through the rivet body tube, which expands it. When the rivet head meets up to target, the pressure increases until the mandrel snaps off inside the flange. The expanded rivet body tube is larger than the hole in your target, so it's locked in place. For some of my designs (the Pinwheel, the generic Cannon, the Spinner), you want to leave the mandrel in place, so stop squeezing the rivet gun as soon as you feel a change in the resistance. If you continue to squeeze harder, you'll pop off the mandrel. You can find more detail in this Youtube video.
You can buy rivets that have varying lengths of rivet bodies for gripping materials together of different thicknesses ("grip range"), and with different diameters of rivet body (usually 1/8th inch, or 3/16ths inch), and with different thicknesses of mandrel (smaller on the 1/8th inch wide rivet than the 3/16ths inch), and with wide-or-normal flanges. For some applications you might want a shorter rivet body (Coinbank, Spinner) and others a longer one (Pinwheels). Wide flanges distribute more force over a larger area, so they're good for binding soft materials like plastic or thin aluminum together. You want to be sure to buy aluminum rivets with an aluminum mandrel, instead of steel ones, because the mandrel will break off easier without distorting the can too much or wearing out your hands and arms.
Rivets are inexpensive and very easy to install. They cost about $0.03 each when bought in bulk (500 rivets) from a site like Grainger.com, and about $0.11 each when bought in small quantities (50 rivets) at Home Depot. Rivet guns are cheap, too. You can pick one up for about $11 from Home Depot.
Probably the hardest thing about using rivets is making the appropriate-sized hole in the target material. If it's too big, the rivet won't hold or might spin. If it's too small, you can't push the rivet in. Getting the right size is particularly important when you're relying on a friction-fit to hold the rivet in place, rather than using a rivet gun.
I find it easiest to make a starter hole in the correct spot with the pushpin, then enlarge that hole with the punch and then a series of larger nails until I can pass a size 6D (#11) nail through the hole. If I wiggle that nail a bit in a circle about its axis, it will enlarge the hole to the right size for the 1/8th inch rivet body. I can then force the rivet body through the material, attach my rivet gun, and pull.
Here are some examples of rivets being used in my designs:
Remember, this isn't just fun; it's riveting!