Preheat a grill to between 300 and 350 degrees F.
Put potatoes, onion, bell peppers, and rosemary in a large bowl and toss to combine. Coat with olive oil and toss again. Add salt and pepper, to taste.
When grill is preheated, place a medium skillet over the fire. Pour the contents of the bowl into the skillet. Continually stir potatoes until cooked thoroughly, approximately 25 minutes. Carefully remove hot skillet from grill and serve.