Choosing the shape of your bolster, as with many other decisions, boils down to personal preference. Let's start with the differences between the two styles:
the flat surface is ideal for poses requiring stable support - such as in Viparita Karani
rectangle bolsters tend to be about 6" in height
the lower height will allow you to fold further into forward bends
tend to be a bit larger with a height of 10"
the taller height provides greater support for those who find forward bends challenging
the larger diameter and the round shape will tend to open up the chest more when the bolster is placed under the spine
But either shape bolster will provide support and stability and can be used in the variety of poses interchangeably. If you have the opportunity to try both at your local yoga studio, please do so. Use both in the variety of poses to see whether you are more interested in the stability of the rectangular bolster or the greater expansion you may get from the round, and so on. Keep in mind which poses you believe you will want to be incorporating into your home practice as well.