I had similar issues when syncing my Microsoft Outlook based calendar – however the amount of days out was consistently only one day although I do recall that some all day events ended up spanning 2 days.
I was somewhat disappointed after the iOS5 upgrade that this issue persisted.
The issue persisted even after no longer syncing with Outlook (via iTunes) but with the iCloud.
However the good news (and the point of my post!) is that once I deleted all the Calendars and started completely afresh starting with a new calendar on iCloud I have completely solved this issue.
It took me an hour or so to recreate all my birthdays, all the NZ holidays and other memorial days etc, and all my existing (and previous while I was at it!) appointments.
Since then my calendar has completely behaved itself. No events or appointments are ever on an incorrect day and all day events now behave perfectly.
If you are still having issues with incorrect days relating to an iCloud calendar – have you made sure to set your time zone in the iCloud settings? (click your name in the main iCloud screen to get to settings).
Hope you manage to get this issue solved – I remember how frustrated I used to be before iCloud came along!