I am using three main sources for my holidays:
1) Holiday Insights (link) - This is the main site I use to track and pick daily holidays.
2) Brownie Locks (link) - This web site has essentially the same information, however it includes many additional monthly and weekly holidays. I keep these around to fill in when there is a day without a holiday.
3) Bizarre Holidays App for iPhone (by Michael King) - This app is $.99 and has been worth every penny. It is a nice compliment to Holiday Insights since there are instances when one has holidays the other does not. It gives us more to chose from and gives a list to use when we are traveling. Very handy!
That's about it. There are many holiday websites and apps to check out, just google or search for "bizarre holidays" or "weird holidays". In the interest of keeping things simple, I just chose these to work with because there are A LOT of wacky holidays out there!
Have fun celebrating!