Something I keep forgetting to suggest whenever someone says; "I got a new (insert animal here)"
Find out where your nearest emergency vet clinic is. These should be open 24/7 including holidays. Get their contact information and put it in a visible place, like on the fridge, and keep a copy in, say, your purse. Because if/when you need it, you are going to be so panicked that you won't be able to look it up on the internet/phone book/whatever. And have a good idea of how to get there; plan your route.
ETA; hedgehog says keep this in mind if you move too. And Laliana says "I would advise saving the number in your phone."
And Ieatmyyoung recommends a slush fund for emergencies and keeping a record of vet visits for exotic pets. And I'd add if your pet is on special med's or has special needs too take that info. with you.
TheBoogieCat has given the poison control link from the ASPCA;http://www.aspca.org/pet-care/anima…
NotoriousGiGi has recommended bookmarking the list of toxic/non-toxic plants from the ASPCA;http://www.aspca.org/Pet-care/poiso…
Dazzle Me has just added a comment about ideas for a vet fund; see comments.
Itsabadgeralright is advising pet insurance. I have mixed feelings about pet insurance; many policies won't cover animals over a certain age, or with pre-existing conditions, and won't cover regular check-ups. I have *cough* several *cough* pets, so for me the cost would be prohibitive. But, if you have only the one pet, it certainly gives peace of mind.
ETA; AT THIS POINT, I'M HAVING TROUBLE KEEPING UP WITH ALL THE GREAT SUGGESTIONS, SO ONCE YOU'VE READ THE POST, READ THE COMMENTS TOO :) THANKS EVERYONE!!