Rotation data is stored in the image's EXIF information. Shouldn't matter which website it's on, unless your web server software modifies the files as it sends them out. I put my images on my own website as well and have had both landscape and portrait images work fine.
What software do you use to view/edit on your computer?
I'm wondering if perhaps your camera doesn't add orientation information (all pics show up on your computer originally as landscape) or perhaps the orientation of the camera confused it (since your test looks like you might have been pointing up at a fairly high angle) then whatever software you used to view and rotate the image just saved that info in a separate file/database rather than adding it to the EXIF info for the image.
You might try a different image editing program to set the orientation. I normally tweak my pics inside iphoto (I use a mac) but also use GIMP (free sorta-Photoshop type program).
This is a portrait orientation image I posted (and host on my own site) a while back: