Hotlink Protection Causes Inadvertent 403 Errors?

[schussat]

Before leaving on vacation a couple of weeks ago, I found someone linking to one of my images from a remote forum. I turned on hotlink protection and thought the issue was resolved, but now that I'm back, I realize that some visitors to my site are unable to get to images and even certain plain html pages.

 

I'm seeing lots of errors that look like this:

>s216638-1.savvis-internet.uslsan3-bsn.savvis.net - - [02/Aug/2004:12:10:00 -0500] "GET /images/37.jpg HTTP/1.1" 403 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"

 

Now, I understand that without a referr from my own site, the 403 is the correct response given that I'm using hotlink protection-- at least, I think that's the way it should work. However, as far as I know, these images are being called from a popup window generated by my own site. You can see it in action (or, at least, try to see it in action at this address (which is just a blog post with some popup jpgs): http://schussman.com/article/793/strawberry-ca-pop-150

 

The twist is that both from home and from work, I can get to the photos just fine. My wife can't get to them from her office, and apparently other visitors can't see them, either, as someone posted a comment to that effect. So I'm sort of stumped -- why would this 403 error appear only sporadically? I'm guessing that turning off hotlink protection will resolve this, but I'm not sure I want to do that. For reference, here's what I think is the relevant part of my .htaccess:

>RewriteEngine on
RewriteCond %{HTTP_REFERER} !^http://melissa.schussman.com/.*$      [NC]
RewriteCond %{HTTP_REFERER} !^http://melissa.schussman.com$      [NC]
RewriteCond %{HTTP_REFERER} !^http://schussman.com/.*$      [NC]
RewriteCond %{HTTP_REFERER} !^http://schussman.com$      [NC]
RewriteCond %{HTTP_REFERER} !^http://www.melissa.schussman.com/.*$      [NC]
RewriteCond %{HTTP_REFERER} !^http://www.melissa.schussman.com$      [NC]
RewriteCond %{HTTP_REFERER} !^http://www.schussman.com/.*$      [NC]
RewriteCond %{HTTP_REFERER} !^http://www.schussman.com$      [NC]
RewriteRule .*\.(jpg|jpeg|gif|png|bmp)$ - [F,NC]
RewriteRule   ^index\.xml$            index.php?rss=1     [L]
RewriteRule   ^atom\.xml$             index.php?atom=1  [L]

I'd love to hear any suggestions! Thanks-

[TCH-Rob]

You got me, we didnt forget you I just dont have an answer yet. Hang tight, we should be able to come up with something.

[TCH-Don]

When I did use hotlink protection via cpanel

I would find occasionally it would stop images from loading even from my site.

I would have to disable it and wait a while to re-enable it again.

[schussat]

Hi guys-

 

Here's a followup from my end. I continued to poke into it, and eventually tried using the following code:

>RewriteCond %{HTTP_REFERER} !^http://(www\.)?schussman.com/ [NC]
RewriteCond %{HTTP_REFERER} !^-?$ 
RewriteRule .*\.(jpg|jpeg|gif|png|bmp)$ - [F,NC]

The second line of which should allow blank referers -- which I thought was the problem. When I used that code, some viewers encountered a new but still not-quite-right phenomenon: no images were displayed inline on my site (ie, banner images and thumbnail images got 403, forbidden, errors), but when they click on the thumbnails to popup a jpg, the popup image appeared properly! Freaky, and I can't figure it out. Is this due to some weird firewall or cache on the other end, perhaps? Is there some other way to to hotlink protection, other than using mod_rewrite? Would the presence of my other rewrite lines

>RewriteRule   ^index\.xml$            index.php?rss=1     [L]
RewriteRule   ^atom\.xml$             index.php?atom=1  [L]

cause the preceding lines to go haywire?

 

For now, I've resorted to watching my logs for hotlinking and then manually restricting the offending referers. This works fine, but is of course not as easy as just stopping all hotlinking.

[bellringr]

Take a look at this thread

 

It may or may not help, but this is one thread I remember where we discussed the same problem a while back.

[schussat]

Take a look at this thread

Thanks for the pointer, Bellringr, I had missed it. It's a good discussion, but still leaves my main issue unresolved: I don't know why what looks like correct code prevents some (legitimate, non-hotlinking) visitors from being able to view images.

 

Anyway, I have a working interim solution for now. It would be nice to make sense of the problem at some point, but it's not an ongoing concern right now, at least. Thanks!

[bellringr]

This is probably a dumb question, but how do you find out if someone is linking to one of your images?

[schussat]

This is probably a dumb question, but how do you find out if someone is linking to one of your images?

When an image is loaded by a genuine visitor to your site, the log should show your own URL as the referer, so you can detect hotlinking by watching your logs for accesses of images using a remote (ie, not from your own URL) referer. You can also watch for spikes in bandwidth and scan through the "last visitors" list, as it will sometimes show a pattern of an image or file being repeatedly accessed. Finally, I think there are some log tools that scan automatically and detect likely hotlinking, but I don't think that AWStats does this.

[TCH-Rob]

I periodically go through my logs and in the list of those that link to you I have found a URL that I did not recognize so I visited and found an image on their site.

[TCH-Dick]

Check out this article.

Smarter Image Hotlinking Prevention

[TCH-Rob]

That be interesting.

[bellringr]

So if I'm looking at the recent visitors and I see one of my photos in the URL field, the referer would be my own domain if they were just visiting that site. But if they were hotlinking, instead of saying ht*p://www.****/pictures.htm it would say ht*p://www.theirdomain.com/whatever, correct?

[schussat]

Bellringr, that's exactly right. Once you've identified what you think is a hotlink, you can confirm it by searching your log for additional instances of the referer -- usually, there will be lots of them, all for the same image and with the same referer.