FreewayTalk
10 replies to this thread. Most Recent
How can we make our web sites secure?
Does anybody know how on earth can we secure our web sites so that its not possible for people to come along with some software like http://www.sitesucker.us/ and simply download the whole lot?
It strikes me that its utterly pointless going to great lengths trying to make a server hacker proof and adding index files to directory folders if all you need to do is insert a url and then press a button to download an entire site in seconds. Photo libraries keep hi-res files on the server ready for downloads and who knows what data banks must keep stored so there must be a way of blocking this practice. Does anybody know the answer? I know this isn’t specifically related to Freeway but it does nevertheless relate to Freeway users as much as anyone else.
:?
Mac Pro 8 Core with 32 gigs of Ram running 10.7 using Freeway 5.6.1 Pro
Sometime around 3/8/07 (at 08:05 -0400) cornishman said:
[quote:1418c354cb]Does anybody know how on earth can we secure our web sites so that its not possible for people to come along with some software like http://www.sitesucker.us/ and simply download the whole lot? [/quote:1418c354cb] Simple: take the sites offline.
Seriously, if something is publically viewable it is downloadable. That’s what happens when someone views a page with a browser: everything referenced in the page code, and the page code itself, is downloaded to the viewer’s computer. It has to be!
Photo libraries only allow controlled access to the high-res images they store. That’s through one of a number of forms of password-style control. But that’s not even remotely feasible if you want to make your site accessible to people and search engines.
If you don’t want something to be openly viewable, don’t publish it.
k
This list is maintained by Softpress Systems - http://www.softpress.com
What Sitesucker does is not so different from what your webbrowser does. The reason we create sites is to make them available for people to download and view. Content that you wish to keep private needs to be in a protected folder, or maybe simply not on a server (-a computer that serves up files) at all. Bjorn On Aug 3, 2007, at 8:05 AM, cornishman wrote:
[quote:47958a1090]Does anybody know how on earth can we secure our web sites so that its not possible for people to come along with some software like http://www.sitesucker.us/ and simply download the whole lot?
It strikes me that its utterly pointless going to great lengths trying to make a server hacker proof and adding index files to directory folders if all you need to do is insert a url and then press a button to download an entire site in seconds. Photo libraries keep hi-res files on the server ready for downloads and who knows what data banks must keep stored so there must be a way of blocking this practice. Does anybody know the answer? I know this isn’t specifically related to Freeway but it does nevertheless relate to Freeway users as much as anyone else.
:?
Freeway Pro 4.3.2
[/quote:47958a1090]
This list is maintained by Softpress Systems - http://www.softpress.com
I downloaded and tried "SiteSucker" on a small site I had made recently. The site uses a small amount of Flash content, and although SiteSucker pulled in the bulk of the site, it failed to grab the Flash content.
Flash offers a publishing option that enables you to block download. It’s not extremely hacker proof, so if SiteSucker is blocked by Flash, I wonder if simple measures are enough to protect the other content. Of course, keep in mind that to be viewable at all, you need to be exposed. You could make all your content out of images, but for numerous reasons that would be impractical.
Just a few (not terribly educated) guesses,
~ Bill
On Aug 3, 2007, at 8:05 AM, cornishman wrote:
[quote:e39ad8c801]Does anybody know how on earth can we secure our web sites so that its not possible for people to come along with some software like http://www.sitesucker.us/ and simply download the whole lot?
It strikes me that its utterly pointless going to great lengths trying to make a server hacker proof and adding index files to directory folders if all you need to do is insert a url and then press a button to download an entire site in seconds. Photo libraries keep hi-res files on the server ready for downloads and who knows what data banks must keep stored so there must be a way of blocking this practice. Does anybody know the answer? I know this isn’t specifically related to Freeway but it does nevertheless relate to Freeway users as much as anyone else.
:?
Freeway Pro 4.3.2
[/quote:e39ad8c801]
This list is maintained by Softpress Systems - http://www.softpress.com
You can do a little bit to keep this from happening, but probably not enough to make it worth your effort. The thing is, if you want your site to be visible on the Web, you have to allow people to download it. After all, that is precisely how a Web browser works—it downloads the HTML file and associated graphics to the client computer, and displays them. The fact that it only holds onto these files in its cache for a short while is an implementation detail. A download does take place in order to display the page. Clearly, you don’t want to block everyone from seeing your page at all.
The things to look into are called user-agent strings, which are sort of like browser fingerprints, and you can test using SiteSucker and Interarchy to see what user-agent they send to the server software as they initiate the download. If you spot one, you could redirect to another page and that would be the end of it. This is something you will need to code on the server.
Let me know off list if you want more detail about this.
Walter
On Aug 3, 2007, at 8:05 AM, cornishman wrote:
[quote:3478c237e7]Does anybody know how on earth can we secure our web sites so that its not possible for people to come along with some software like http://www.sitesucker.us/ and simply download the whole lot?
It strikes me that its utterly pointless going to great lengths trying to make a server hacker proof and adding index files to directory folders if all you need to do is insert a url and then press a button to download an entire site in seconds. Photo libraries keep hi-res files on the server ready for downloads and who knows what data banks must keep stored so there must be a way of blocking this practice. Does anybody know the answer? I know this isn’t specifically related to Freeway but it does nevertheless relate to Freeway users as much as anyone else.
:?
Freeway Pro 4.3.2
[/quote:3478c237e7]
This list is maintained by Softpress Systems - http://www.softpress.com
Freeway user since 1997
There is a site I will be putting together soon which will have content available for sale and immediate download. I am not so much worried about the images which will be placed in the general gallery part because they will have a "preview" watermark, making them less interesting to anybody who simply wants to drag them to the desktop, while I’ll use the Image Guardian action for Freeway as an additional means to to protect the files. Image Guardian will use some DHTML commands to disable right click etc but that only works in a browser where Java is enabled. The bit which concerns me are the non watermarked images which will have to be kept somewhere on the server for when a purchase is made. It feels a bit like locking the door of a house but then giving the key to a burglar.
Because of the way that Freeway works, all content gets automatically added to the Resources folder so its only necessary to type in something like www.whatever.com/Resources to see the full contents of the site. The html source code will tell you where to look. You can get around that by placing an index file in the directory folder, so a casual searcher will be blocked but I believe sitesucker can rip through that without blinking. Please correct me if I am wrong here. I hope I am. In an older version of Sitesucker I saw it had the user agent feature but that now seems to be gone and lets not forget there are a dozen bits of software out there that essentially do the same thing so it would probably be impractical to exclude anybody who appears to be using a certain browser.
Speaking to a friend a little earlier who produces a lot of e-commerce web sites he suggested that the final un-watermarked images be stored in a database using MySQL 5 or better still PostgreSQL, which I have never heard of before. I know flash tends to be safer but its impractical in this case and I have seen software at versiontracker which will simply download the flash made film content on youtube and convert it to some basic film format like .mov. I am still looking into this but it seems clear that a web site like Amazon must be using some kind of iron clad setup which safeguards them from this kind of practice.
Mac Pro 8 Core with 32 gigs of Ram running 10.7 using Freeway 5.6.1 Pro
The only real solution is to deface the images with a watermark.
Anyone can do a command-shift-4 and take what ever they want, as long they can see a good enough screen image, albeit low resolution.
Perhaps trade clients to look at images, if they part with their email address and details as a prospective client. And then a URL is e-mailed to them, after reading and agreeing to a stern copyright notice.
David
www.ineedwebhosting.co.uk | Twitter Offers and Web Design News
Freeway Friendly web hosting and Domain Names. (Create an account it’s Free!)
PrintlineAdvertising.co.uk | The Printline Advertising Blog
Print Design, Digital and Litho Printers, Promotional Merchandise and Corporate Gifts.
There are tons of different approaches to this problem, it is so common in the Web world of public access + private (or for-sale) content.
One way to do this is to put a gatekeeper between your users and the real content. When I built the store for Softpress many years ago, I used this technique for the downloads of the "full" versions of the Freeway software.
On your server, you make a folder somewhere "above" the site root. If your web site files are all in a folder called /Users/yourname/Sites/sitename, then simply make a folder in /Users/yourname called files (or whatever you like). You should be able to write to this folder, but if not, ask your hosting provider where you can do this.
The way this works is that Apache is only going to serve files to the public that are inside the site root (/Users/yourname/Sites/sitename in the above example). Anything that is not inside that folder (or one of its child folders) will simply never get served out.
Then, you need your gatekeeper. I would do this in PHP, but it’s trivial to do in most dynamic languages. Your gatekeeper should be able to look up customers and what they’ve bought from your sales system. I’m not going to show you actual code below, this is just sort of a block diagram of what you need to do.
The browser gets a request like this:
get_file.php?id=12345&file=67890
get_file.php takes those two variables, id and file, and looks them up in a database:
Did user 12345 purchase file 67890 from us?
If no, then display a stern warning or whatnot.
If yes, then issue a file request for the real file, the actual filename of which was gotten in the previous lookup, and stream it to the browser (actual code ahead!):
$file = file_get_contents(‘/Users/yourname/files/mona_lisa.tif’); header("Content-type: image/tiff"); header("Content-length: " . strlen($file)); print $file;
The file is only sent to valid purchasers, and no amount of trickery (short of hacking your store application somehow) is going to get the false credentials into the database so the file will be sent.
Note that this is a very simplified version of the Softpress store, which also limits you to a 2 week download window, in addition to further cryptographic goodies to keep people from sharing URLs with all zillion of their friends. As always, scale the effort to the potential loss.
Walter
On Aug 3, 2007, at 12:34 PM, cornishman wrote:
[quote:d8f08fe0a4]There is a site I will be putting together soon which will have content available for sale and immediate download. I am not so much worried about the images which will be placed in the general gallery part because they will have a "preview" watermark, making them less interesting to anybody who simply wants to drag hem to the desktop, while I’ll use the Image Guardian action for Freeway as an additional means to to protect the files. Image Guardian will use some DHTML commands to disable right click etc but that only works in a browser where Java is enabled. The bit which concerns me are the non watermarked images which will have to be kept somewhere on the server for when a purchase is made. It feels a bit like locking the door of a house but then giving the key to a burglar.
Because of the way that Freeway works, all content gets automatically added to the Resources folder so its only necessary to type in something like www.whatever.com/Resources to see the full contents of the site. The html source code will tell you where to look. You can get around that by placing an index file in the directory folder, so a casual searcher will be blocked but I believe sitesucker can rip through that without blinking. Please correct me if I am wrong here. I hope I am.
In an older version of Sitesucker I saw it had the user agent feature but that now seems to be gone and lets not forget there are a dozen bits of software out there that essentially do the same thing so it would probably be impractical to exclude anybody who appears to be using a certain browser.
Speaking to a friend a little earlier who produces a lot of e-commerce web sites he suggested that the final un-watermarked images be stored in a database using MySQL 5 or better still PostgreSQL, which I have never heard of before. I know flash tends to be safer but its impractical in this case and I have seen software at versiontracker which will simply download the flash made film content on youtube and convert it to some basic film format like .mov. I am still looking into this but it seems clear that a web site like Amazon must be using some kind of iron clad setup which safeguards them from this kind of practice.
Freeway Pro 4.3.2
[/quote:d8f08fe0a4]
This list is maintained by Softpress Systems - http://www.softpress.com
Freeway user since 1997
Sometime around 3/8/07 (at 12:34 -0400) cornishman said:
[quote:cba83ad5be]You can get around that by placing an index file in the directory folder, so a casual searcher will be blocked but I believe sitesucker can rip through that without blinking. Please correct me if I am wrong here. I hope I am. [/quote:cba83ad5be] To my knowledge, nothing can persuade a server to deliver a directory listing if it doesn’t want to or has a default page in there. But of course software can look through any web pages to find all the files that they reference. It can build up a pretty decent picture of what’s in the Resources folder like that, and when it knows a filename to request… it just has to request it.
k
This list is maintained by Softpress Systems - http://www.softpress.com
I posted my tuppence on this thread at 1:21pm, but it didn’t show up for perhaps seven hours. I’ve noticed a delay before, but this one was a bad ‘un. Anyone have any ideas?
k
This list is maintained by Softpress Systems - http://www.softpress.com
Wow, there is a lot to learn here and I suspect it may well be more that I am able to handle on my own so I shall have to seek some help from somebody who has done this kind of thing before. My server managers told me it simply wasn’t possible which doesn’t inspire great confidence but I shall take note of all these suggestions and try to see what else I can dig up. I’ll report back if I discover anything more.
Mac Pro 8 Core with 32 gigs of Ram running 10.7 using Freeway 5.6.1 Pro
