Syndication Security Problem
November 13, 2005 4:47 AM Subscribe
SystemFilter: As mentioned before, I have a sudoku website, where I host tons of puzzles that I have generated. I am in the midst of writing a system to syndicate these puzzles, so anybody can embed my puzzles and solver in their site by using a bit of JS code (similar to google adsense). I've got a bit of a security problem that I can't quite solve...
My system is quite similar to the way AdSense works, and I've got quite far already. You setup your syndication account and get given a snippet of JS, and a unique ID, to load the puzzle into your site. Each user will get a few hundred puzzles per day for free, so it's mainly targetted at small website operators.
Here's the pinch. Because it's a block of JS code, I could go to X's website, copy the JS code, embed it in another site (client Y) and syndicate puzzles to that site, using X's client ID. Client X only has 250 puzzles per day and they're getting used by client Y, who is effectively stealing them from client X, so client X is getting hard done by. How can I prevent this?
The actual solver component is flash, so I can embed some intelligence in that, or customise it quite easily for each client. Perhaps making the solver check against what URL it is supposed to syndicate puzzles to? I don't believe in security through obscurity, so I would prefer something that is simple and transparent.
My system is quite similar to the way AdSense works, and I've got quite far already. You setup your syndication account and get given a snippet of JS, and a unique ID, to load the puzzle into your site. Each user will get a few hundred puzzles per day for free, so it's mainly targetted at small website operators.
Here's the pinch. Because it's a block of JS code, I could go to X's website, copy the JS code, embed it in another site (client Y) and syndicate puzzles to that site, using X's client ID. Client X only has 250 puzzles per day and they're getting used by client Y, who is effectively stealing them from client X, so client X is getting hard done by. How can I prevent this?
The actual solver component is flash, so I can embed some intelligence in that, or customise it quite easily for each client. Perhaps making the solver check against what URL it is supposed to syndicate puzzles to? I don't believe in security through obscurity, so I would prefer something that is simple and transparent.
You could do tracking of referrer strings at your end. Some browsers will not send a referrer string, but let those slide. In any case, if you get referrer strings that aren't vaguely like those of the particular user's site, then reject the request. This will cull 90% of the problem.
Or, better, you could add a layer to your JavaScript include which checks document.location. That'd be more foolproof.
posted by wackybrit at 5:10 AM on November 13, 2005
Or, better, you could add a layer to your JavaScript include which checks document.location. That'd be more foolproof.
posted by wackybrit at 5:10 AM on November 13, 2005
The JavaScript is actually being downloaded by, and executed on, the enduser's browser. Rather than have the affiliate copy the JavaScript, have the affiliate link to a script on your site.
The enduser's browser will make a request for that script, and will send a referer header when it does. Your server will check the referer and return the Sudoku JavaScript if it's a valid affiliate url that hasn't yet used up its quota of puzzles; otherwise you'll return a JavaScript that alerts the user that the referer url is invalid.
Of course the enduser can fake (or remove) a referer url, but I don't believe that a site can tell a browser to use a fake url. And individual endusers aren't going to go to the trouble of faking a referer header just to play Sudoku.
A better solution would involve the affiliate making a secure authenticated connection to your site to get puzzle data, but that requires the affiliate to do more than just add a JavaScript to his page.
posted by orthogonality at 5:26 AM on November 13, 2005
The enduser's browser will make a request for that script, and will send a referer header when it does. Your server will check the referer and return the Sudoku JavaScript if it's a valid affiliate url that hasn't yet used up its quota of puzzles; otherwise you'll return a JavaScript that alerts the user that the referer url is invalid.
Of course the enduser can fake (or remove) a referer url, but I don't believe that a site can tell a browser to use a fake url. And individual endusers aren't going to go to the trouble of faking a referer header just to play Sudoku.
A better solution would involve the affiliate making a secure authenticated connection to your site to get puzzle data, but that requires the affiliate to do more than just add a JavaScript to his page.
posted by orthogonality at 5:26 AM on November 13, 2005
A document.location check would only be useful if it had a proper URL to check against that wasn't also included in the JavaScript.
You could use AJAX to run a quick db check against the proper list, but XMLHttpRequest sometimes dislikes requests coming outside of their parent domains. It's a bit finicky at the very least.
Also, with the referrer string method, remember that no server-side scripts (if I'm understanding correctly) are being executed to generate this. There's a client-side Javascript and the Flash movie itself, which I believe is stored on his server. Referrer strings also remove the capability to have the game linked to by anyone or by bookmark.
The key challenge here is to verify that the server that is hosting this instantiation of the game is in fact the server linked to the affiliate code. Most simple checking routines would verify things against the client requesting the data, which is why you have to have the two components (hosted page & game script/movie) checked against each other to verify it's authorized.
There might also be a way to restrict "hotlinking" to an SWF by host name using .htaccess...
In fact, you could dynamically write to the .htaccess file the valid host name (valid hosting-server URL) as linked to the SWF file. So, you could have it only allow access to sudoku.swf if the referrer is one of those allowed. Since the state of control would be at the .htaccess level, it would be impossible to counter, especially if you verify things further on the SWF side.
For reference, here's an .htaccess-hotlinking tutorial.
posted by disillusioned at 5:29 AM on November 13, 2005
You could use AJAX to run a quick db check against the proper list, but XMLHttpRequest sometimes dislikes requests coming outside of their parent domains. It's a bit finicky at the very least.
Also, with the referrer string method, remember that no server-side scripts (if I'm understanding correctly) are being executed to generate this. There's a client-side Javascript and the Flash movie itself, which I believe is stored on his server. Referrer strings also remove the capability to have the game linked to by anyone or by bookmark.
The key challenge here is to verify that the server that is hosting this instantiation of the game is in fact the server linked to the affiliate code. Most simple checking routines would verify things against the client requesting the data, which is why you have to have the two components (hosted page & game script/movie) checked against each other to verify it's authorized.
There might also be a way to restrict "hotlinking" to an SWF by host name using .htaccess...
In fact, you could dynamically write to the .htaccess file the valid host name (valid hosting-server URL) as linked to the SWF file. So, you could have it only allow access to sudoku.swf if the referrer is one of those allowed. Since the state of control would be at the .htaccess level, it would be impossible to counter, especially if you verify things further on the SWF side.
For reference, here's an .htaccess-hotlinking tutorial.
posted by disillusioned at 5:29 AM on November 13, 2005
Macromedia has more or less accommodated the scenarios you describe: look into Flash's allowDomain and crossdomain.xml capabilities.
Using allowDomain, you can prevent Flash on unwanted sites from calling MovieClip.loadMovie() against SWFs that you control.
Using crossdomain.xml, you can prevent Flash on unwanted sites from calling XML.load() against XML documents stored on your site.
Since you have a list of registered users, you can construct a list of the domains on which the Flash is allowed to run. Then, apply the proper security schema depending on how the game data is loaded; either each game is a child SWF loaded by a parent or each game is defined in an XML document.
I would urge you to pursue the latter. The Javascript object can be generated to pass along a query string argument to the SWF (i.e. http://www.yourdomain.com/flash.swf?uid=1234567890) which the SWF passes along to the XML query (i.e. http://www.yourdomain.com/gamedata.php?uid=1234567890), and your game data delivery system tracks the number of requests that occur per day. If the crossdomain.xml policy disallows a rogue domain, the game data request will never go through, and the load will never be counted against the valid affiliate. This functionality is built into Flash player itself, so there is little that can be done to circumvent it without using a cracked version of the player.
posted by Danelope at 7:25 AM on November 13, 2005
Using allowDomain, you can prevent Flash on unwanted sites from calling MovieClip.loadMovie() against SWFs that you control.
Using crossdomain.xml, you can prevent Flash on unwanted sites from calling XML.load() against XML documents stored on your site.
Since you have a list of registered users, you can construct a list of the domains on which the Flash is allowed to run. Then, apply the proper security schema depending on how the game data is loaded; either each game is a child SWF loaded by a parent or each game is defined in an XML document.
I would urge you to pursue the latter. The Javascript object can be generated to pass along a query string argument to the SWF (i.e. http://www.yourdomain.com/flash.swf?uid=1234567890) which the SWF passes along to the XML query (i.e. http://www.yourdomain.com/gamedata.php?uid=1234567890), and your game data delivery system tracks the number of requests that occur per day. If the crossdomain.xml policy disallows a rogue domain, the game data request will never go through, and the load will never be counted against the valid affiliate. This functionality is built into Flash player itself, so there is little that can be done to circumvent it without using a cracked version of the player.
posted by Danelope at 7:25 AM on November 13, 2005
Response by poster: disillusioned: It's easy enough to pass the affiliate ID and syndication ID into the flash movie, you just pass it in URL style when you load the movie:
movie.swf?affiliate=123&syndicate=sd1298hwo
They are then available as variable in the _root level. I'm not sure the Kevin Lynch solution is quite what I'm after, I'd like a minimal amount of end-user code. Also, what do you mean by:
Referrer strings also remove the capability to have the game linked to by anyone or by bookmark
Does this mean that if I have a bookmark to a homepage that contains a syndiated puzzle on it, the puzzle won't load because there will be no referrer string? If I load page X and page X loads script Y from my server, when my server serves script Y, will it have no referrer or will it have page X as the referrer? Does the same happen with flash? If page X has flash movie Y on it, and movie Y loads something from my server, does my server see page Y as the referrer?
wackybrit: The server should track each time a given syndicate ID accesses a puzzle. If they've accessed too many it'll serve no more until the following day. It is possible to verify document.location in the JS, but this is too easily circumvented at the client end, just amputate the portion of JS code that checks document.location. The final say about serving the puzzle should happen at the server end, the only portion of the process that is actually fully under my control.
orthogonality: Yes, the end user could fake the referrer URL, this would only allow them to locally access the puzzles, I think you're right about the fact the actual live site couldn't spoof the referrer.
Danelope: If I employ this method, does this mean that I have to recompile the flash every time I get a new affiliate? I'd prefer the flash movie to understand the validation process and be able to validate the affiliate.
From what I read of the comments so far, the JavaScript that the user would need to embed would be this:
<script src="http://vanhegan.net/sudoku/syndicate.js.php?syndicate=62bc6d8182ebd8a">
The script would check the referrer and if it corresponds to the website for that syndicate, then it would server the flash solver. The flash solver would then talk to the puzzle database with the syndicate ID, and perform the same check. If the syndicate matches the referer URL, it serves the appropriate puzzle. This solution works because it's a few small lines of JS for the syndicate, and there's two layers of authentication involved in the process. I suppose I could even throw in the document.location check as well, just for good measure.
Basically there's not much point in trying to steal the puzzles, you can sign up for a syndicate account for free and get free puzzles, or you can get all of them for free on my site anyway.
posted by gaby at 9:21 AM on November 13, 2005
movie.swf?affiliate=123&syndicate=sd1298hwo
They are then available as variable in the _root level. I'm not sure the Kevin Lynch solution is quite what I'm after, I'd like a minimal amount of end-user code. Also, what do you mean by:
Referrer strings also remove the capability to have the game linked to by anyone or by bookmark
Does this mean that if I have a bookmark to a homepage that contains a syndiated puzzle on it, the puzzle won't load because there will be no referrer string? If I load page X and page X loads script Y from my server, when my server serves script Y, will it have no referrer or will it have page X as the referrer? Does the same happen with flash? If page X has flash movie Y on it, and movie Y loads something from my server, does my server see page Y as the referrer?
wackybrit: The server should track each time a given syndicate ID accesses a puzzle. If they've accessed too many it'll serve no more until the following day. It is possible to verify document.location in the JS, but this is too easily circumvented at the client end, just amputate the portion of JS code that checks document.location. The final say about serving the puzzle should happen at the server end, the only portion of the process that is actually fully under my control.
orthogonality: Yes, the end user could fake the referrer URL, this would only allow them to locally access the puzzles, I think you're right about the fact the actual live site couldn't spoof the referrer.
Danelope: If I employ this method, does this mean that I have to recompile the flash every time I get a new affiliate? I'd prefer the flash movie to understand the validation process and be able to validate the affiliate.
From what I read of the comments so far, the JavaScript that the user would need to embed would be this:
<script src="http://vanhegan.net/sudoku/syndicate.js.php?syndicate=62bc6d8182ebd8a">
The script would check the referrer and if it corresponds to the website for that syndicate, then it would server the flash solver. The flash solver would then talk to the puzzle database with the syndicate ID, and perform the same check. If the syndicate matches the referer URL, it serves the appropriate puzzle. This solution works because it's a few small lines of JS for the syndicate, and there's two layers of authentication involved in the process. I suppose I could even throw in the document.location check as well, just for good measure.
Basically there's not much point in trying to steal the puzzles, you can sign up for a syndicate account for free and get free puzzles, or you can get all of them for free on my site anyway.
posted by gaby at 9:21 AM on November 13, 2005
Danelope: If I employ this method, does this mean that I have to recompile the flash every time I get a new affiliate?
Only if you're going the MovieClip.loadMovie() route.
Since you already have a database (and assumedly PHP), it would be simple to query the database via PHP and return an XML document containing the actual puzzle data. Whenever the Flash movie attempts to retrieve a puzzle, it would call XML.load() and first check the crossdomain.xml file hosted on your site. That file would contain a properly-formatted list of all of the domains allowed to access your data, and that list could be automatically generated from registration data.
If the end user attempts to load the Flash within a page on a disallowed domain, Flash security directives will cause that attempt to fail. If that attempt fails, the db query never takes place. If that db query never takes place, the actual affiliate doesn't get dinged for the n puzzles-per-day limit.
posted by Danelope at 9:36 AM on November 13, 2005
Only if you're going the MovieClip.loadMovie() route.
Since you already have a database (and assumedly PHP), it would be simple to query the database via PHP and return an XML document containing the actual puzzle data. Whenever the Flash movie attempts to retrieve a puzzle, it would call XML.load() and first check the crossdomain.xml file hosted on your site. That file would contain a properly-formatted list of all of the domains allowed to access your data, and that list could be automatically generated from registration data.
If the end user attempts to load the Flash within a page on a disallowed domain, Flash security directives will cause that attempt to fail. If that attempt fails, the db query never takes place. If that db query never takes place, the actual affiliate doesn't get dinged for the n puzzles-per-day limit.
posted by Danelope at 9:36 AM on November 13, 2005
Best answer: Here's the workflow I'm talking about:
posted by Danelope at 9:52 AM on November 13, 2005
- Person A wants to become an affiliate
- Person A fills out the registration form, providing their domain in the process.
- The server stores user's registration data and assigns them a unique affiliate ID
- The server adds the user's domain to the crossdomain.xml file.
- Person A receives a Javascript snippet to paste into their page, which constructs a Flash <object> that passes their affiliate ID to the SWF.
- Person B visits Person A's site, loading the Flash movie.
- The Flash receives the affiliate ID via the <object> tag.
- The Flash attempts to retrieve a puzzle, passing the affiliate ID in the query string
- Macromedia Flash player checks crossdomain.xml on your site to see whether the domain from which Person B is visiting is in the whitelist.
- If the domain is accepted, the database query goes through, puzzle data is returned as an XML document, and Person A has used one puzzle for the day.
- If the domain is not accepted, the XML request is rejected, the database query does not go through, puzzle data is not returned, and the affiliate isn't unfairly dinged.
posted by Danelope at 9:52 AM on November 13, 2005
Response by poster: That is absolultey cracking, thanks. I can dynamically build the crossdomain.xml file, I can even run my own site as an affiliate of myself, so I only have to manage one version of a flash solver. Where does the crossdomain.xml file get checked, before or after the loading process? I'd prefer some sort of sensible error message in the event of a domain failure rather than nothing loading at all. I presume that the location of the crossdomain is hardwired in the flash file?
posted by gaby at 11:36 AM on November 13, 2005
posted by gaby at 11:36 AM on November 13, 2005
Flash Player checks the crossdomain.xml file (which must live in the root path of your domain, i.e. http://www.yourdomain.com/crossdomain.xml) the first time the SWF attempts to access XML data on said domain. You can check whether there's been a failure loading the XML and deliver an error message accordingly, but you should have that anyway in case your db goes down for whatever reason, etc.
I'm fairly certain you'll have to list both www.domain.com and domain.com for each affiliate, so that the query works regardless of how the end user accesses that affiliate's site. In Flash 8, I'm fairly certain Macromedia allows for wildcards in both crossdomain.xml and allowDomain(), but Flash 7 requires explicit declaration for each. You should still be compiling for Flash 7 unless you have compelling reasons to do otherwise.
posted by Danelope at 2:52 PM on November 13, 2005
I'm fairly certain you'll have to list both www.domain.com and domain.com for each affiliate, so that the query works regardless of how the end user accesses that affiliate's site. In Flash 8, I'm fairly certain Macromedia allows for wildcards in both crossdomain.xml and allowDomain(), but Flash 7 requires explicit declaration for each. You should still be compiling for Flash 7 unless you have compelling reasons to do otherwise.
posted by Danelope at 2:52 PM on November 13, 2005
Oh, and make sure the Javascript includes the affiliate ID in both the <object> and <embed> tags (assuming you're using the standard embedding procedure; if you're using the Flash Satay method you only have one affected parameter.) Otherwise, the code will work in browsers that use ActiveX (Internet Explorer) but not browser that use plug-in architecture (Mozilla, Firefox, Netscape, Safari, et cetera.)
posted by Danelope at 2:57 PM on November 13, 2005
posted by Danelope at 2:57 PM on November 13, 2005
Response by poster: Oh, and make sure the Javascript includes the affiliate ID in both the <object> and <embed> tags
Yup, I've already fallen foul of that one :)
posted by gaby at 3:41 PM on November 13, 2005
Yup, I've already fallen foul of that one :)
posted by gaby at 3:41 PM on November 13, 2005
As a cautionary note, the crossdomain.xml method recommended by Danelope is vulnerable to an exploit by an unscrupulous affiliate (lets call him Evil). Affiliate Evil can use another affiliate's valid ID (lets call him Victim) instead of his own and the use will be charged to Victim's account instead of Evil's. The use won't be recognized as a forgery because Evil is an affiliate and as such Evil's domain name appears in crossdomain.xml.
You can't directly prevent this, but there are two simple countermeasures that will make Danelope's scheme more secure. First, the exploit I described above required Evil to know Victim's ID (ideally he'd actually want to know many such IDs and rotate through them). You should make sure affiliate ID's are either not guessable (i.e. don't use sequential numbers or other predicatable scheme) or, if you want guessable IDs, require in addition to the ID a token parameter that must match and is generated randomly when the affiliate ID is assigned. While you can't prevent Evil from learning other IDs by viewing the source of other affiliates' web sites, you want to make sure he to expend that effort. Second, because the exploit relies on Evil being a registered affiliate, you have his URL in your database. This suggests a straightforward countermeasure: implement a simple web crawler that periodically visits your affiliate's registered URLs and verifies that they are using the correct affiliated ID and haven't modified your javascript.
posted by RichardP at 4:34 PM on November 13, 2005
You can't directly prevent this, but there are two simple countermeasures that will make Danelope's scheme more secure. First, the exploit I described above required Evil to know Victim's ID (ideally he'd actually want to know many such IDs and rotate through them). You should make sure affiliate ID's are either not guessable (i.e. don't use sequential numbers or other predicatable scheme) or, if you want guessable IDs, require in addition to the ID a token parameter that must match and is generated randomly when the affiliate ID is assigned. While you can't prevent Evil from learning other IDs by viewing the source of other affiliates' web sites, you want to make sure he to expend that effort. Second, because the exploit relies on Evil being a registered affiliate, you have his URL in your database. This suggests a straightforward countermeasure: implement a simple web crawler that periodically visits your affiliate's registered URLs and verifies that they are using the correct affiliated ID and haven't modified your javascript.
posted by RichardP at 4:34 PM on November 13, 2005
Response by poster: I've actually built a number of levels of redundancy into the system, at the JS level, the PHP level and the flash level. The affiliate ID is used as the identifier, and the scripts that are capable of checking the affiliate against the referrer url are used, so even if Evil B tried to use affilliate A's ID, the system would reject them because the system will only allow ID A from the domain A has specified for that syndication channel. If id A comes from any domain other than the one specified, the request is rejected at whichever level it gets through to.
I'm relying on many layers of security. Security in depth, and at every level possible.
posted by gaby at 10:48 AM on November 14, 2005
I'm relying on many layers of security. Security in depth, and at every level possible.
posted by gaby at 10:48 AM on November 14, 2005
This thread is closed to new comments.
If the Flash movie can read the affiliate id, or if you use a named anchor or some such to pass the affiliate id into the Flash movie, then you can have the movie verify against either a database, or within the movie itself, that the current URL matches the set URL for that affiliate id.
If the location href isn't passed properly into the Flash movie, have it generate a Flash-based error explaining the problem.
This problem's been plauging me quite a bit, especially since any standard IP address-based verification would likely be easily spoofed, either through the Javascript directly or otherwise.
This is the link I found about tracking browser states through Flash.
Tell me if this is too complicated or if there is any logical issues I'm missing with this sort of solution.
posted by disillusioned at 5:04 AM on November 13, 2005