Can a website be continued by someone else
January 31, 2018 12:03 PM Subscribe
A government operated website from the USDA, Supertracker, is being discontinued. How easy is it to copy that site and continue it? I realize the domain name would change, but that is not what I'm thinking.
USDA Supertracker is used by 27 million people and has been in service as a calorie, exercise and weight tool for many years. On January 31, 2018, the USDA announced it would deactivate Supertracker in Jun 2018 and delete all of the data. There are other tools in the commercial market, but for users who have gotten used to Supertracker and have all of their historical data stored there, it is a huge inconvenience to move their data considering the USDA does not all that data to be easily exported. I want to recreate this taxpayer paid for site so, that for a fee, users can continue to use it as it is.
I've always thought all public available government information is essentially free and can potentially be repurposed by the public. So, can this be done with this website? When I look at the source page, it seems all of the code needed for the site is there. Could a programmer recreate it I'm sure it's not that simple, but I don't think it's impossible. Is there something in the middle?
USDA Supertracker is used by 27 million people and has been in service as a calorie, exercise and weight tool for many years. On January 31, 2018, the USDA announced it would deactivate Supertracker in Jun 2018 and delete all of the data. There are other tools in the commercial market, but for users who have gotten used to Supertracker and have all of their historical data stored there, it is a huge inconvenience to move their data considering the USDA does not all that data to be easily exported. I want to recreate this taxpayer paid for site so, that for a fee, users can continue to use it as it is.
I've always thought all public available government information is essentially free and can potentially be repurposed by the public. So, can this be done with this website? When I look at the source page, it seems all of the code needed for the site is there. Could a programmer recreate it I'm sure it's not that simple, but I don't think it's impossible. Is there something in the middle?
The source code you are viewing in your browser is final *rendered* code. It does not take into account anything that is executing on the server end, javascript that is manipulating the DOM, content stored in a database, etc.
From a purely technical standpoint, if you were to scrape the site and then mirror it, you would not be guaranteed a fully functioning site.
posted by Wossname at 12:16 PM on January 31, 2018 [6 favorites]
From a purely technical standpoint, if you were to scrape the site and then mirror it, you would not be guaranteed a fully functioning site.
posted by Wossname at 12:16 PM on January 31, 2018 [6 favorites]
Right, you would need access to the user data and I don't see how you could get that without database access.
posted by dawkins_7 at 12:19 PM on January 31, 2018 [2 favorites]
posted by dawkins_7 at 12:19 PM on January 31, 2018 [2 favorites]
To continue: An ideal-world scenario would be everything handed to you on a disc. But the current site is written and hosted on Microsoft technologies and could mean it is using proprietary/expensive modules with their own licensing rules.
A good programmer could review the site, come up with functional specs, and then recreate the functionality in their dev platform of choice, drawing from their personal experience and intuition as to how everything was originally coded. But extracting all the various nutritional data would be a chore. I do see that the USDA has an API for their nutritional data here, which would make things easier: https://ndb.nal.usda.gov/ndb/
Also, you'd still never get the user data. For all sorts of privacy reasons.
posted by Wossname at 12:24 PM on January 31, 2018 [2 favorites]
A good programmer could review the site, come up with functional specs, and then recreate the functionality in their dev platform of choice, drawing from their personal experience and intuition as to how everything was originally coded. But extracting all the various nutritional data would be a chore. I do see that the USDA has an API for their nutritional data here, which would make things easier: https://ndb.nal.usda.gov/ndb/
Also, you'd still never get the user data. For all sorts of privacy reasons.
posted by Wossname at 12:24 PM on January 31, 2018 [2 favorites]
As USDA notes on their discontinuation info page, since they launched this a lot of private sector apps that do the same thing, and more, have come along, both free and paid. So even if you could replicate all the functionality, you'd have to charge something, or find some other revenue source, and you'd be up against all those other apps as competitors.
With regard to data exporting, individual users can pull their data from Supertracker, as explained on the FAQ page.
posted by beagle at 12:49 PM on January 31, 2018 [1 favorite]
With regard to data exporting, individual users can pull their data from Supertracker, as explained on the FAQ page.
posted by beagle at 12:49 PM on January 31, 2018 [1 favorite]
I think you are well intentioned but if I were using a government site to track my own personal data and that data could just be handed over/scraped by a private company I would flip out.
posted by warriorqueen at 9:48 PM on January 31, 2018 [1 favorite]
posted by warriorqueen at 9:48 PM on January 31, 2018 [1 favorite]
An issue that hasn't come up yet is that because there are private companies doing this, the government basically can't help you, as it would be disadvantaging the others / favoritism. So it would be as hard for you to make an app as those other companies.
This isn't like the government climate websites, where no one else was doing it and the scientific community rallied to make copies. It sounds like the public data is still available, just not in the format you like.
posted by momus_window at 7:52 AM on February 1, 2018
This isn't like the government climate websites, where no one else was doing it and the scientific community rallied to make copies. It sounds like the public data is still available, just not in the format you like.
posted by momus_window at 7:52 AM on February 1, 2018
Best answer: You could try filing a FOIA request for the documentation and the back-end code that does the actual work, if not all the actual website code, and see what you get back. No clue what the results are likely to be. The personal data won't and shouldn't be available.
posted by snuffleupagus at 1:25 PM on February 1, 2018
posted by snuffleupagus at 1:25 PM on February 1, 2018
An issue that hasn't come up yet is that because there are private companies doing this, the government basically can't help you, as it would be disadvantaging the others / favoritism. So it would be as hard for you to make an app as those other companies.
Something that's in the public domain doesn't uniquely advantage anyone. Presumably those commercial applications were developed and succeeded because they added something to the existing free functionality available through this tracker.
posted by snuffleupagus at 1:26 PM on February 1, 2018
Something that's in the public domain doesn't uniquely advantage anyone. Presumably those commercial applications were developed and succeeded because they added something to the existing free functionality available through this tracker.
posted by snuffleupagus at 1:26 PM on February 1, 2018
This thread is closed to new comments.
posted by praemunire at 12:09 PM on January 31, 2018 [2 favorites]