Create a simple database website (and gather the data)
February 19, 2008 3:46 PM Subscribe
I'm undertaking a personal web development and database building project, but I don't do web development or database building. Please help me assess the best options to have someone else do the work for me. I think the stuff I want is fairly easy, but please tell me what you think.
I want to do a set of three things (in priority order):
1. Have a simple form where users will input information and that will go into a database. Input screens will be 2-3 pages total, and I want to easily play with the forms to test conversion, throughput, etc. A+ answers will let me easily do split A/B testing on the front end.
2. Have a searchable database of contacts, all with the same set of data. Basically contact information, a few links, and some commentary.
3. Have someone scrape a whole bunch of different websites to assemble the initial contact database. (Perl scripting work here probably?)
4. Have a contact entry form that will insert data into the contact database.
5. All of this should be indexible / crawled by google, and of course have the ability to insert ads.
6. Have a non-ugly design.
7. (Optional, at outset) have user ability to comment and "rate" these contacts.
8. (Optional, at outset) create mashup of contact information and google maps.
How best to build this? Hire a rent-a-coder, college student, etc? (If so, how do I go about finding the right person?) Use an off the shelf product and play with it myself? How much should something like this cost? (Would love to hire a Mefite - so bids accepted on Mefimail!) What decisions should I make now that will give me somthing extensible? What are the main technologies that I should look out for? I'm sure many other questions here that I've not thought of -- so fire away! Thanks in advance hive mind!
I want to do a set of three things (in priority order):
1. Have a simple form where users will input information and that will go into a database. Input screens will be 2-3 pages total, and I want to easily play with the forms to test conversion, throughput, etc. A+ answers will let me easily do split A/B testing on the front end.
2. Have a searchable database of contacts, all with the same set of data. Basically contact information, a few links, and some commentary.
3. Have someone scrape a whole bunch of different websites to assemble the initial contact database. (Perl scripting work here probably?)
4. Have a contact entry form that will insert data into the contact database.
5. All of this should be indexible / crawled by google, and of course have the ability to insert ads.
6. Have a non-ugly design.
7. (Optional, at outset) have user ability to comment and "rate" these contacts.
8. (Optional, at outset) create mashup of contact information and google maps.
How best to build this? Hire a rent-a-coder, college student, etc? (If so, how do I go about finding the right person?) Use an off the shelf product and play with it myself? How much should something like this cost? (Would love to hire a Mefite - so bids accepted on Mefimail!) What decisions should I make now that will give me somthing extensible? What are the main technologies that I should look out for? I'm sure many other questions here that I've not thought of -- so fire away! Thanks in advance hive mind!
Have someone scrape a whole bunch of different websites to assemble the initial contact database.Really? I'd think twice about that; it sounds like you are skating the edge of polite web behavior, and if you're planning on emailing those screen-scraped folks, then it's definitely spamming.
Other than that, you're definitely looking at finding a person to help; I can't think of anything off-the-shelf that does all that. DarlingBri's suggestion is good.
If you have a college nearby that offers web development programs, you might be able to get a student, but working with students can be tricky. You'll need to be much clearer about exactly what you want, and be able to head off extraneous enthusiasms. You might also have to deal with them dropping the project at the end of the quarter/semester. (I worked at a community college for 6 years as web manager and worked with a number of students, had both fantastic experiences and horrid ones.)
posted by epersonae at 9:10 AM on February 20, 2008
I've worked for around 3-4 years doing web development for clients ranging from small businesses to corporations, both on small one-off projects and larger backends.
Drupal may actually be a pretty good fit as an initial framework to build your custom functionality on top of.
You might also look at SugarCRM, depending on how you want to use the data -- if it's for an internal CRM purpose, then a front-end that posts into SugarCRM either through the SOAP interface or directly will work fine for you.
Either one of these options should take an experienced Drupal or SugarCRM coders less than a month, and probably not more than a few weeks, depending on how vague your spec is.
and speaking of specs, this is just a summary of what you want to do, right? You do have a REAL SPECIFICATION lined up for this project, right? If not, start by writing one. WHICHEVER route you go, having a good spec will help to ensure that you are getting what you ask for, and help YOU figure out what YOU really want. A good spec will also bring to light what features are MUST-HAVEs and what features are NICE-TO-HAVE, and which features have dependencies, because I can almost guarantee that your project is going to go over budget or past deadlines and at that point you will be really glad to have a triage plan b. Here is a nice, plain-english tutorial on creating a functional spec.
As for cost, well -- my initial estimate without seeing a full spec would be around a month.* Most freelancers will charge you anywhere from $40-80 an hour; meaning you're looking at a bill from around $6-12k. Add another week or two to pay for a designer (at around the
same rates), and you're looking at $10-15k. Double, triple or quadruple the time for everything if you're trying to make this look really nice or have really smooth UI (ie, if it's the base for a public-facing web 2.0 application looking for investor funding).
If this sounds expensive to you, you may not be in the market to hire developers. Because $10-15k is not really a lot of money for application development.
You CAN get outsourced development for around $15 an hour (well, in 2004 you could -- due to the falling dollar this may be a lot higher these days), which would cut your costs to around $2400 a month, but I also think it would take twice as long due to communication issues, which means you're still looking at $5-6k or so.
In order of preference, I'd go
1. Freelance developer you get via a reference.
2. rent-a-coder-type-thingy
3. college student.
Ideally this would be a nice project for a small development shop, but that would double costs.
The moral of the story is that software development, particularly web development, where the discipline standards are often much lower ("fuck it, updates are free! we can just fix the bugs while the site is being used!") can very easily turn into a super-expensive boondoggle. Do the extra work upfront to make sure your needs are clearly communicated and what is an acceptable final result is a known and measurable quantity. Although I hate advising this, you may want to avoid paying the complete amount before delivery; you're going to need some leverage if the project is not completed to your satisfaction. If you're dealing with a contractor in the US, write a spec and a contract, have the contract looked over by a lawyer, AGREE on the meaning of the spec with the contractor before engaging them (IN WRITING) and force them to stick to it. I've sat in on many a phone call where we argued with contractors about whether Feature X or Feature Y met the spec's definition, and whether we were asking for a bug fix (covered in the contract) or a new feature (extra cash).
* here's how that works: my inner programmer says "I could do that shit in a week", but my inner programmer is ALWAYS wrong and usually doesn't do test-driven development which pretty much guarantees bugs, so I double it. Then, because your spec is very vague, I double it again: voila, one month.
posted by fishfucker at 2:51 PM on February 23, 2008
Drupal may actually be a pretty good fit as an initial framework to build your custom functionality on top of.
You might also look at SugarCRM, depending on how you want to use the data -- if it's for an internal CRM purpose, then a front-end that posts into SugarCRM either through the SOAP interface or directly will work fine for you.
Either one of these options should take an experienced Drupal or SugarCRM coders less than a month, and probably not more than a few weeks, depending on how vague your spec is.
and speaking of specs, this is just a summary of what you want to do, right? You do have a REAL SPECIFICATION lined up for this project, right? If not, start by writing one. WHICHEVER route you go, having a good spec will help to ensure that you are getting what you ask for, and help YOU figure out what YOU really want. A good spec will also bring to light what features are MUST-HAVEs and what features are NICE-TO-HAVE, and which features have dependencies, because I can almost guarantee that your project is going to go over budget or past deadlines and at that point you will be really glad to have a triage plan b. Here is a nice, plain-english tutorial on creating a functional spec.
As for cost, well -- my initial estimate without seeing a full spec would be around a month.* Most freelancers will charge you anywhere from $40-80 an hour; meaning you're looking at a bill from around $6-12k. Add another week or two to pay for a designer (at around the
same rates), and you're looking at $10-15k. Double, triple or quadruple the time for everything if you're trying to make this look really nice or have really smooth UI (ie, if it's the base for a public-facing web 2.0 application looking for investor funding).
If this sounds expensive to you, you may not be in the market to hire developers. Because $10-15k is not really a lot of money for application development.
You CAN get outsourced development for around $15 an hour (well, in 2004 you could -- due to the falling dollar this may be a lot higher these days), which would cut your costs to around $2400 a month, but I also think it would take twice as long due to communication issues, which means you're still looking at $5-6k or so.
In order of preference, I'd go
1. Freelance developer you get via a reference.
2. rent-a-coder-type-thingy
3. college student.
Ideally this would be a nice project for a small development shop, but that would double costs.
The moral of the story is that software development, particularly web development, where the discipline standards are often much lower ("fuck it, updates are free! we can just fix the bugs while the site is being used!") can very easily turn into a super-expensive boondoggle. Do the extra work upfront to make sure your needs are clearly communicated and what is an acceptable final result is a known and measurable quantity. Although I hate advising this, you may want to avoid paying the complete amount before delivery; you're going to need some leverage if the project is not completed to your satisfaction. If you're dealing with a contractor in the US, write a spec and a contract, have the contract looked over by a lawyer, AGREE on the meaning of the spec with the contractor before engaging them (IN WRITING) and force them to stick to it. I've sat in on many a phone call where we argued with contractors about whether Feature X or Feature Y met the spec's definition, and whether we were asking for a bug fix (covered in the contract) or a new feature (extra cash).
* here's how that works: my inner programmer says "I could do that shit in a week", but my inner programmer is ALWAYS wrong and usually doesn't do test-driven development which pretty much guarantees bugs, so I double it. Then, because your spec is very vague, I double it again: voila, one month.
posted by fishfucker at 2:51 PM on February 23, 2008
here's how that works: my inner programmer says "I could do that shit in a week", but my inner programmer is ALWAYS wrong and usually doesn't do test-driven development which pretty much guarantees bugs, so I double it.EVERYONE'S inner programmer is WRONG. Period. Always. That's the most important lesson for every dev to get in their heads, me included. When the exceedingly rare exceptions to this rule are found, they are usually drummed out of projects for being "pessimists." ;-)
+1 to everything ff said in his post; the importance of nailing down the spec can't be overstated. Even though it will probably evolve and shift as development progresses, realizing that you are deviating from that spec -- and adjusting things accordingly -- will make your life so much better it's not even funny.
posted by verb at 5:44 PM on February 24, 2008
This thread is closed to new comments.
You might post your project in Jobs, too.
posted by DarlingBri at 4:18 PM on February 19, 2008