Software Development Proposal and Hiring Process for n00bs
October 5, 2014 4:09 PM Subscribe
If only one of us could program, and our department was able to fully fund a research position, we'd be set. As that is not the case, how can 10-15 busy and financially strapped masters and doctoral students find and fund someone to create a piece of software to create, run, and output data for human learning experiments? Challenge level: no institutional support likely, northeast US.
I am a PhD student member of the first and only lab in the only PhD program at my institution. We are a human learning lab that studies how people learn complex concepts. The lab is so new that we do not yet have projects running other than students' Master's theses and doctoral dissertations. We are need of a piece of software that can be customized for similar but separate projects by individual student experimenters, that presents experimental stimuli and records participant responses for our later analysis. (We currently have a piece of software created pro bono for a graduated student, but we no longer have a relationship with the software creator and the software is no longer functional for our needs.) Currently, approximately 10 graduate students' ability to graduate depends on this software being functional.
What we don't have: Programming skills or experience doing anything like this, time to learn, grants or any probability of obtaining them, likely financial support from our institution for this (on approach, our advisor offered to financially assist via personal funds), extensive personal funds (we are grad students living off of student loans and/or otherwise tight budgets), internal candidates for this type of work at our institution, unlimited time to get this completed (we would need it by next semester, e.g., mid-January 2015), relationships with programmers who could do a pro bono consultation on this project. Hi, hivemind!
What we do have: A clear and detailed vision of exactly what the software needs to do. We need a web-based (?probably - we don't have a physical "lab" space large enough for our needs and run experiments in other rooms at our institution that we reserve) piece of software with individual student experimenter user accounts (so that each experimenter can create their own projects), that can present a large number of sequential experimental trials displaying an array of text/images that an experimental participant clicks on. Which stimuli are clicked on are recorded by the software and then accessed by the experimenter following the conclusion of the experiment. Which set of trials is presented changes dynamically based on participant performance (e.g., a participant who does not select a certain number of "correct" stimuli receives remedial trials while a participant who does advances to the next "set" of trials). Experimenters upload stimuli, set rules about what is "correct" and how participants "advance", as well as some other parameters. We would want ongoing support from whoever develops this as we would want it for long-term use.
Questions:
How do we find someone to help us propose this project using language and specifications appropriate for presentation as a job to a pool of qualified individuals?
How do we access this pool of individuals? Based on other similar questions, I looked on Elance, but would that suit our needs re: ongoing support? What is the general professional designation of the individual we'd need? Developer? Programmer?
If you are able to give more information, based on the limited details provided, approximately what time/cost ranges is reasonable for this type of project? Is it so large that we ought to research additional funding avenues before even approaching someone to do this for us? (We are currently reaching out to other, better-funded labs who do similar work.)
How does billing work for something like this? Is it a flat fee + hourly for follow-up support? Or is it hourly throughout? Or might this vary?
Are there any funding opportunities of which we could avail ourselves? We are all new to just about every aspect of this process.
Thanks in advance for your consideration and expertise.
I am a PhD student member of the first and only lab in the only PhD program at my institution. We are a human learning lab that studies how people learn complex concepts. The lab is so new that we do not yet have projects running other than students' Master's theses and doctoral dissertations. We are need of a piece of software that can be customized for similar but separate projects by individual student experimenters, that presents experimental stimuli and records participant responses for our later analysis. (We currently have a piece of software created pro bono for a graduated student, but we no longer have a relationship with the software creator and the software is no longer functional for our needs.) Currently, approximately 10 graduate students' ability to graduate depends on this software being functional.
What we don't have: Programming skills or experience doing anything like this, time to learn, grants or any probability of obtaining them, likely financial support from our institution for this (on approach, our advisor offered to financially assist via personal funds), extensive personal funds (we are grad students living off of student loans and/or otherwise tight budgets), internal candidates for this type of work at our institution, unlimited time to get this completed (we would need it by next semester, e.g., mid-January 2015), relationships with programmers who could do a pro bono consultation on this project. Hi, hivemind!
What we do have: A clear and detailed vision of exactly what the software needs to do. We need a web-based (?probably - we don't have a physical "lab" space large enough for our needs and run experiments in other rooms at our institution that we reserve) piece of software with individual student experimenter user accounts (so that each experimenter can create their own projects), that can present a large number of sequential experimental trials displaying an array of text/images that an experimental participant clicks on. Which stimuli are clicked on are recorded by the software and then accessed by the experimenter following the conclusion of the experiment. Which set of trials is presented changes dynamically based on participant performance (e.g., a participant who does not select a certain number of "correct" stimuli receives remedial trials while a participant who does advances to the next "set" of trials). Experimenters upload stimuli, set rules about what is "correct" and how participants "advance", as well as some other parameters. We would want ongoing support from whoever develops this as we would want it for long-term use.
Questions:
How do we find someone to help us propose this project using language and specifications appropriate for presentation as a job to a pool of qualified individuals?
How do we access this pool of individuals? Based on other similar questions, I looked on Elance, but would that suit our needs re: ongoing support? What is the general professional designation of the individual we'd need? Developer? Programmer?
If you are able to give more information, based on the limited details provided, approximately what time/cost ranges is reasonable for this type of project? Is it so large that we ought to research additional funding avenues before even approaching someone to do this for us? (We are currently reaching out to other, better-funded labs who do similar work.)
How does billing work for something like this? Is it a flat fee + hourly for follow-up support? Or is it hourly throughout? Or might this vary?
Are there any funding opportunities of which we could avail ourselves? We are all new to just about every aspect of this process.
Thanks in advance for your consideration and expertise.
This doesn't add up. Software that is fundamental to the success of 10 graduate students and indeed your entire lab, and no budget at all? Someone needs to be pushing back, hard, up the ladder. It is possible to shoestring this with a freelancer, but what if they are sloppy about security and you get hacked and all your private research data goes out? What if they screw up the math and accidentally bias your results in a way you don't detect for a few years? Also, ongoing support is not possible on a shoestring.
I would expect to pay a competent freelancer $50 to $100 an hour for both development and support. In terms of the size of the project, I would expect 5 hours of meetings to figure out the requirements, 10 hours of work to prototype the system, and then a large amount of hours (20 or more) to iterate and refine until it's exactly the way you want it. So 40 hours at minimum, which makes this a $2000 minimum project and maybe more like $4000 or $5000, which is not bad if your entire lab is resting on it. Budget $10/month for hosting and additional $100/month for support hours in perpetuity, more if you anticipate extensive customization. Double your costs if you want to go with someone more established and experienced.
My recommendation is that someone in your department learn to program, though I know it seems daunting. As a lab in a field which depends on programming skills, especially to be able to customize experiments, you really need this expertise in house. Plus, developing these skills will help you tremendously in all kinds of future projects and potential jobs. I recognize the limited timeline, and it would be great to be able to offload this, but offloading takes budget; the less you want to have to think about yourself, the more budget.
Look more closely at your institution. Do you have a computer science program? You can find a competent programmer there who will likely charge on the low end of the scale. You can connect to such people through internal university channels. You won't get ongoing support that way, but that's fine if someone in your group can take that over once they learn to program. If you can frame this in your mind as less of "someone is going to solve this problem for me", and more as "someone is going to a build a working system with my input, and I am going to learn as I go so understand how it works and can take over future development and support", you will have way more options in the future, and I would say a much higher chance of a good outcome.
Alternatively, it is possible an off-the-shelf app exists, especially if this is a common problem. This might rescue you. Do your homework and figure out how other researchers in your field solve this problem. Possibly there are open source tools that exist in which case you only need a techie to set them up for you, which is now a quick job. Peripherally, I have heard of something called 'Psychopy" which is an experimental psychology toolkit in Python, which might possibly be what you want. Then you need to learn a bit of Python to figure out how it works and how to use it, but once you do that, you are in control.
posted by PercussivePaul at 4:54 PM on October 5, 2014 [10 favorites]
I would expect to pay a competent freelancer $50 to $100 an hour for both development and support. In terms of the size of the project, I would expect 5 hours of meetings to figure out the requirements, 10 hours of work to prototype the system, and then a large amount of hours (20 or more) to iterate and refine until it's exactly the way you want it. So 40 hours at minimum, which makes this a $2000 minimum project and maybe more like $4000 or $5000, which is not bad if your entire lab is resting on it. Budget $10/month for hosting and additional $100/month for support hours in perpetuity, more if you anticipate extensive customization. Double your costs if you want to go with someone more established and experienced.
My recommendation is that someone in your department learn to program, though I know it seems daunting. As a lab in a field which depends on programming skills, especially to be able to customize experiments, you really need this expertise in house. Plus, developing these skills will help you tremendously in all kinds of future projects and potential jobs. I recognize the limited timeline, and it would be great to be able to offload this, but offloading takes budget; the less you want to have to think about yourself, the more budget.
Look more closely at your institution. Do you have a computer science program? You can find a competent programmer there who will likely charge on the low end of the scale. You can connect to such people through internal university channels. You won't get ongoing support that way, but that's fine if someone in your group can take that over once they learn to program. If you can frame this in your mind as less of "someone is going to solve this problem for me", and more as "someone is going to a build a working system with my input, and I am going to learn as I go so understand how it works and can take over future development and support", you will have way more options in the future, and I would say a much higher chance of a good outcome.
Alternatively, it is possible an off-the-shelf app exists, especially if this is a common problem. This might rescue you. Do your homework and figure out how other researchers in your field solve this problem. Possibly there are open source tools that exist in which case you only need a techie to set them up for you, which is now a quick job. Peripherally, I have heard of something called 'Psychopy" which is an experimental psychology toolkit in Python, which might possibly be what you want. Then you need to learn a bit of Python to figure out how it works and how to use it, but once you do that, you are in control.
posted by PercussivePaul at 4:54 PM on October 5, 2014 [10 favorites]
Yeah, I think your only solution is to find an existing product. Reach out to all your professional networks. Research related email lists are good for this as they help you reach people you don't know personally. Get your advisor to contact all their professional networks too. It's possible that even if there isn't something freely available, some other lab might have something custom built they are willing to share. Or you might find someone in your field who does have programming ability and is willing to work with you on this in exchange for their name on future publications.
posted by lollusc at 5:28 PM on October 5, 2014 [2 favorites]
posted by lollusc at 5:28 PM on October 5, 2014 [2 favorites]
We need a web-based (?probably - we don't have a physical "lab" space large enough for our needs and run experiments in other rooms at our institution that we reserve) piece of software with individual student experimenter user accounts (so that each experimenter can create their own projects), that can present a large number of sequential experimental trials displaying an array of text/images that an experimental participant clicks on. Which stimuli are clicked on are recorded by the software and then accessed by the experimenter following the conclusion of the experiment. Which set of trials is presented changes dynamically based on participant performance (e.g., a participant who does not select a certain number of "correct" stimuli receives remedial trials while a participant who does advances to the next "set" of trials). Experimenters upload stimuli, set rules about what is "correct" and how participants "advance", as well as some other parameters. We would want ongoing support from whoever develops this as we would want it for long-term use.
This sounds to me like you don't need a from-scratch custom piece of software but rather an implementation of an existing software package. If your lab is on a tight budget, you may balk at buying such software as E*Prime or Matlab. Open source alternatives such as PsychoPy exist, and have friendly communities that can help you get started as a user.
A few additional pieces of advice:
- You shouldn't need a web-based tool for this purpose, especially if you are using the same set of computers over and over and the administrators of those rooms allow you to create permanent accounts where you can set up an identical environment and download any software dependencies. A web-based tool isn't a bad approach, but you probably don't need to limit yourself to that.
- I know you said that you don't have any spare time, but I strongly encourage you and your lab mates to prioritize understanding some basics of programming, especially things like version control and unit testing. This knowledge is becoming more and more important in scientific research and will enhance both the quality of your work and the attractiveness of your cv. A great place to start is Software Carpentry, which runs free-to-attend, short (1-2 day) workshops teaching the basics of programming to scientists.
Feel free to memail me. I've created scripts for running psychology experiments in a variety of ways, and am happy to share what I've learned.
posted by galaxy rise at 6:35 PM on October 5, 2014
This sounds to me like you don't need a from-scratch custom piece of software but rather an implementation of an existing software package. If your lab is on a tight budget, you may balk at buying such software as E*Prime or Matlab. Open source alternatives such as PsychoPy exist, and have friendly communities that can help you get started as a user.
A few additional pieces of advice:
- You shouldn't need a web-based tool for this purpose, especially if you are using the same set of computers over and over and the administrators of those rooms allow you to create permanent accounts where you can set up an identical environment and download any software dependencies. A web-based tool isn't a bad approach, but you probably don't need to limit yourself to that.
- I know you said that you don't have any spare time, but I strongly encourage you and your lab mates to prioritize understanding some basics of programming, especially things like version control and unit testing. This knowledge is becoming more and more important in scientific research and will enhance both the quality of your work and the attractiveness of your cv. A great place to start is Software Carpentry, which runs free-to-attend, short (1-2 day) workshops teaching the basics of programming to scientists.
Feel free to memail me. I've created scripts for running psychology experiments in a variety of ways, and am happy to share what I've learned.
posted by galaxy rise at 6:35 PM on October 5, 2014
when i was in grad school a professor i taught with had written a free program that would allow you to script these sort of experiments called BCR. it's not web-based but i wouldn't want to run those sort of experiments outside a controlled lab setting anyway. it looks like it's still available on his site: http://www.unc.edu/~gpitz/bcr/
he had college juniors and seniors writing experiments in it, so hopefully you should be able to figure it out. he was a friendly fellow and might be willing to help with specific questions.
posted by noloveforned at 8:07 PM on October 5, 2014
he had college juniors and seniors writing experiments in it, so hopefully you should be able to figure it out. he was a friendly fellow and might be willing to help with specific questions.
posted by noloveforned at 8:07 PM on October 5, 2014
You say you don't have funds, but what you do have is the power to beef up an undergrad CS major's CV as a published coauthor. You may also be able to help them out by satisfying a degree project requirement.
That said, bringing on an untested undergrad for a mission critical project is risky, so you might want to combine this approach with a few hours from a freelancer.
posted by qxntpqbbbqxl at 8:31 PM on October 5, 2014
That said, bringing on an untested undergrad for a mission critical project is risky, so you might want to combine this approach with a few hours from a freelancer.
posted by qxntpqbbbqxl at 8:31 PM on October 5, 2014
You can do stuff like this in MATLAB.
That said, if your academic program is really this unsupportive of its first round of students, I would consider transferring to another institution. As a grad student, you are poor, vulnerable, and not necessarily responsible for this type of thing. Not to mention, you're paying people money with your loans to help you be successful.
posted by oceanjesse at 6:21 PM on October 6, 2014 [1 favorite]
That said, if your academic program is really this unsupportive of its first round of students, I would consider transferring to another institution. As a grad student, you are poor, vulnerable, and not necessarily responsible for this type of thing. Not to mention, you're paying people money with your loans to help you be successful.
posted by oceanjesse at 6:21 PM on October 6, 2014 [1 favorite]
This thread is closed to new comments.
posted by Potomac Avenue at 4:54 PM on October 5, 2014