I want to program. for science!
June 15, 2010 10:39 AM Subscribe
How can I break in to scientific programming?
When I get to the point where I want to change jobs, I think I'd like to be a programmer for a research lab or something. I don't have to change jobs omg today, but I want to think ahead and know what steps I could take to make it more likely that I could find a programming job for a research lab in a few years.
I only have undergrad degrees. one in computer science, one in psychology. I got them at the same time. Should I give up unless I go to grad school?
Right now I do java programming for backend web services along with framework type of stuff. for fun I like python, but since I don't program constantly in that I am not an expert. I once had a job doing embedded c, but that was a very long time ago (around 6 years ago).
I live in Chicago. are there any places I can look in to that wouldn't required a long commute out of the city? I hate commuting.
I'm interested in all sorts of things--obviously computer science and psychology, but not limited to those fields.
When I get to the point where I want to change jobs, I think I'd like to be a programmer for a research lab or something. I don't have to change jobs omg today, but I want to think ahead and know what steps I could take to make it more likely that I could find a programming job for a research lab in a few years.
I only have undergrad degrees. one in computer science, one in psychology. I got them at the same time. Should I give up unless I go to grad school?
Right now I do java programming for backend web services along with framework type of stuff. for fun I like python, but since I don't program constantly in that I am not an expert. I once had a job doing embedded c, but that was a very long time ago (around 6 years ago).
I live in Chicago. are there any places I can look in to that wouldn't required a long commute out of the city? I hate commuting.
I'm interested in all sorts of things--obviously computer science and psychology, but not limited to those fields.
Oh, and as for places to work near Chicago: Argonne National Lab is hiring programmers.
posted by dorque at 11:22 AM on June 15, 2010
posted by dorque at 11:22 AM on June 15, 2010
Look at the universities, especially those with a strong research focus like UChicago and Northwestern.
A brief search on the UChicago academic jobs search engine under the "Information Technology" category reveals programming jobs in computational biology, astronomy/astrophysics, pediatrics hematology, and more.
posted by marteki at 11:24 AM on June 15, 2010
A brief search on the UChicago academic jobs search engine under the "Information Technology" category reveals programming jobs in computational biology, astronomy/astrophysics, pediatrics hematology, and more.
posted by marteki at 11:24 AM on June 15, 2010
Response by poster: dorque, I know Argonne is hiring, but that is a hell of a commute for me.
posted by bleary at 11:33 AM on June 15, 2010
posted by bleary at 11:33 AM on June 15, 2010
I have a pack of friends in the scientific computing area, but take a grain of salt, we don't talk "business" that much. These are impressions.
First, what do you mean by "scientific computing"? Is it "I want to write programs that solve science problems" or is it "I think being a programmer at a university would be cool"? Are you thinking of hard science (e.g. physics, chemistry, engineering) or soft science (e.g. human factors)?
Step one, what is it you really want to do?
That said...
Without a grad degree, or great contacts and astounding track record of hardcore, in-discipline programming, you'll pretty much be at best a peon in any research lab setting. You'll be competing with unpaid or minimally paid undergraduate and even graduate interns. You'll also be competing with people who are in-field and likely programming in that field for quite a while. You will get little or no respect.
As to language, it is highly dependent on what sort of research is being done. I'd want to be extremely proficient on C/C++ no matter what, because it's pretty much the lingua franca these days. If it's hard science, you want to be at least competent in FORTRAN, because you'll be building on work done in FORTRAN. Decades of work done in FORTRAN, often (but not always) by really, really competent programmers. Most importantly, you'll want to know enough languages that learning a new one, indeed a new paradigm, isn't a show stopper for you. Because if the professor with the grant says "I've decided that functional programming is the way of the future and everything is going to be F#", you don't get to say "that's stupid".
Maybe with a CS+Psych degree and web/java/python experience you could get on with some sort of human factors shop as a flunky, but, really, the skillset isn't particularly unique or interesting. If this is *really* what you want, then I'd suggest you ponder deeply, figure out what "scientific computing" means to you, and start applying to grad school.
YMMV.
posted by kjs3 at 11:44 AM on June 15, 2010
First, what do you mean by "scientific computing"? Is it "I want to write programs that solve science problems" or is it "I think being a programmer at a university would be cool"? Are you thinking of hard science (e.g. physics, chemistry, engineering) or soft science (e.g. human factors)?
Step one, what is it you really want to do?
That said...
Without a grad degree, or great contacts and astounding track record of hardcore, in-discipline programming, you'll pretty much be at best a peon in any research lab setting. You'll be competing with unpaid or minimally paid undergraduate and even graduate interns. You'll also be competing with people who are in-field and likely programming in that field for quite a while. You will get little or no respect.
As to language, it is highly dependent on what sort of research is being done. I'd want to be extremely proficient on C/C++ no matter what, because it's pretty much the lingua franca these days. If it's hard science, you want to be at least competent in FORTRAN, because you'll be building on work done in FORTRAN. Decades of work done in FORTRAN, often (but not always) by really, really competent programmers. Most importantly, you'll want to know enough languages that learning a new one, indeed a new paradigm, isn't a show stopper for you. Because if the professor with the grant says "I've decided that functional programming is the way of the future and everything is going to be F#", you don't get to say "that's stupid".
Maybe with a CS+Psych degree and web/java/python experience you could get on with some sort of human factors shop as a flunky, but, really, the skillset isn't particularly unique or interesting. If this is *really* what you want, then I'd suggest you ponder deeply, figure out what "scientific computing" means to you, and start applying to grad school.
YMMV.
posted by kjs3 at 11:44 AM on June 15, 2010
I'm going to disagree with what kjs3 says about languages. In bioinformatics, there are people doing great work in R, Java, Ruby, Perl, Python, C, and a dozen others.
Try to find the big centers in your area. For example, the Human Genome Sequencing Center here at BCM employs a few dozen programmers, few of whom have advanced degrees. Check out the big universities in the area - who's seriously getting into sequencing? Who's processing huge data sets from observatories or particle accelerators?
Wherever there is big data, there's a need for people to process and massage it.
posted by chrisamiller at 12:07 PM on June 15, 2010 [1 favorite]
Try to find the big centers in your area. For example, the Human Genome Sequencing Center here at BCM employs a few dozen programmers, few of whom have advanced degrees. Check out the big universities in the area - who's seriously getting into sequencing? Who's processing huge data sets from observatories or particle accelerators?
Wherever there is big data, there's a need for people to process and massage it.
posted by chrisamiller at 12:07 PM on June 15, 2010 [1 favorite]
I agree that most of the programming research assistants are underpaid relative to what they would make in the private market. In a lab you are competing with null/small paid undergrads etc, but those people typically produce *terrible* code and don't have the commitment to maintain software over several years. We have several programmers who were not extensively qualified and have been allowed to grow into the position.
I'm one of those terrible code generating people. We have several programmers whose job it is to take my awful (try-catch what?) rapidly prototyped (in R or perl), and customized idea and, with guidance, turn it into something useful for people who aren't me. Seconding that you'll want C/fortran familiarity. Lots of bio people love perl and younger bio people love python.
posted by a robot made out of meat at 12:09 PM on June 15, 2010
I'm one of those terrible code generating people. We have several programmers whose job it is to take my awful (try-catch what?) rapidly prototyped (in R or perl), and customized idea and, with guidance, turn it into something useful for people who aren't me. Seconding that you'll want C/fortran familiarity. Lots of bio people love perl and younger bio people love python.
posted by a robot made out of meat at 12:09 PM on June 15, 2010
@chrisamiller: there are people doing great work in R, Java, Ruby, Perl, Python, C, and a dozen others.
How, precisely, is my point that "learn C/C++, learn FORTRAN in some fields, but be prepared to learn anything" wrong? Or did you see C and FORTRAN and stop reading?
posted by kjs3 at 12:30 PM on June 15, 2010
How, precisely, is my point that "learn C/C++, learn FORTRAN in some fields, but be prepared to learn anything" wrong? Or did you see C and FORTRAN and stop reading?
posted by kjs3 at 12:30 PM on June 15, 2010
kjs3, I was referring to this bit:
I'd want to be extremely proficient on C/C++ no matter what, because it's pretty much the lingua franca these days. If it's hard science, you want to be at least competent in FORTRAN, because you'll be building on work done in FORTRAN.
These two languages are not at all a necessity in my experience, and in fact, I'd discourage people who want to get into my line of work (bioinformatics) from learning them. Your time would be more productively spent learning a scripting language for data-munging, gaining familiarity with the packages and libraries available to you, and brushing up on the biology (or chemistry or physics) that will provide a basic understanding of the problems you're working on.
posted by chrisamiller at 1:29 PM on June 15, 2010
I'd want to be extremely proficient on C/C++ no matter what, because it's pretty much the lingua franca these days. If it's hard science, you want to be at least competent in FORTRAN, because you'll be building on work done in FORTRAN.
These two languages are not at all a necessity in my experience, and in fact, I'd discourage people who want to get into my line of work (bioinformatics) from learning them. Your time would be more productively spent learning a scripting language for data-munging, gaining familiarity with the packages and libraries available to you, and brushing up on the biology (or chemistry or physics) that will provide a basic understanding of the problems you're working on.
posted by chrisamiller at 1:29 PM on June 15, 2010
Best answer: Given your cs and psychology background, have you thought about labs that concentrate in agent-based modeling? I'm thinking of interdisciplinary projects that develop computer models with the goal of understanding how human psychology and decision-making shape the dynamics of various social systems.
I'm sure you could find quite a few labs doing this sort of research either at Northwestern or U of Chicago. For example, check out the list of projects at the Northwestern Institute on Complex Systems:
http://www.northwestern.edu/nico/
posted by JumpW at 1:37 PM on June 15, 2010
I'm sure you could find quite a few labs doing this sort of research either at Northwestern or U of Chicago. For example, check out the list of projects at the Northwestern Institute on Complex Systems:
http://www.northwestern.edu/nico/
posted by JumpW at 1:37 PM on June 15, 2010
RE: C and Fortran, @chrisamiller and @kjs3
It depends whether you want to use R, python/numpy/scipy/sage, or matlab/octave, or if you want to develop new packages for them. A great deal of the hardcore numerics work in any of those is actually done in C or Fortran with thin layers on top of them. You may not need to write a lot of it, but you'll almost certainly want to understand it. On the flipside if your computations/datasets are small, you probably won't have too many worries. Depends on the applications.
Of course, if you get more towards AI applications, a little LISP never hurt. Maybe leveraging you Java experience and working in clojure would be good for this. It would give you some LISP experience. The cool kids doing data analysis in Clojure are using incanter. That would cover a few bases. You are unlikely to find a lot of shops doing work in clojure/incanter, but if you could ingratiate yourself into that community it would go along way to giving you some street cred.
As to making inroads with the career change, I don't have a lot of advice. In my experience though, Phds really value academic backgrounds and coming in with only an undergraduate could hurt you quite a bit. Your best bet would be to become active in one of these communities and prove your worth that way. Easier said than done, admittedly. Try hitting up some local meetups. A cursory look suggests these might be of interest:
Chicago R user group
Chicago clojure
Chicago hadoop user group
posted by yeoldefortran at 2:31 PM on June 15, 2010
It depends whether you want to use R, python/numpy/scipy/sage, or matlab/octave, or if you want to develop new packages for them. A great deal of the hardcore numerics work in any of those is actually done in C or Fortran with thin layers on top of them. You may not need to write a lot of it, but you'll almost certainly want to understand it. On the flipside if your computations/datasets are small, you probably won't have too many worries. Depends on the applications.
Of course, if you get more towards AI applications, a little LISP never hurt. Maybe leveraging you Java experience and working in clojure would be good for this. It would give you some LISP experience. The cool kids doing data analysis in Clojure are using incanter. That would cover a few bases. You are unlikely to find a lot of shops doing work in clojure/incanter, but if you could ingratiate yourself into that community it would go along way to giving you some street cred.
As to making inroads with the career change, I don't have a lot of advice. In my experience though, Phds really value academic backgrounds and coming in with only an undergraduate could hurt you quite a bit. Your best bet would be to become active in one of these communities and prove your worth that way. Easier said than done, admittedly. Try hitting up some local meetups. A cursory look suggests these might be of interest:
Chicago R user group
Chicago clojure
Chicago hadoop user group
posted by yeoldefortran at 2:31 PM on June 15, 2010
Should I give up unless I go to grad school?
You should go to grad school eventually (if only to avoid being shut out of higher paying jobs that insist on that credential), but not necessarily before you look for a scientific computing job. A bachelor's degree in CS can be enough to get you an entry-level position, and you'll get more out of grad school if you have a little experience with the types of problems that the dry academics can be used for. And from the other angle, there are science geniuses out there who can barely write a comprehensible main() and have never heard of revision control; having someone on the team who understands maintainability, much less OOP, can be a godsend.
posted by roystgnr at 3:29 PM on June 15, 2010
You should go to grad school eventually (if only to avoid being shut out of higher paying jobs that insist on that credential), but not necessarily before you look for a scientific computing job. A bachelor's degree in CS can be enough to get you an entry-level position, and you'll get more out of grad school if you have a little experience with the types of problems that the dry academics can be used for. And from the other angle, there are science geniuses out there who can barely write a comprehensible main() and have never heard of revision control; having someone on the team who understands maintainability, much less OOP, can be a godsend.
posted by roystgnr at 3:29 PM on June 15, 2010
@chrisamiller: Well...that's a way to not answer my point, or at best to say "in my field (bioinformatics), C/C++ and FORTRAN are useless and you should learn scripting".
I don't agree that's a good career decision, not the least because I think that grounding in fundamental skills are never wasted, but more importantly, since the OP never specified a field of science, I'm still going to stand on being a good, flexible programmer in languages that are broadly used in many fields, and being prepared to learn the local rules once the OP figured out what he wanted to do.
If you want to get pigeonholed into a data-massager gig in soft science, then by all means ignore how much of science does things.
YMMV.
posted by kjs3 at 6:57 PM on June 15, 2010
I don't agree that's a good career decision, not the least because I think that grounding in fundamental skills are never wasted, but more importantly, since the OP never specified a field of science, I'm still going to stand on being a good, flexible programmer in languages that are broadly used in many fields, and being prepared to learn the local rules once the OP figured out what he wanted to do.
If you want to get pigeonholed into a data-massager gig in soft science, then by all means ignore how much of science does things.
YMMV.
posted by kjs3 at 6:57 PM on June 15, 2010
(Take my advice with a grain of salt: I'm still (sort of) an undergrad at a mid-sized research university, and in a social science-y field, but I've done some research programming work.)
As an undergrad, did you ever do programming work for a research study? That sort of experience would certainly help with your finding a research programming job.
I know that in the context of hiring research programmers for psychology, the labs I've been associated with have always been happy to find someone with academic psychology experience who can actually program. In cog psych, especially in virtual reality labs, there is definitely a lot of Python use. Python, especially in the context of SciPy and NumPy, are very popular in academic environments.
A lot of labs, depending on the discipline, hire programmers who were lab assistants and are now in between their masters and doctoral degrees as programmers, or hire undergraduate co-op students. But as far as anecdata is useful, I know a handful of recent-ish grads who program in academic environments but only have undergrad degrees.
There are less opportunities for scientific programmers in psych than there are in the other sciences (even if you are looking at schools that do a lot of neuroscience work), so I would suggest looking outside of the social sciences. On preview, kjs3 has a good point about what you consider as "scientific programming". With your experience, you're probably going to be more oriented towards soft-ish programming tasks in psych or human factors, often involving data munging or statistical work; this sort of work is almost a disjoint field from what kjs3 was talking about. But, if you want to be a "flunky" as a certain person upthread referred to such programmers as, the option is out there, and there's some chance that you might find it rewarding.
posted by thisjax at 10:12 PM on June 15, 2010
As an undergrad, did you ever do programming work for a research study? That sort of experience would certainly help with your finding a research programming job.
I know that in the context of hiring research programmers for psychology, the labs I've been associated with have always been happy to find someone with academic psychology experience who can actually program. In cog psych, especially in virtual reality labs, there is definitely a lot of Python use. Python, especially in the context of SciPy and NumPy, are very popular in academic environments.
A lot of labs, depending on the discipline, hire programmers who were lab assistants and are now in between their masters and doctoral degrees as programmers, or hire undergraduate co-op students. But as far as anecdata is useful, I know a handful of recent-ish grads who program in academic environments but only have undergrad degrees.
There are less opportunities for scientific programmers in psych than there are in the other sciences (even if you are looking at schools that do a lot of neuroscience work), so I would suggest looking outside of the social sciences. On preview, kjs3 has a good point about what you consider as "scientific programming". With your experience, you're probably going to be more oriented towards soft-ish programming tasks in psych or human factors, often involving data munging or statistical work; this sort of work is almost a disjoint field from what kjs3 was talking about. But, if you want to be a "flunky" as a certain person upthread referred to such programmers as, the option is out there, and there's some chance that you might find it rewarding.
posted by thisjax at 10:12 PM on June 15, 2010
Wow, it's clear that we're looking at this from very different perspectives, kjs3. Let me try to take a step back here and describe what I see as the problem.
There is really something of a spectrum of people who write code in science. On one end are people who I'll refer to as programmers. They spend their time writing highly optimized code, building toolsets, and not really dealing directly with data analysis. On the other end are people who I'll call scientists, and their goal is to gain insight into the systems that they're studying, often using computational tools.
The projects that these two work on are very different, as are the endpoints. The programmer will write some code, clean it up, and the product is a nice usable tool or package. A scientist will use whatever tools are necessary to gather data from a system, analyze it, and come to a conclusion about some underlying properties of that system.
They're both doing programming, but doing so at very different levels of complexity. They're also both doing science, though at very different depths of understanding the underlying system. Keep in mind that it's a continuum, and there are niches to be found all along it.
So we could bicker all day, but I suppose we really need more input from the OP to answer this question well. So OP, what are you interested in? Using computational tools to do scientific research, or doing basic design and development of analysis tools? What fields are you interested in? Particle physics? Molecular dynamics? Genomics? Protein folding? Your answers to these questions will really determine what the most useful skills to pick up will be.
posted by chrisamiller at 10:15 PM on June 15, 2010 [1 favorite]
There is really something of a spectrum of people who write code in science. On one end are people who I'll refer to as programmers. They spend their time writing highly optimized code, building toolsets, and not really dealing directly with data analysis. On the other end are people who I'll call scientists, and their goal is to gain insight into the systems that they're studying, often using computational tools.
The projects that these two work on are very different, as are the endpoints. The programmer will write some code, clean it up, and the product is a nice usable tool or package. A scientist will use whatever tools are necessary to gather data from a system, analyze it, and come to a conclusion about some underlying properties of that system.
They're both doing programming, but doing so at very different levels of complexity. They're also both doing science, though at very different depths of understanding the underlying system. Keep in mind that it's a continuum, and there are niches to be found all along it.
So we could bicker all day, but I suppose we really need more input from the OP to answer this question well. So OP, what are you interested in? Using computational tools to do scientific research, or doing basic design and development of analysis tools? What fields are you interested in? Particle physics? Molecular dynamics? Genomics? Protein folding? Your answers to these questions will really determine what the most useful skills to pick up will be.
posted by chrisamiller at 10:15 PM on June 15, 2010 [1 favorite]
Best answer: Plenty of people in bioinformatics also need competent people who are actual programmers to write and maintain web tools, and your background actually sounds pretty appropriate for that. It could also be a good stepping stone towards other scientific programming opportunities, since you'd be able to pick up some more skills on the job and could get some additional networking opportunities.
Preferred languages do vary by subdiscipline (and even by lab/individual), as other people have mentioned. If you're more interested in stuff like numerical simulation (e.g. aerospace engineering), that's going to be a lot of lower-level languages because of speed - Fortran being one of them, since lots of previous work has been done in Fortran. However, browsing through job openings at e.g. LANL and Boeing I see plenty of interest in C++ and Java as well, either in addition to or instead of Fortran. My impression is that doing numerical methods tend to require more specialized knowledge, and so if this is really what you want to get into you might want to think about an advanced degree in engineering or applied math.
By contrast, if you're more interested in data analysis, there's a bit more diversity in terms of common languages. Off the top of my head, the most common are a) R (to build on very well-established statistical code), b) Perl/Python (for automating dataset processing and building web interfaces -- Ruby is an up-and-comer here, especially for web stuff with Rails), and c) C++/Java (for speed, when you really need it). R and Numeric/Scientific Python both make use of Fortran linear algebra libraries (BLAS/LAPACK), so a little Fortran wouldn't be totally useless here, but I wouldn't make it a top priority. This genre of programming is a little less specialized, so none of the languages listed above would probably be strict prerequisites as long as you have programming experience.
Looking at job postings of both types I see a lot of demand for relational database expertise (e.g. SQL), so that's maybe something to concentrate on picking up.
A final suggestion: I haven't seen anyone mention parallelization yet, but if you end up doing more compute-intensive stuff, you will almost certainly be running it on clusters. I would say that learning about different types of clusters, from fancy stuff like BlueGene to more humble workstation clusters, and how to program appropriately for each type of platform would really help you here. (GPU clusters are also really trendy right now for certain types of problems, for what it's worth.)
posted by en forme de poire at 10:54 PM on June 15, 2010
Preferred languages do vary by subdiscipline (and even by lab/individual), as other people have mentioned. If you're more interested in stuff like numerical simulation (e.g. aerospace engineering), that's going to be a lot of lower-level languages because of speed - Fortran being one of them, since lots of previous work has been done in Fortran. However, browsing through job openings at e.g. LANL and Boeing I see plenty of interest in C++ and Java as well, either in addition to or instead of Fortran. My impression is that doing numerical methods tend to require more specialized knowledge, and so if this is really what you want to get into you might want to think about an advanced degree in engineering or applied math.
By contrast, if you're more interested in data analysis, there's a bit more diversity in terms of common languages. Off the top of my head, the most common are a) R (to build on very well-established statistical code), b) Perl/Python (for automating dataset processing and building web interfaces -- Ruby is an up-and-comer here, especially for web stuff with Rails), and c) C++/Java (for speed, when you really need it). R and Numeric/Scientific Python both make use of Fortran linear algebra libraries (BLAS/LAPACK), so a little Fortran wouldn't be totally useless here, but I wouldn't make it a top priority. This genre of programming is a little less specialized, so none of the languages listed above would probably be strict prerequisites as long as you have programming experience.
Looking at job postings of both types I see a lot of demand for relational database expertise (e.g. SQL), so that's maybe something to concentrate on picking up.
A final suggestion: I haven't seen anyone mention parallelization yet, but if you end up doing more compute-intensive stuff, you will almost certainly be running it on clusters. I would say that learning about different types of clusters, from fancy stuff like BlueGene to more humble workstation clusters, and how to program appropriately for each type of platform would really help you here. (GPU clusters are also really trendy right now for certain types of problems, for what it's worth.)
posted by en forme de poire at 10:54 PM on June 15, 2010
I work as a programmer in a large genetics research lab in the UK. I have a basic computer degree and, when I interviewed for my job, no real knowledge of biology or bioinformatics.
Most of the programming work on site is probably in Perl, with a lot of the heavy work done in C or C++. The next most popular language is Java and others include Python, R, Ruby and shell scripting. There is also a lot of web work that goes on.
I have no academic background and I got my job mainly because I knew C. Many of the scientists can get by in scripting languages but getting good at C/C++ takes more time and effort than they can afford to give. As for the science, I pick that up as I go along.
I think the secret is to go for a job in a big research lab/university/institute, one large enough to employ specialists outside the organisation's field of study.
posted by antiwiggle at 10:58 PM on June 15, 2010 [1 favorite]
Most of the programming work on site is probably in Perl, with a lot of the heavy work done in C or C++. The next most popular language is Java and others include Python, R, Ruby and shell scripting. There is also a lot of web work that goes on.
I have no academic background and I got my job mainly because I knew C. Many of the scientists can get by in scripting languages but getting good at C/C++ takes more time and effort than they can afford to give. As for the science, I pick that up as I go along.
I think the secret is to go for a job in a big research lab/university/institute, one large enough to employ specialists outside the organisation's field of study.
posted by antiwiggle at 10:58 PM on June 15, 2010 [1 favorite]
Best answer: I wrote this long, meandering answer, but after disagreeing ... err ... reading a couple of the responses, it doesn't really matter. Long story short, I am (was?) pretty much you - I started out as a triple major in Computer Science, Psychology and Math, with an emphasis in Computational Mathematics. At some point, I managed to collect enough credits to graduate with my Computer Science degree right around the time of the dot-com boom, bailed out of school. Old school Catholic-guilt set in a few years later, thus marking my return to finish up my Psychology and Math degrees, with an eye on graduate work.
During the course of finishing up my undergraduate work, I took a course with a professor in Psychology, and at some point during the course of the semester, it slipped that I was an engineer, and we started talking and realizing we had a number of similar scientific interests in the domain of semantic analysis. At the end of the semester, he pulls me aside, and mentions this idea he's had for what amounts to quite some time, and point-blank asks me if I'd like to implement it for his lab.
"Yeah."
And so began a 4.5 year crash course into the scientific method. At the time, the initial focus of the lab was a simulator study. I even had the opportunity to train and run participants through it, and truth be told, I had a lot of fond memories of just that experience alone. And I haven't even begun to describe the backend that was written to measure and analyze the results. That in itself was just an amazing feat of engineering, written by one of the coolest applied statisticians I've ever met in my entire life. Practically a mentor, although I'm not sure he ever realized it, who to this day has a huge influence on the way I think as an engineer.
About a 2 years into my time at the lab, the focus of the lab suddenly shifted to the idea the professor and I had originally discussed. It had literally gone from lingering idea to a sustainable experiment within 2 years. And during that time, we encountered a number of setbacks, but we also had our successes, and I'd like to think that we contributed our little bit to science.
But after 4.5 years, I just couldn't help but observe what was going on outside of the science. You'd have someone come up with an idea, and suddenly, weeks or even days later, it'd be deployed to consumers. The development cycles were becoming so rapid, and it reminded me of why I started hacking out code as kid. I knew I always wanted to build things, it was in my blood. But the cost-to-joy ratio of code is ridiculously low in comparison to say, most traditional forms of "hardware" engineering. I mean, really, the only true cost besides the bootstrapping cost of the hardware is, well, time. The notion I could have the implementation of an idea deployed to another user within hours, that notion all of sudden became exciting again.
So, I left. And I mean, I "left." I dropped out of graduate school, I turned down a maybe more prestigious position with my professor's mentor at another university. And I think maybe some people were disappointed. But I look back on that whole experience with nothing but fond memories, and when I say it shaped the way I think? I mean it.
I mean, I currently work in a XP development shop, and one of things I like about it so much is the whole idea of pairing. That alone provides so much value, it's ridiculous. And that collaborative nature, with open channels communication - that's so much more natural in the sciences. I remember ten years ago, corporate-types were huge on this whole convoluted idea of knowledge capture and dissemination. And the answer is so simple - communicate, guys. I also recently had a chance to go through some strengths training, and discovered that I happen to be a "learner," and also happen to be "deliberative." And I couldn't agree more with that assessment. The ability to acquire material independently, and to also be agressive about verification and validation - these are essential traits in science. And I learned very quickly that these are things I needed to be vigilant about.
Why am I meandering again? I'm trying to get you psyched up for what I think is the real scoop, at least in Academia. 'Cause really, I think you'd be selling yourself short if you *weren't* pursuing a graduate degree in the field that you were working in.
At many universities, it's extremely difficult to hire an employee - any employee. Typically, universities have policies regarding the way the hiring process works. In order for a lab to hire a programmer as a *university employee*, they'd normally have to place a request through HR, who would in turn filter candidates on the basis of criteria within the request over a minimum number of weeks, or worse, months. Then, HR would require that the lab interviews a minimum number of applicants, and would also have to verify that the applicant who was hired was the best suited for the job.
Have you ever seen a university lab job posting that seems so specific, that you can't imagine that more than one or two people can fit that role? It's probably because their are only one or two people who can fit that role, it just so happens that the lab specified such tight criteria that they could hire the person they originally had in mind, while circumventing the HR politics. Most labs choose to go the other route, and hire a student as a research assistant or workstudy. This tends to be a much simpler process, and an actual "hire" can be done within days. And realistically, I think they prefer this, because the "hire" has the potential to have more interest invested in what they do. And this is where I think you can really shine.
Take Psychology, for instance. There's probably a fair number of Psychology students who can "kind of" program, at least enough get by on simple problems. Now, if a lab decides to look outside the field for better programming talent, say from the Computer Science department, they'll probably find it. However, that student won't necessarily have as much invested in the line of work as someone from within the field. And they might "kind of" know something about Psychology. Almost a catch-22. Almost.
Now, if you had someone who studied Computer Science as a undergrad, and decided to jump ship to another field for grad school, you've suddenly got the best of both situations. You have the investment, and you have the programming experience. And in spite of what the others may have said, I think you'd be genuinely surprised at how many of these faculty members would be psyched to have someone who can *program* on their team. I saw that same enthusiasm all the time in *both* Psychology *and* Math.
Your other option is look for an opportunity at a *major* laboratory, i.e., a national one. Often times, they'll *pay* for you to go to grad school, in fact they want you to go.
I really think you have a lot of options to work with, but I say go for it only *as long* as you have some idea of what kind of work you want to pursue. "Cause it really doesn't make any sense if you don't.
posted by aflores at 12:30 AM on June 16, 2010
During the course of finishing up my undergraduate work, I took a course with a professor in Psychology, and at some point during the course of the semester, it slipped that I was an engineer, and we started talking and realizing we had a number of similar scientific interests in the domain of semantic analysis. At the end of the semester, he pulls me aside, and mentions this idea he's had for what amounts to quite some time, and point-blank asks me if I'd like to implement it for his lab.
"Yeah."
And so began a 4.5 year crash course into the scientific method. At the time, the initial focus of the lab was a simulator study. I even had the opportunity to train and run participants through it, and truth be told, I had a lot of fond memories of just that experience alone. And I haven't even begun to describe the backend that was written to measure and analyze the results. That in itself was just an amazing feat of engineering, written by one of the coolest applied statisticians I've ever met in my entire life. Practically a mentor, although I'm not sure he ever realized it, who to this day has a huge influence on the way I think as an engineer.
About a 2 years into my time at the lab, the focus of the lab suddenly shifted to the idea the professor and I had originally discussed. It had literally gone from lingering idea to a sustainable experiment within 2 years. And during that time, we encountered a number of setbacks, but we also had our successes, and I'd like to think that we contributed our little bit to science.
But after 4.5 years, I just couldn't help but observe what was going on outside of the science. You'd have someone come up with an idea, and suddenly, weeks or even days later, it'd be deployed to consumers. The development cycles were becoming so rapid, and it reminded me of why I started hacking out code as kid. I knew I always wanted to build things, it was in my blood. But the cost-to-joy ratio of code is ridiculously low in comparison to say, most traditional forms of "hardware" engineering. I mean, really, the only true cost besides the bootstrapping cost of the hardware is, well, time. The notion I could have the implementation of an idea deployed to another user within hours, that notion all of sudden became exciting again.
So, I left. And I mean, I "left." I dropped out of graduate school, I turned down a maybe more prestigious position with my professor's mentor at another university. And I think maybe some people were disappointed. But I look back on that whole experience with nothing but fond memories, and when I say it shaped the way I think? I mean it.
I mean, I currently work in a XP development shop, and one of things I like about it so much is the whole idea of pairing. That alone provides so much value, it's ridiculous. And that collaborative nature, with open channels communication - that's so much more natural in the sciences. I remember ten years ago, corporate-types were huge on this whole convoluted idea of knowledge capture and dissemination. And the answer is so simple - communicate, guys. I also recently had a chance to go through some strengths training, and discovered that I happen to be a "learner," and also happen to be "deliberative." And I couldn't agree more with that assessment. The ability to acquire material independently, and to also be agressive about verification and validation - these are essential traits in science. And I learned very quickly that these are things I needed to be vigilant about.
Why am I meandering again? I'm trying to get you psyched up for what I think is the real scoop, at least in Academia. 'Cause really, I think you'd be selling yourself short if you *weren't* pursuing a graduate degree in the field that you were working in.
At many universities, it's extremely difficult to hire an employee - any employee. Typically, universities have policies regarding the way the hiring process works. In order for a lab to hire a programmer as a *university employee*, they'd normally have to place a request through HR, who would in turn filter candidates on the basis of criteria within the request over a minimum number of weeks, or worse, months. Then, HR would require that the lab interviews a minimum number of applicants, and would also have to verify that the applicant who was hired was the best suited for the job.
Have you ever seen a university lab job posting that seems so specific, that you can't imagine that more than one or two people can fit that role? It's probably because their are only one or two people who can fit that role, it just so happens that the lab specified such tight criteria that they could hire the person they originally had in mind, while circumventing the HR politics. Most labs choose to go the other route, and hire a student as a research assistant or workstudy. This tends to be a much simpler process, and an actual "hire" can be done within days. And realistically, I think they prefer this, because the "hire" has the potential to have more interest invested in what they do. And this is where I think you can really shine.
Take Psychology, for instance. There's probably a fair number of Psychology students who can "kind of" program, at least enough get by on simple problems. Now, if a lab decides to look outside the field for better programming talent, say from the Computer Science department, they'll probably find it. However, that student won't necessarily have as much invested in the line of work as someone from within the field. And they might "kind of" know something about Psychology. Almost a catch-22. Almost.
Now, if you had someone who studied Computer Science as a undergrad, and decided to jump ship to another field for grad school, you've suddenly got the best of both situations. You have the investment, and you have the programming experience. And in spite of what the others may have said, I think you'd be genuinely surprised at how many of these faculty members would be psyched to have someone who can *program* on their team. I saw that same enthusiasm all the time in *both* Psychology *and* Math.
Your other option is look for an opportunity at a *major* laboratory, i.e., a national one. Often times, they'll *pay* for you to go to grad school, in fact they want you to go.
I really think you have a lot of options to work with, but I say go for it only *as long* as you have some idea of what kind of work you want to pursue. "Cause it really doesn't make any sense if you don't.
posted by aflores at 12:30 AM on June 16, 2010
Response by poster: There's been a lot of activity on this since yesterday, and I didn't mean to leave questions to me unanswered. I've just been busy. I'll try to answer a few now.
kjs3, You make some presuppositions, and I don't know where to start with them. For example, unless I've read you wrong, you presupposed that I would not be willing to learn different languages. This is not the case. I should be able to pick up a language and also work in a different problem domain. I've done it before.
Btw, people in non-academic settings don't necessarily get to tell their bosses that the technical direction they are moving in is unwise.
posted by bleary at 7:34 AM on June 16, 2010
kjs3, You make some presuppositions, and I don't know where to start with them. For example, unless I've read you wrong, you presupposed that I would not be willing to learn different languages. This is not the case. I should be able to pick up a language and also work in a different problem domain. I've done it before.
Btw, people in non-academic settings don't necessarily get to tell their bosses that the technical direction they are moving in is unwise.
posted by bleary at 7:34 AM on June 16, 2010
Response by poster: There've been lots of questions on what exactly I want to do or what I'm interested in. I left my question vague since I didn't want to limit the responses, but, I've also had problems figuring out what I want to do since I tend to get interested in most anything. Physics and cognition (animal and humans) do seem to be big themes. At my age (30s), I doubt I should go for any physics stuff.
tl;dr section
I've been out of school for 11 years, but I will recap that for you: I did some physics while also being interested in all sorts of things. While studying physics I asked good questions for a newbie and got a referral to hang out in a lab that was doing numerical general relativity stuff. It turns out I also have bipolar tendencies and that intervened and due to an interesting sequence of events I ended up transferring to a completely different school in another state, married to someone I met in the physics department. (all the while not realizing I had bipolar tendencies)
At that school, I switched to computer science, since it seemed like a good way to be involved in any number of disciplines. Doing that on its own seemed limiting, so I dithered for a while trying to decide what other discipline to add (philosophy? economics? psychology?) and settled on psychology with a focus on cognitive psychology. Since I also tended to ask good questions I got invited to hang out in someone's cognition lab. Where I helped proctor memory experiments and also wrote a little c to model some stuff to see if it matched experimental results.
Also, due to moving cross country I had out of state tuition and was working part time doing sysadmin flunky type of stuff to help pay the bills. I ended up making more money doing that than I would have taking the job in someone's CS lab that I got invited to do. what a stupid mistake. false economy.
Also, due to the unrecognized bipolar tendencies my attention in class would go all over the map. e.g. I loved the topic of computational theory but almost flunked it because I didn't stay focused and do the work.
By the time I graduated, I thought perhaps I should apply to grad school, but I could tell I had some sort of unresolved problems so I decided not to do that because I'd be wasting my time going to school with problems like that.
I got a job doing programming (got it due to an hr person searching for tcl/tk in a resume) and ended up doing embedded stuff. which was weird because I never focused on the more practical aspects of the CS degree. That place had a lot of bureaucracy and HR foo, so I left to join a startup where they were doing stuff on embedded devices whilst talking to them with a tcl/tk based app, and I thought I'd get to play around in the whole system, but it didn't turn out the way I expected and the place had a melt down.
One of my friends said, too bad I didn't know java, because his friend was looking for java developers. I told him that I could write in any turing complete language, so whatever. He introduced me to his friend and they hired me even though I didn't know java. I thought the place might be boring compared to working on crypto systems (that was my first job) but it was a little less boring than I expected since what I work on is a bunch of interconnected services in a big system. so that's held my interest for a bit.
There is a machine learning group here, but they are small, and I've never actually asked to join them. Not sure how easy it would be to get in.
posted by bleary at 7:55 AM on June 16, 2010
tl;dr section
I've been out of school for 11 years, but I will recap that for you: I did some physics while also being interested in all sorts of things. While studying physics I asked good questions for a newbie and got a referral to hang out in a lab that was doing numerical general relativity stuff. It turns out I also have bipolar tendencies and that intervened and due to an interesting sequence of events I ended up transferring to a completely different school in another state, married to someone I met in the physics department. (all the while not realizing I had bipolar tendencies)
At that school, I switched to computer science, since it seemed like a good way to be involved in any number of disciplines. Doing that on its own seemed limiting, so I dithered for a while trying to decide what other discipline to add (philosophy? economics? psychology?) and settled on psychology with a focus on cognitive psychology. Since I also tended to ask good questions I got invited to hang out in someone's cognition lab. Where I helped proctor memory experiments and also wrote a little c to model some stuff to see if it matched experimental results.
Also, due to moving cross country I had out of state tuition and was working part time doing sysadmin flunky type of stuff to help pay the bills. I ended up making more money doing that than I would have taking the job in someone's CS lab that I got invited to do. what a stupid mistake. false economy.
Also, due to the unrecognized bipolar tendencies my attention in class would go all over the map. e.g. I loved the topic of computational theory but almost flunked it because I didn't stay focused and do the work.
By the time I graduated, I thought perhaps I should apply to grad school, but I could tell I had some sort of unresolved problems so I decided not to do that because I'd be wasting my time going to school with problems like that.
I got a job doing programming (got it due to an hr person searching for tcl/tk in a resume) and ended up doing embedded stuff. which was weird because I never focused on the more practical aspects of the CS degree. That place had a lot of bureaucracy and HR foo, so I left to join a startup where they were doing stuff on embedded devices whilst talking to them with a tcl/tk based app, and I thought I'd get to play around in the whole system, but it didn't turn out the way I expected and the place had a melt down.
One of my friends said, too bad I didn't know java, because his friend was looking for java developers. I told him that I could write in any turing complete language, so whatever. He introduced me to his friend and they hired me even though I didn't know java. I thought the place might be boring compared to working on crypto systems (that was my first job) but it was a little less boring than I expected since what I work on is a bunch of interconnected services in a big system. so that's held my interest for a bit.
There is a machine learning group here, but they are small, and I've never actually asked to join them. Not sure how easy it would be to get in.
posted by bleary at 7:55 AM on June 16, 2010
Response by poster: With respect to animal cognition, I think crows are pretty damn interesting and even was considering going on a vacation where I got to visit one of the labs where they study New Caledonian crows. Whoever I emailed at Oxford didn't want to allow tours of the place so as not to disturb the work. Whoever I ended up emailing in New Zealand, when I noticed that their university website allowed students to sign up to observe the birds in the wild, seemed to not mind the idea of allowing me to vacation over there and visit the blinds and offer to observe. I offered to provide references and so on. It might be a fun vacation. but a bit daunting in terms of finances to be able to travel there.
I like the idea of thinking about cognition, and what exactly constitutes intelligence, regardless of hardware platform (as an analogy). Sort of like asking what information is, if that analogy gives you a way of understanding what I mean by wondering what intelligence is.
But, with thinking about things in an abstract manner, I like to be able to eventually come down and observe things in a natural setting.
People tend to be small details or big picture, right? This is not true for me. I seem to flip back and forth.
posted by bleary at 8:01 AM on June 16, 2010
I like the idea of thinking about cognition, and what exactly constitutes intelligence, regardless of hardware platform (as an analogy). Sort of like asking what information is, if that analogy gives you a way of understanding what I mean by wondering what intelligence is.
But, with thinking about things in an abstract manner, I like to be able to eventually come down and observe things in a natural setting.
People tend to be small details or big picture, right? This is not true for me. I seem to flip back and forth.
posted by bleary at 8:01 AM on June 16, 2010
Response by poster: Anyway, I'm not sure I have the right temperament to focus on research, so perhaps just working with people who do would be good.
I've been pondering what exactly I enjoy in life, and one of them just seems to be that I enjoy thinking about stuff.
posted by bleary at 8:03 AM on June 16, 2010
I've been pondering what exactly I enjoy in life, and one of them just seems to be that I enjoy thinking about stuff.
posted by bleary at 8:03 AM on June 16, 2010
Response by poster: aflores, I like the ability to be able to do small iterations of coding work to just get things done, and I'm not sure there would be a lot of that in some academic settings. thanks for the meandering post.
posted by bleary at 8:06 AM on June 16, 2010
posted by bleary at 8:06 AM on June 16, 2010
Response by poster: Oh, and it frightens me that when I get old my brain will decay and I won't be able to function as well as I do now, so I was thinking about channeling that in to something productive -- in terms of human factors, I guess that would be developing systems that could work as cognitive prosthesis to mediate one's interaction with the world, with caregivers, etc.
I can see stuff like that happening (I like following science and technology news) already. Not sure how to break in to something like that.
HENCE my question. I don't want to do something immediately. I'd like to plan out for a few years to get me going in a good direction.
posted by bleary at 8:10 AM on June 16, 2010
I can see stuff like that happening (I like following science and technology news) already. Not sure how to break in to something like that.
HENCE my question. I don't want to do something immediately. I'd like to plan out for a few years to get me going in a good direction.
posted by bleary at 8:10 AM on June 16, 2010
Response by poster: yeoldefortran, I went to the inaugural meeting of the hadoop user group.
posted by bleary at 8:12 AM on June 16, 2010
posted by bleary at 8:12 AM on June 16, 2010
@Bleary: You make some presuppositions, and I don't know where to start with them. For example, unless I've read you wrong, you presupposed that I would not be willing to learn different languages. This is not the case. I should be able to pick up a language and also work in a different problem domain. I've done it before.
I presuppose because you provided minimal detail. That said, you read me wrong. I did not say you "would not willing to learn different languages", I said you needed to be willing and equipped to learn different languages. Equipped being to important part.
posted by kjs3 at 8:09 PM on June 16, 2010
I presuppose because you provided minimal detail. That said, you read me wrong. I did not say you "would not willing to learn different languages", I said you needed to be willing and equipped to learn different languages. Equipped being to important part.
posted by kjs3 at 8:09 PM on June 16, 2010
Response by poster: kjs3, there are free fortran compilers. It would be easy to be equiped.
posted by bleary at 7:51 AM on June 17, 2010
posted by bleary at 7:51 AM on June 17, 2010
Anecdotes: I have two friends who are programmers in research labs. One, who graduated 15 years earlier with a BS in chem, had no scientific programming experience before getting his first such job, the other's scientific programming experience dated back 30 years to his grad school days. Both though felt that their past science education, however remote, give them an edge in getting hired over others with similar programming experience.
I'm not actually sure what one of them does, but the other works in a genomics lab. A lot of what he does is data wrangling, management and munging. In terms of lines of code, I think the overwhelming bulk of his work is one off bash + sed/awk. Occasionally he'll bust out Python. At one point he was contemplating some Java. I don't think he's ever been concerned about his lack of C/C++ fluency. I think he's always been a pretty work-a-day developer, so the fact that the work isn't challenging from an engineering point of view isn't an issue for him, because the science is interesting.
Oh, I know one other scientific programmer. He's a PhD in math. He used to work for Cray, where his job was basically optimizing the inner loops of customer's scientific codes to run on Cray's hardware. These days the minuta he deals with seems to be administrative, he's managing compliance issues around some code his employer, a defense and civillian aerospace company, has licensed from a large government agency.
posted by Good Brain at 10:42 AM on June 17, 2010
I'm not actually sure what one of them does, but the other works in a genomics lab. A lot of what he does is data wrangling, management and munging. In terms of lines of code, I think the overwhelming bulk of his work is one off bash + sed/awk. Occasionally he'll bust out Python. At one point he was contemplating some Java. I don't think he's ever been concerned about his lack of C/C++ fluency. I think he's always been a pretty work-a-day developer, so the fact that the work isn't challenging from an engineering point of view isn't an issue for him, because the science is interesting.
Oh, I know one other scientific programmer. He's a PhD in math. He used to work for Cray, where his job was basically optimizing the inner loops of customer's scientific codes to run on Cray's hardware. These days the minuta he deals with seems to be administrative, he's managing compliance issues around some code his employer, a defense and civillian aerospace company, has licensed from a large government agency.
posted by Good Brain at 10:42 AM on June 17, 2010
« Older Where can I find plastic inserts for T-shirts? | Why won't my Mac recognize my external hard drive?... Newer »
This thread is closed to new comments.
A friend of mine was pure CS in undergrad and now works for a bio lab programming things for them. He knew basically no bio before he went in -- he's been picking up the science as he goes along. I don't think he jumped through any special hoops to get the job, just replied to their posting looking for a programmer.
I work in a large, multifaceted research facility and part of my job is scientific programming; my training was in math and I had no specific background other than one intro scientific computing course and lots of experience in Matlab. (I do happen to also be a grad student, but my existing programming skills were not at issue when I signed on to my research project -- just my ability to pick up new languages and models quickly.) Most people here use python because the numpy and scipy packages are so thorough. Other coding in the lab is done in C/C++, FORTRAN where necessary (e.g. supporting legacy code), or Matlab if we haven't converted them yet.
posted by dorque at 11:20 AM on June 15, 2010