How should I go about making a difficult career decision?
June 24, 2021 10:16 PM   Subscribe

So I'm in CS. I've been doing software engineering for 3 years now since I graduated. I have a chance to go into data science and switch out my career. Headed to grad school in Canada for that purpose. I want focus on math long term and shift away from software. I want to then move into something like quant finance. However, I also want to open a business one day.

To start software just doesn't do it for me anymore. I don't like crazy release schedules, I don't like how business often means compromising the quality of work. What I hate the most is the code monkey feeling. I really hate that and I feel like I'm just another cog. I like software engineering as a hobby and I'm willing to give it the benefit of the doubt because thus far the companies I've worked at are terrible.

Shifting to another career is all but certain. I'm certain that data science is assured. However, I want to head into finance afterwards and this is definitely going to take more time. It would probably imply further commitment to education, I may need more math courses and it might also mean getting another masters degree, going for CFA or doing actuarial exams. All of those things require time, it might take years. At the very minimum around 4 to 5 years.

I would also like to open a business one day. It's one of my major goals. That is also something that requires a lot of time.

In addition to that I am 27, not that old, but I also sometimes worry about just how old I'll be when all of this is done.

To me the best path would be do all of these things. However, I fear at times that jumping into finance and opening a business will come into conflict and that I have to decide between one or the other.

I'm kind of stuck with this conundrum and I'm not sure what to do or which path I should commit to. I want to do math, but finance isn't needed for that and yet everything about the field excites me like nothing else. I kind of wish I was a few years younger or that I had known this a while ago. If I had known these things back then I would have chosen math or economics instead of CS.

I'm unsure of what I should do.
posted by Tarsonis10 to Work & Money (11 answers total) 3 users marked this as a favorite
 
A lot of what you're saying you don't like about being in software is not actually about software. When you say "I hate crazy release schedules" and "I hate compromising quality for business concerns" I think "this person hasn't worked on a high functioning engineering team in a company that values engineering." Which you pretty much say.

You'll find bad workplaces in any field. Let me just give you a flavor of the analogous complaints I hear from data scientists (who I've hired + managed for 5+ years):
  • People only want to hear from me if I tell them they're right and confirm all their biases.
  • I never have time to do anything technically sophisticated, it's just dashboards and metrics all day.
  • I'm just an English-to-SQL converter.
  • I spend all my time cleaning bad data but no engineering teams will work with me to make the data better.
Not all teams are like that. But lots are, even well-meaning ones.

That's all to say, I wonder if you have a normal level of disgust for bad teams? Especially early in your career when it's harder to tell whether the team you're joining is healthy or not, it's easy to feel like this is all there is.

I see a connection to "I want to have my own business" too. Is that about autonomy and control and "doing it right?" Being the person in control is one way for that to be true. But being in control in my experience is just as much about having to make the compromises yourself. Maybe that feels better than someone else making the compromises for you without you having a voice. But, it's also possible to have good management that is making good compromises without you having to worry about it and you get to focus on doing your job well.

One last side note. This question of education is a red herring. If you are a professional SDE today, you can lateral your way into entry level data jobs trivially. Brush up on your SQL, read up on experimentation basics and refresh your memory on mean/median/mode and some basic descriptive stats. Maybe take a coursera course. And then just apply for jobs and I'm confident you'll get callbacks. Engineering to DS is very common. You might also consider Data Engineering as a bridge, which will leverage your engineering instincts but apply them to data problems.
posted by heresiarch at 10:54 PM on June 24, 2021 [19 favorites]


I really want to double what heresiarch said, because I think they nailed some very important points. I know a ton of people in DS and they have a ton of complaints that would line up with yours. I mean I think it's fine to hop around, try different roles, etc...but I think there's a good chance you'll get into DS and find that the issue isn't the domain, but rather your relationship to your work. Your sense of autonomy, for example. They are also totally right about being able to lateral your way into data jobs. If anything, the main complaint I get from my DS friends with less of an eng background is that all the jobs want people with eng skills...in my experience, companies are much more excited about helping an engineering get a bit into DS than about a DS person who has no eng skills. Of course, this doubles back to the many, many complaints that my DS friends have, namely "they just want me to do data engineering."

Something I can give you some direct advice on though: "Shifting to another career is all but certain. I'm certain that data science is assured. However, I want to head into finance afterwards and this is definitely going to take more time. It would probably imply further commitment to education, I may need more math courses and it might also mean getting another masters degree, going for CFA or doing actuarial exams. All of those things require time, it might take years. At the very minimum around 4 to 5 years."

I think you really need to speak to people in quant finance! I worked in quant finance, and nobody there cared about the CFA, and certainly not the actuarial exams! Definitely do research on quant finance before you commit to further education, because what you find may surprise you. If you're a good enough engineer, it's totally possible to get into that world...it sort of just depends on what trajectory you want, eg at many shops you can go in as an engineer, and then over time get more involved on the quant side. Not all places will be as open to this.

Some (many!) quant finance shops do care about education, but they don't like people who studied "finance" and in my experience, the best quant shops will actively avoid people they feel are "financey." All the shops I know of like people with say...math PhDs, stats PhDs, basically any sort of difficult, heavily quantitative PhD.

Actuarial exams are for becoming an actuary.

As far as a CFA, I know many CFAs but they generally aren't working in quant finance...they are usually working in more sell side research, or big mutual funds, that sort of thing.

Definitely do more research on the sorts of roles you think you'd be happy in, because a lot of the issues you have will crop up everywhere. If anything, like heresiarch said, as you get more experience you will have more leverage around the sorts of teams you join, and you'll also get better at figuring out what sorts of teams you work best on, and how to sus that out during interviews (though this is a difficult thing to do!)

As far as starting your company etc etc, honestly, you have time! I guess unless you're antsy to meet other non-career life goals ("I want 3 kids!"), tech is about as forgiving as it gets when it comes to doing all sorts of things. Large salaries and a huge demand for labor means that there's basically no other field where you can move around so easily. I have a very compelling tech resume and I am on an "extended sabbatical" studying languages...I still get people wanting me to go work for them. You can go do data science, then try and get into quant finance, then start you own shop...you might not do it all as quickly as you like, but you certainly can do all of that if you want!

That said, it's really really important to remember that "wherever you go, there you are." Sometimes you will realize that the issue is yourself, or at least, your relationship to your work. I certainly have had this issue (thus the extended sabbatical :P). I only harp on this because I've found it very important...especially in tech where moving around is so easy and companies are so desperate for good talent, it's really easy to convince ourselves that we are perfect and the companies are all bad. Of course, the hard part is that a a lot of companies are bad!

"I'm kind of stuck with this conundrum and I'm not sure what to do or which path I should commit to. I want to do math, but finance isn't needed for that and yet everything about the field excites me like nothing else. I kind of wish I was a few years younger or that I had known this a while ago. If I had known these things back then I would have chosen math or economics instead of CS."

You're 27. You have your whole life ahead of you! It's silly to think that if you don't nail exactly what you want to do at 18 that you've failed. The best time to start was a decade ago, fine, but the next best time to start is today.

This isn't true for everyone on earth, of course, but you are basically in the most favorable job and earning position on earth. Programmers in north america have the best wages and easiest time finding a job of literally anyone on earth. So I mean...take advantage of your amazing luck! And you know, donate generously and support progressive causes which make it so you don't have to luck into the best job market ever to have this sort of flexibility, but I digress...

I feel like you should go do a masters or PhD in math, and then just sort of...see where you feel after that. The great thing about math is everyone respects it. Quant finance? Yes. DS? Yes. Doesn't mean you won't have to learn other things to bridge into it, but if you know where your passion is and can chase it without making reckless decisions, then it's worth it. So don't go into debt to get a masters in math, but otherwise, it certainly won't hurt you when looking for a job in the future.
posted by wooh at 11:18 PM on June 24, 2021 [9 favorites]


Best answer: When faced with a particularly tough life decision, I recommend doing a Pro/Pro list. It’s essentially where you focus on each option, one at a time, and max out on all the positives, without considering any negatives whatsoever. For each option, ask yourself questions like: if I committed to this 100% and went all in, how far could I go with it? What’s the best impact I could make on others here? What are the benefits for me? How would I feel if I were massively successful with this direction? Where could this all lead?

This is useful because the negatives unfairly sway our thinking with fuzzy, illogical math cancellations that just aren’t true! Plus, you can always come up with a strategy for dealing with or accepting any obstacle (the negatives).

With your two lists of pros, you then ask yourself which reasons do you like best? What’s most exciting? Rewarding? Fulfilling? Aligned with your values and life goals? Makes the biggest impact? Gives you the feelings you enjoy most?

It’s much easier to decide anything from the place of how we want to feel and the vision we imagine for our life.
posted by iamkimiam at 11:29 PM on June 24, 2021 [8 favorites]


Please keep in mind that shifting career may NOT get you away from what you're trying to avoid. They just appear in different forms.

I was in software and IT, and I've taken enough data science courses to roughly know what I'm talking about.

With data science, your job is "to find the story in the data", basically. So you will very likely to be PUSHED by people of various levels to either come to a decision (even though the data is inconclusive) or wanted to influence the story you're trying to extract (in covert and overt ways) because it suits their own agenda(s).

It's really not THAT different than software engineer being pushed by various executives or users to add features, do more with less, accelerate schedule, and so on and so forth, just different forms.

The problem also depends on the size of the company vs the size of the team. In an ideal team, like a SCRUM team of about, say 7 or so people, NO ENDUSER should approach an engineer directly. Instead, all requests should go through the project manager, who will add to the to-do list, prioritize, and work on the "most impactful" stuff first (much as SCRUM dictates), and tasks should be small enough that progress can be made in a day or two, so you have something to report on every day, so the SCRUM master and project manager can reprioritize as needed like task X is going slow, we will put aside task Y and assign that person to help with task X.

It sounds like you're not getting that sort of support, as your manager just emphasizes deadline, deadline, and what happens to you and the department if the deadline is not met. That's more of a sign of an ineffective manager than problems endemic in software engineering.

Note that I am NOT really trying to discourage you from switching fields. You know what you like. Nothing wrong with having TWO degrees or areas of expertise (or more than two!) I am just pointing out that what you're trying to get away from is far more... widespread than you may suspect.

On the other hand, with your software engineering background, maybe you can specialize in the hybrid field: data engineering. It's exactly what it sounds like: data science, put into practice, about how to gather the data cleanly, pipe and transform it efficiently, and store them safely so they can be analyzed and manipulated and insights extracted. So you need to know both.

And that can move you into the field you're aiming for. As for how you start your own biz... Maybe you will come up with a platform of your own, or you can start a consultancy one day that can setup such an solution for a company that needs one ASAP. But that's way off into the future.
posted by kschang at 12:20 AM on June 25, 2021 [1 favorite]


heresiarch is correct and you'll have a better way into becoming a quant by getting a phd in a hard math discipline. If that doesn't seem like something you want to do the only other people I know who became quants lucked into it for lack of a better term.

But what you say about software engineering is what I experience every time I get on a new project with a team that doesn't value me or a technical lead who is way in over their head, which is unfortunately often. In my line of work technical lead is usually the first one who made contact with the client so if you get on a project as an "architect" or "lead" but someone has been talking to the client your SOL, unless they're grossly incompetent they're going to run the show. A lot of people take crazy release cycles and things you describe as signs of a healthy project. It is endemic. Unfortunately your complaints are not just software engineering, I know project managers who became project managers to get out of the software grind only to find a new grind, etc.

I have friends who stuck to the product side, found a good smallish startup where the founder values engineering, but doesn't get involved in the day to day, and they're happy. Not super satisfied but they focus on things outside of work like their family and they're not longer putting in weekends and nights to get things done, things happen on a cadence and they're able to do good work and feel pride about what they've done. Not the feeling of rushing things out the door all the time.

I would stick to it and work on networking to find the niche or company that does it for you. One thing I regret at your age is that I did do the crazy hours and I got stuck in a consulting rut and at a certain point it is hard to get out of that when your resume for 20 years is nothing but consulting. I wish I spent more time networking and sticking to people who I knew were good, seeing where they went, etc.

Owning your own is not a panacea though I encourage you to try. For example, to make rent I'm doing an Exchange to Office 365 migration because someone heard I'm "good with computers" and to be honest the change of pace into something I can't code myself out of is nice, but it is a month to month thing with ironically more pressure and more of a code monkey feel because when you're small unless you have a product, which means you have investors, you're always going from hustle to hustle.
posted by geoff. at 1:03 AM on June 25, 2021


Best answer: To echo the above: the bad experiences you've had are not representative of all software engineering jobs. My experience as a software engineer is that there are some companies that value your contribution as a person and not as a code monkey / faceless cog, release features at a sustainable pace, and are generally pleasant places to work.

As a software engineer, you are currently in high demand and could seek better working conditions. If you want to explore continuing software development at other companies, I found this People First Jobs board to be helpful. You can also try searching for companies that match the values you want to work under via Key Values.
posted by flicken at 3:48 AM on June 25, 2021 [5 favorites]


When you're in grad school, take an ethics class or two.
posted by SaltySalticid at 5:02 AM on June 25, 2021 [2 favorites]


I found the lessons learnt by this blog post interesting and matched patterns I've seen before: https://seldo.com/posts/what-i-ve-learned-about-data-recently.
posted by k3ninho at 5:42 AM on June 25, 2021 [2 favorites]


Response by poster: Will give software the benefit of the doubt. Been in companies with dubious management. That's still not convincing enough for me to say I'm sold on it.

Besides I have a chance fo try out data science and econometrics.

Will talk more to quants. No way I'm giving up on running a business some day. The hustles, investor stuff and compromises aren't really what worry me about that. I know what it entails and I don't really mind.
posted by Tarsonis10 at 10:17 AM on June 25, 2021


I’m an engineer type person in tech with a CS PhD. Total +1 that DS is not ideal either, and I think you’re more likely to see career growth and better compensation by staying in engineering.

Since you are willing to study to switch careers, why not take some time to prepare for interviews at FAANGs where the engineering culture may be a bit better?

Soft -1 on the recommendation to do a math PhD in order to get into quant finance. Believe me, no one gets a math PhD thinking of going into finance. They do it because they are deeply, passionately in love with pure math for its own sake — and then they give up on academia and settle for finance because the math academic market is dead. If you haven’t been spending your free time basking in the marvels of Lie algebras and measure theory, I’m sorry, but you’re going to hate being a math PhD student, even if you are smart enough to do it (which I can say I’m not — and I was a pure math major in college).
posted by redlines at 1:18 AM on June 26, 2021


Response by poster: I don't want to go to FAANG, honestly. And yeah I do like math a lot, but that ship has sailed. Unless, I dedicate myself to being a math major then there's no way I'm getting into a PhD. That's not a time commitment I'm willing to make. Everyone always says this, but I don't really care for it.

I also think you're getting me wrong, I like finance, but I don't like it for its own sake, I like it because of the math problems in it. The relation to real world things makes it all the better.
posted by Tarsonis10 at 8:05 AM on June 26, 2021


« Older Salad Season   |   Statistics Workbook Newer »
This thread is closed to new comments.