[AnxietyFilter] Am I going to get left behind by technology?
September 30, 2018 10:52 AM   Subscribe

I landed my first job in the tech industry this year, in a data warehouse architect kind of role. My organization is not very modern when it comes to what tools we use, and with everything seemingly moving to the cloud, I'm starting to get anxious that by the time I'm able to move to a more modern company (1 to 2 years), I'm not going to have the skills to get hired. Am I right to be anxious? What should I do here?

At my organization, we use on premise SQL Server, SSIS, and SSRS, nothing cloud based. I've been watching everything on Twitter about MS Ignite, and I've been staying active on LinkedIn trying to stay up to date on the industry, and it seems like everything is going to cloud based, big data, machine learning, etc. And, anecdotally, when I was speaking to a recruiter the other day about the technology we use in my current role, she also mentioned the move to AWS and Amazon Redshift specifically.

I'm worried that I will be left behind by the rapidly changing technology in the data industry, and I am not sure what I should be doing to combat that. Getting my organization to move to the cloud is a non-starter, and I don't necessarily think it's even worth it for our needs. But I want to move to a company that has a more robust, modern data platform, so I feel like I need the experience with cloud based data warehousing, data lakes, etc, to be competitive.

Am I worrying too much? What practical steps can I take to stay competitive in the industry?
posted by motioncityshakespeare to Work & Money (17 answers total) 8 users marked this as a favorite
You’re worrying too much. The adoption rate of new technology happens a lot, A LOT more slowly than the rate at which it is developed. Companies just don’t have the time and money to throw around at radically changing their processes every year or two.

By staying informed on Twitter and LinkedIn, you’re already doing what would be most effective.
posted by Autumnheart at 10:57 AM on September 30, 2018 [17 favorites]

Not all companies are cutting edge. There's a whole spectrum of different methodologies that current companies use for tracking information, stretching all the way back to paper-files-in-filing-cabinets. There will always be legacy systems that need to be maintained, and wages for people who are experts at dealing with those types of systems will only go up as the pool of qualified workers shrinks over time. Meanwhile, much of the stuff that is cutting-edge right now will inevitably turn out to be a fad—meaning that expertise being developed today will turn out to be largely irrelevant in five or ten years.

Long term, I think most people's best bet is to be near the middle of the pack—and that's where I think you are. I'm not in your field, but even I know that SQL is very, very widely used. By all means keep up with current trends—some of that stuff will become mainstream and you'll want to know about it when it filters down to the rank-and-file folks doing the steady, unglamorous, absolutely necessary work that keeps mainstream businesses going. Don't get complacent, do keep developing your skills. But it's OK to make a conscious choice to target your expertise at the kind of mainstream, middle-of-the-pack technologies that serve the bulk of most businesses' needs. Just because a platform came out more than five years ago doesn't mean that there's not huge value in being really, really good at it.
posted by Anticipation Of A New Lover's Arrival, The at 11:05 AM on September 30, 2018 [8 favorites]

In my area, privacy laws prohibit storing certain data outside of Canada. This means that cloud storage is really a non starter. My guess is that there will always be a need for non cloud storage.
If you’re worried, get yourself an AWS account, and play with fake data on there. There are lots of online resources, and AWS is pretty cheap.
posted by Valancy Rachel at 11:10 AM on September 30, 2018 [2 favorites]

This is my field as well and I have found that Anticipation is exactly correct. If you want to work for big, modern companies you'll find all the "big data" trappings but there are loads of smaller, older companies that will use sql server until it drops. I read an article yesterday speculating that by 2025 private businesses will quit having their own datacenters. As long as HIPAA lives that won't be the case in the US. Then there's all the companies that have custom solutions that can't be easily replaced by the cloud.

Don't panic, just sign up for an AWS class online if you can find one and remember, sql is sql. It works everywhere. Heck, Oracle is still widely used as well. If you want to expand your toolkit maybe start there. Good luck, fellow data geek.
posted by irisclara at 11:22 AM on September 30, 2018 [5 favorites]

Response by poster: Lots of good perspectives hear that I hadn't considered. HIPAA and other standards make sense, as does the fact that not all of these technologies will become mainstream. I'll try to keep these in mind and relax a bit. Thank you all for the insight!
posted by motioncityshakespeare at 11:29 AM on September 30, 2018

I think this is natural regardless of what tech vertical you are in. There is simply too much innovation for any one person to know about let alone master.

One thing to keep in mind that while specific tech skills do become obsolete e.g. very few to zero people need Excel 2003 optimization tricks, your soft skills in this space (analyzing a problem domain, querying customers, data modeling, etc.) will not atrophy at nearly the same rate. You are not a valuable resource because you know a hot tech skill that won't be hot in a year but rather because you are a domain expert that knows how to bring a range of tech and processes to solve a specific customer problem.
posted by mmascolino at 11:35 AM on September 30, 2018 [1 favorite]

I'll suggest yes, but not exactly how you're thinking. Technically you should have little problem, the tech gets wrapped up in new (too often obtuse) abstractions that can be a bit of a slog to unwind but it's all code and read/process/write. But those "new" abstraction come laden with buzz words and newer folks that don't get that javascript is not the only language (remember how a year after java's introduction there were job postings requiring 7 years of experience). So keep up with the buzzword bingo, cloud is just someone else's computer (when was the last time you've touched a production box? Everything is cloud:-)
posted by sammyo at 12:00 PM on September 30, 2018 [1 favorite]

When I'm interviewing people, one thing I'm looking for is whether they've demonstrated that they can tackle problems that they've never encountered before. What someone learned in 2003 might not be directly useful now, but the fact that they learned it without anyone available to explain it to them is a big plus to me. The system that someone built at home might not have been useful to any business, but the fact that they built it impresses me.

IOW, it's not about whether you're afraid of falling behind - we all are - but what you're doing about it.
posted by clawsoon at 12:04 PM on September 30, 2018 [3 favorites]

Even -- or especially -- in technology, what has been will be again, what has been done will be done again. I am told that people used to terminal into mainframes, large computation centers that they just used and didn't control. (Sound familiar?)

Storage is storage: the abstractions are the same, regardless of where the data lives. Having data in the cloud might change the practicalities you contend with, but the concepts (schemata, indices) stay the same. Cloud providers know this -- tech writers and UX designers are keenly aware of where their audiences are starting from: if it's not comprehensible, it won't sell.

Put another way: do you have trouble understanding Google Sheets from Excel? Sure, some formula names might have changed; you just look those up. The feature set might be different, which might lead to frustration followed by a workaround, but conceptually, a spreadsheet is a spreadsheet, and whether you can create charts / balance budgets does not depend on whether it's stored in your desktop or in a data center.

As long as you don't demand that the icons stay in the same place on your iPad, I wouldn't worry about your resilience to trends in infrastructure.
posted by batter_my_heart at 1:13 PM on September 30, 2018 [2 favorites]

I don't think there is a reason to panic, and I say this as someone with an anxiety disorder in a similar tech role using things that do not have a long-term future. But I do think that this is a reason why people in tech need to accept that a certain amount of ongoing not-directly-related-to-the-job training is just a part of this career track. I think you're in a better position than I am as far as skills transferring and I'm still not sure this cloud thing is going to be a permanent change in the way some people think it will, but it can't hurt to spend some time keeping familiar with at least the basics of the new things that are getting popular, so that you're in a good position to learn more if it becomes useful.

Like, if you were coming from where I am, it's not necessarily useful for me to become an expert React developer, because I don't use React day-to-day and I only have so much free time, but I am trying to learn enough to talk usefully about the differences between React and what I do now, and to be able to learn more if it becomes a good idea in the coming months/years, but also potentially to learn whatever the hot new thing is after React at that point.
posted by Sequence at 1:48 PM on September 30, 2018 [1 favorite]

I manage a pair of developers to which I'm about to add a third. Most of the team's work has been alongside our data operations team, and they've been pretty cloud-forward: They drew down our old Postgres data service, adopted Kubernetes and Google Big Query, and have done some cool things with Airflow to automate problems like GDPR records removal and other stuff. They did that with a slight headwind from the IT operations side of the organization, where there's some resistance to breaking the "keep it on-prem because it's easier to put it behind a firewall" mentality.

Both are lovely, talented people, but they're also pretty new to the field. One is a self-taught/worked up from the helpdesk kinda person; the other just graduated from a CS program. I've connected them with a mentor in my company's products engineering group to help them get some architectural perspective, and to replace the insights of a senior dev who moved to another role, but neither of them knew this stuff a year ago. Independent research and watching their field—not mastering every single thing in it—allowed them to adopt these technologies in pretty short order.

We're out looking for that third person now, and honestly the experience-with-cloud-tech piece is in our list of "also usefuls." We're confident the existing team can bring someone up on what they've done so far on these platforms, and we're actually more interested in finding someone who's stronger in distributed development practices, and who understands test-driven development.

If we come across someone with strong fundamentals who has a lot of data engineering work under their belt, but has done so on older platforms and wants to try some of the new stuff, I'll be thrilled. We need the fundamentals expressed as good, solid habits, and an ability to learn.
posted by mph at 3:39 PM on September 30, 2018 [1 favorite]

You can always interview for jobs you have no intention of taking, before you leave this one, just to see how your plans are working out. I suppose informational interviews are implicated here, but I think you'd want more rubber-to-road data to base your decisions on.

Then you can find your real next job.
posted by rhizome at 4:13 PM on September 30, 2018

I'm worried that I will be left behind by the rapidly changing technology

Welcome to the tech industry!

I am not sure what I should be doing to combat that.

The key phrase is "buzzword compliant". When the time comes take some online courses so you can add "Certficate Of Fnord" to your resume to get past the auto-filters. Knowledge of Fnord isn’t as good as direct experience with Fnord but it will take you surprisingly far.

And as others have mentioned above, when learning about the Grand New Paradigm that is sweeping the tech industry be very prepared to say "Well crap, that’s just _____ rewrapped with some fancy stickers on it!”
posted by Tell Me No Lies at 4:15 PM on September 30, 2018 [1 favorite]

Oh, and while it’s not something you’d want to plan your career around there are some programmers making very good livings maintaining Cobol and Fortran code. The industry is overdue for a mass shift in querying languages, but SQL will be a major job skill for the rest of our lives.
posted by Tell Me No Lies at 4:28 PM on September 30, 2018

A lot of industries require close control of customer data and aren't using the cloud, or are using the cloud in limited ways. My company keeps no data on outside servers, yet we use Microsoft Azure DevOps (VSTS) for project management... we just don't put customer information there. And we use machine learning as well, though in a limited way as we figure what it can do for us.

In many established companies, new technologies are put into play when and where they makes sense, and they are put in place carefully, so as to enhance, not disrupt, working systems that help us make money day after day.
posted by lhauser at 4:56 PM on September 30, 2018 [1 favorite]

A lot of IT guys I worked with over the years were able to keep up with the new things while working with the old things. I was never quite sure how they did it, but mostly they ran some sort of small network at home, and tried things. OTOH, since I was really more interested in business issues than tech, I worked happily well behind the bleeding edge.

There were two times that I ran into a jump in technology similar to what you are worrying about. These were the move from procedural programming to event-oriented programming, and the second was the advent of web-programming and writing for browsers. In the first case, I told my employers I was going to take a week off (unpaid) to take a training class (which shamed them into paying for it). In the second case, I had a chance to try a simple project, and I decided not to compete in that market. I was just a few years from retirement (I thought) and there would be plenty of legacy work to keep me happy.

There was one other situation that might be instructive. I was looking for work, and I thought that SAS was a good match for my background and experience and could be a good area for me. I took a computer-based certification course and asked around a bit. In the end, I got a different opportunity, but I'm pretty sure it would have worked if I'd needed it to.

So, the bottom line is that the things that look like big tech changes can be handled with a week or two of study, given a decent background. Remember, a new job is not going to require knowing everything new. It's only going to require one new environment, and most of that is going to something you already know, maybe with different labels.
posted by SemiSalt at 8:40 AM on October 1, 2018 [2 favorites]

HIPAA does not require on-prem solutions.
The big cloud provider has specific services that are certifiable.
posted by bastionofsanity at 10:18 AM on October 2, 2018 [1 favorite]

« Older Name our new vehicular addition! Difficulty:...   |   Non steroidal treatments for my dog's panniculitis... Newer »
This thread is closed to new comments.