Oldish dog seeks new interview questions.
November 24, 2014 2:50 PM Subscribe
Software testing job interviews: help me ask the important questions.
I'll probably be interviewing for some software testing jobs in the near future. The job I have now is the first testing job I ever got... and I've been there for almost a decade.
Obviously some of the questions I will ask will pertain directly to the nature of the work. I know I don't want to be a full-time automator so I'll make sure that I cover that area well. Things like whom I'd report to, what their role is in the organization, seem pretty obvious to me. Agile/waterfall is an important area too, although I have only a cave-shadow idea of what Agile is in practice. So I might miss some important questions there because of lack of domain knowledge.
Let me know your thoughts, even if you're a developer/business analyst/PM/ITwhatnot and not a tester. Agile developers who've worked with testers, I would like very much to hear from you.
I'll probably be interviewing for some software testing jobs in the near future. The job I have now is the first testing job I ever got... and I've been there for almost a decade.
Obviously some of the questions I will ask will pertain directly to the nature of the work. I know I don't want to be a full-time automator so I'll make sure that I cover that area well. Things like whom I'd report to, what their role is in the organization, seem pretty obvious to me. Agile/waterfall is an important area too, although I have only a cave-shadow idea of what Agile is in practice. So I might miss some important questions there because of lack of domain knowledge.
Let me know your thoughts, even if you're a developer/business analyst/PM/ITwhatnot and not a tester. Agile developers who've worked with testers, I would like very much to hear from you.
Best answer: "Agile" can be interpreted many ways; that word alone tells you little. Some process-oriented questions you might ask:
- Walk me through the lifecycle of a bug, and of a feature.
- How long are your sprints? What different activities happen during as the sprint progresses?
- Approximately how many developers are there per tester?
- How are teams structured? Are testers part of the dev/PM team, or do they float among teams?
- What does team communication look like? Does the team do standups? Weekly status meetings? Heavy email communication?
- What kind of communication should I expect with devs, PMs, etc.? Who is responsible for writing and maintaining test plans?
- To what extent will I be working with other testers?
I am a developer who has worked with and works with testers, in environments that have been loosely agile.
posted by orangejenny at 4:24 PM on November 24, 2014 [3 favorites]
- Walk me through the lifecycle of a bug, and of a feature.
- How long are your sprints? What different activities happen during as the sprint progresses?
- Approximately how many developers are there per tester?
- How are teams structured? Are testers part of the dev/PM team, or do they float among teams?
- What does team communication look like? Does the team do standups? Weekly status meetings? Heavy email communication?
- What kind of communication should I expect with devs, PMs, etc.? Who is responsible for writing and maintaining test plans?
- To what extent will I be working with other testers?
I am a developer who has worked with and works with testers, in environments that have been loosely agile.
posted by orangejenny at 4:24 PM on November 24, 2014 [3 favorites]
Building on orangejenny's response. Who is responsible for prioritizing bugs / what is the process for pulling them into the sprint? On more than one occasion where this was poorly defined I have seen QA members feel like their work was not valued and Devs upset that QA was changing the scope of the work in the sprint.
posted by phil at 4:44 PM on November 24, 2014
posted by phil at 4:44 PM on November 24, 2014
Not sure how many people use it, but you might want to become familiar with Kanban Boards.
It turns Agile/scrum it on its head a bit. Instead of organizing around a weekly sprint (agile), it organizes around individual bug/features (kanban) and a visual indication of process flow and status of bugs/features. I've never used it in practice, but I took a course on it and really like the idea of kanban for software development.
That said, like all "systems", I'm sure there times when it doesn't work well...
posted by sarah_pdx at 6:12 PM on November 24, 2014 [1 favorite]
It turns Agile/scrum it on its head a bit. Instead of organizing around a weekly sprint (agile), it organizes around individual bug/features (kanban) and a visual indication of process flow and status of bugs/features. I've never used it in practice, but I took a course on it and really like the idea of kanban for software development.
That said, like all "systems", I'm sure there times when it doesn't work well...
posted by sarah_pdx at 6:12 PM on November 24, 2014 [1 favorite]
Best answer: Testing has changed a lot over the last ten years, and most of that change has been a broadening of the kinds of things that fall under 'tester'. If you are moving to a new position as a tester the main thing I think you should make sure of is that you understand what they think your job would be and you want to do that job. orangejenny has the sort of questions I think you should be focusing on, and I would add:
- Does this position involve frontend testing, backend testing, or both?
- Are there unit tests for the code, and if so, are they owned by test or developers? (Or are there two parallel sets)
- Does this position involve stress/performance testing?
- Does this position involve analyzing and/or reporting on the results of test runs?
- Does this position involve security/penetration testing?
- How many test configurations (browsers and OSes) does a tester typically manage? How much work on mobile devices is there?
- Say some new code is released into production and a bug is found, what's the process to understand and deal with it?
- How much of the work associated with a release is validating that all the old functionality is still correct, and how much checking the new stuff?
Finally, it might not be politic to ask this explicitly, but I wouldn't take a job where you couldn't answer to your satisfaction why they need testers. I have worked a number of places where they didn't really need testers (because the developers did most of their own testing and/or it was the kind of software where releasing with some bugs wasn't that big a deal) and had testers anyway because that's how you design software, and the result was testers were mostly treated as speedbumps and had to beg developers to explain the code to them so they could write tests that weren't needed.
posted by inkyz at 7:57 PM on November 24, 2014
- Does this position involve frontend testing, backend testing, or both?
- Are there unit tests for the code, and if so, are they owned by test or developers? (Or are there two parallel sets)
- Does this position involve stress/performance testing?
- Does this position involve analyzing and/or reporting on the results of test runs?
- Does this position involve security/penetration testing?
- How many test configurations (browsers and OSes) does a tester typically manage? How much work on mobile devices is there?
- Say some new code is released into production and a bug is found, what's the process to understand and deal with it?
- How much of the work associated with a release is validating that all the old functionality is still correct, and how much checking the new stuff?
Finally, it might not be politic to ask this explicitly, but I wouldn't take a job where you couldn't answer to your satisfaction why they need testers. I have worked a number of places where they didn't really need testers (because the developers did most of their own testing and/or it was the kind of software where releasing with some bugs wasn't that big a deal) and had testers anyway because that's how you design software, and the result was testers were mostly treated as speedbumps and had to beg developers to explain the code to them so they could write tests that weren't needed.
posted by inkyz at 7:57 PM on November 24, 2014
Best answer: I really dislike separating workplaces into "agile" and "waterfall" because it implies that "agile development" is a skill that one can acquire, or a series of processes that can be instituted in any organization. I think of agile as more of a particular mindset of an engineering team which can be summarized as "Make changes quickly and avoid stuff that slows us down." Some workplaces try to say they're "agile" because they have some agile consultant or they use the estimation systems or some other such nonsense. None of those things really makes a team "agile". It's really about putting power in the hands of developers and taking away the slow, clunky, intra- and inter-departmental siloing of knowledge that comes from trying too hard to manage developers.
For example, at my current job, I (a developer) have the power to cut and deploy code to production whenever I need to. It is implied that with that power comes responsibility, and that I must test and verify my changes and get someone else to sign off on them before doing so. In theory, this sounds much more risky than at a previous company I worked at where we had a release every two weeks and if you missed the cut off date your code couldn't be deployed until next month. But in practice, it means I can release my changes when they're actually ready. I don't have to force my way through a burdensome QA process and then persuade a designated committer to promote my changes in the SCM system, nor do I have to get a string of people to sign off on deploying code. It means I don't feel pressure to cut corners on testing because it's all on me. The effect has been, for me personally, a greatly enhanced sense of responsibility for the applications I contribute to. I feel more ownership over my work. There are some costs associated with this model too, of course. We've had a few oopsies where some junior-level developer accidentally broke something in the production database, or a deploy went badly and had to be rolled back. But that's the price we pay for being able to move quickly.
In this environment, our testers are more focused on the comprehensive, end-to-end tests that no single developer has time to write. They also keep tabs on what's going into production and verify things in our regular once-a-week deploys (which is how non-critical things get into production).
Some people were anxious about bringing in QA to the team because we didn't want to get slowed down by waiting for somebody to accept a ticket. A lot of people in startups don't like QA. There is a general belief (not an good one, IMO) that test engineers aren't necessary if you know how to write good tests. I think this is just bitterness from working in bad companies where the QA process is very non-collaborative.
Anyway, that might give you some idea of where the smaller companies are coming from. I actually think test engineers are really valuable and I fought to get some in our company. But I think given our speed of development, they have to take more of a comprehensive approach to quality rather than just approve or reject tickets all day. The QA guys are right now building some automation tools to cover some tests that everyone wants to write but are very time consuming and don't fall under the purview of any single development team.
posted by deathpanels at 8:55 PM on November 24, 2014 [1 favorite]
For example, at my current job, I (a developer) have the power to cut and deploy code to production whenever I need to. It is implied that with that power comes responsibility, and that I must test and verify my changes and get someone else to sign off on them before doing so. In theory, this sounds much more risky than at a previous company I worked at where we had a release every two weeks and if you missed the cut off date your code couldn't be deployed until next month. But in practice, it means I can release my changes when they're actually ready. I don't have to force my way through a burdensome QA process and then persuade a designated committer to promote my changes in the SCM system, nor do I have to get a string of people to sign off on deploying code. It means I don't feel pressure to cut corners on testing because it's all on me. The effect has been, for me personally, a greatly enhanced sense of responsibility for the applications I contribute to. I feel more ownership over my work. There are some costs associated with this model too, of course. We've had a few oopsies where some junior-level developer accidentally broke something in the production database, or a deploy went badly and had to be rolled back. But that's the price we pay for being able to move quickly.
In this environment, our testers are more focused on the comprehensive, end-to-end tests that no single developer has time to write. They also keep tabs on what's going into production and verify things in our regular once-a-week deploys (which is how non-critical things get into production).
Some people were anxious about bringing in QA to the team because we didn't want to get slowed down by waiting for somebody to accept a ticket. A lot of people in startups don't like QA. There is a general belief (not an good one, IMO) that test engineers aren't necessary if you know how to write good tests. I think this is just bitterness from working in bad companies where the QA process is very non-collaborative.
Anyway, that might give you some idea of where the smaller companies are coming from. I actually think test engineers are really valuable and I fought to get some in our company. But I think given our speed of development, they have to take more of a comprehensive approach to quality rather than just approve or reject tickets all day. The QA guys are right now building some automation tools to cover some tests that everyone wants to write but are very time consuming and don't fall under the purview of any single development team.
posted by deathpanels at 8:55 PM on November 24, 2014 [1 favorite]
(Sorry, forgot a relevant anecdote.)
Just for some perspective:
- At the big company I worked for, it was controversial when someone suggested making it possible for any developer to promote code to the release. This is just giving developers the power to promote to the upcoming release! No immediate impact on production at all, and everything would still have been approved individually by a QA person using the set of tests developed by the QA department (which nobody else had access to or could run). Still, the notion of letting ordinary developers have that power was deemed unacceptable. (Waterfall)
- At the small company I work for now, I am going to deploy production tomorrow morning with some changes I finished today. There is no QA sign off. The testers are busy working on building a testing strategy for a complete rewrite of the application I work on. It would take too long to get them up to speed on what I'm doing, and I'm already the best person to be testing this change, so I don't bother. I'll also have my team lead take a look at it before going to production and then watch it like a hawk to make sure everything went well. (Agile.)
posted by deathpanels at 9:05 PM on November 24, 2014
Just for some perspective:
- At the big company I worked for, it was controversial when someone suggested making it possible for any developer to promote code to the release. This is just giving developers the power to promote to the upcoming release! No immediate impact on production at all, and everything would still have been approved individually by a QA person using the set of tests developed by the QA department (which nobody else had access to or could run). Still, the notion of letting ordinary developers have that power was deemed unacceptable. (Waterfall)
- At the small company I work for now, I am going to deploy production tomorrow morning with some changes I finished today. There is no QA sign off. The testers are busy working on building a testing strategy for a complete rewrite of the application I work on. It would take too long to get them up to speed on what I'm doing, and I'm already the best person to be testing this change, so I don't bother. I'll also have my team lead take a look at it before going to production and then watch it like a hawk to make sure everything went well. (Agile.)
posted by deathpanels at 9:05 PM on November 24, 2014
Response by poster: slenderloris: At my current job, there are rumblings of "We're practicing Agile" but what constitutes Agile is quite murky. The projects proceed more like miniwaterfalls, I think. I as a tester get involved from the requirements phase on, but I don't look at the code - say it's a Web service - until that service is fully built out and I can bring a WSDL into SoapUI.
And if I write any code to "test" a service, it's done after that service is built - I don't get involved in writing unit tests. We don't use an acceptance testing framework. That is one of the bees in my bonnet because I'm not a good enough programmer (YET) to bring in that practice all on my own (and that is what would have to happen).
No interviews scheduled yet but I'll update if that changes soon.
posted by Sheydem-tants at 1:17 PM on November 25, 2014
And if I write any code to "test" a service, it's done after that service is built - I don't get involved in writing unit tests. We don't use an acceptance testing framework. That is one of the bees in my bonnet because I'm not a good enough programmer (YET) to bring in that practice all on my own (and that is what would have to happen).
No interviews scheduled yet but I'll update if that changes soon.
posted by Sheydem-tants at 1:17 PM on November 25, 2014
This thread is closed to new comments.
Anyway, at my workplace (in my division --- other divisions do it differently), the requirements analysts are also the testers, and they're embedded with the scrum teams. They set out the requirements for every user story toward the beginning of the iterations and then test them at the end. This usually involves a spot-check in the developer's local environment as well as a second, better-documented test in the integrated domain that builds from the repo. They also participate in holistic deployment testing sessions before each release, on every second Monday/Tuesday.
IME, as someone who has worked in waterfall and Agile shops, the Agile ones typically have a better relationship between the testers and the devs. The cycles are shortened and problems are caught earlier, so it's easier for the business/test analyst to say "This isn't the error message that I specified." The dev can go change it in 5 minutes and check it in again, NBD. In waterfall shops (and I've heard horror stories from people who worked in way more siloed orgs than i ever have), that kind of crap can be a change order, new POs, bills that someone has to pay, general bad news. I don't honestly know if Agile is better for the company in the monetary sense, but in my 8 years in IT it definitely has a better track record with regard to workplace harmony and job satisfaction among testers.
YMMV. I am interested to see others' responses. Feel free to mefi mail me if you like.
posted by slenderloris at 3:53 PM on November 24, 2014