What's New In Software Development?
October 25, 2015 7:02 AM Subscribe
Almost a decade ago I read (and believed in) Cockburn's Cooperative Game. What's new since then?
I'm particularly looking for one or two key books from the last decade that might change how I think. But I'm also curious for ideas in general.
As far as I can tell, not that much has changed. Agile splintered somewhat, and became more commercial, but is still the main game in town. Testing and continuous deployment have become more popular, but aren't particularly new.
New things I'm aware of include: DevOps (VMs, Docker, Puppet etal); big data; mobile; thicker web clients (Ajax); NoSQL. But they seem peripheral to development. Have I missed something? Are there any really good books on those I should read?
I'm particularly looking for one or two key books from the last decade that might change how I think. But I'm also curious for ideas in general.
As far as I can tell, not that much has changed. Agile splintered somewhat, and became more commercial, but is still the main game in town. Testing and continuous deployment have become more popular, but aren't particularly new.
New things I'm aware of include: DevOps (VMs, Docker, Puppet etal); big data; mobile; thicker web clients (Ajax); NoSQL. But they seem peripheral to development. Have I missed something? Are there any really good books on those I should read?
The rise of GitHub is something to take into consideration as well. There of course were public facing, open source revision control systems antecedents but it feels like to me that GitHub has taken this to another order of magntiude
posted by mmascolino at 8:49 AM on October 25, 2015 [1 favorite]
posted by mmascolino at 8:49 AM on October 25, 2015 [1 favorite]
Yeah, you're trying to separate out technologies from development processes, and the problem is that we really don't have good ways to measure outcomes from those development processes. So every few years we come up with agile, or pair programming, or open environments, and come up with some grand and glorious stories we tell ourselves about why these are awesome, and then we look at output and go back to Mythical Man Month....
(I just went looking for data to support Agile development, and found a bunch of writing saying "well, the books were just published in 2002, it's still too soon to tell", and a bunch of more recent skepticism.)
We do have data suggesting that Test Driven Development increases output.
And I'd reiterate tempythethird's suggestion to immerse yourself in functional programming, at least enough to absorb how to move some of those lessons into your language of choice.
posted by straw at 9:05 AM on October 25, 2015
(I just went looking for data to support Agile development, and found a bunch of writing saying "well, the books were just published in 2002, it's still too soon to tell", and a bunch of more recent skepticism.)
We do have data suggesting that Test Driven Development increases output.
And I'd reiterate tempythethird's suggestion to immerse yourself in functional programming, at least enough to absorb how to move some of those lessons into your language of choice.
posted by straw at 9:05 AM on October 25, 2015
Response by poster: thanks for the answers so far. i'm aware of fp (have used haskell, ml, and clojure - the latter was within the last decade so i guess that's a valid point).
i'm starting to wonder if my question should have been something more like "are there any exciting, new, still under-the-radar ideas in software dev?"
i'm also starting to understand the old guys who, when i was starting out, claimed nothing was new....
posted by andrewcooke at 9:29 AM on October 25, 2015
i'm starting to wonder if my question should have been something more like "are there any exciting, new, still under-the-radar ideas in software dev?"
i'm also starting to understand the old guys who, when i was starting out, claimed nothing was new....
posted by andrewcooke at 9:29 AM on October 25, 2015
One trend we are seeing is an an incredible democratization of technology, which has given rise to unfathomable variety.
LLVM is just over a decade old, but as a result of it being free and accessible, software engineers have been able to craft a number of really interesting new programming languages that are nearly as fast as C.
Similarly, the JVM ecosystem is thriving, languages like Scala and Clojure are giving developers tons of alternatives to Java.
Likewise, on the web, Javascript has become a compiler target, via emscripten and other technologies. Typescript, Clojurescript, Dart, etc, are all giving programmers more choice.
So yea, huge proliferation of new languages, and the old joke is still true, most of them are just poor imitations of lisp(aside from Clojure, which is basically Lisp).
Meanwhile, there is a trend even in systems programming languages, to incorporate functional programming concepts, lambdas / clojures in C++, Java, Rust, etc.
The 90s saw the rise of a number of high level prototyping languages, ala python, ruby, objective C, perl, but now there is a movement in the opposite direction. Specifically, more safety. Rust, with its memory sanitization, Swift, Typescript, etc.
Looking towards the future, I expect this trend will continue. Dependent types are starting to creep into the mainstream, ATS, Idris, etc, design by contract will start to see some traction.
Given we've reached a kind of plateau for clock speeds, yet moore's law marches on, so we've seen the rise of multicore processors and languages better equipped to program for them. Go is one example, Erlang which is older has seen a spike in popularity. Also the cloud is really starting to take off. Similarly, programmable GPUs are changing the game for some types of programming problems.
Oh, and mobile is everything. We're starting to see the emergence of the 'internet of things' and 'write once, run everywhere' frameworks.
TDLR:
1) Democritization of technology and choice
2) Safety and functional programming
3) Concurrency, GPGPU and the cloud
4) Mobile everything, internet of things
posted by jalitt at 9:51 AM on October 25, 2015 [2 favorites]
LLVM is just over a decade old, but as a result of it being free and accessible, software engineers have been able to craft a number of really interesting new programming languages that are nearly as fast as C.
Similarly, the JVM ecosystem is thriving, languages like Scala and Clojure are giving developers tons of alternatives to Java.
Likewise, on the web, Javascript has become a compiler target, via emscripten and other technologies. Typescript, Clojurescript, Dart, etc, are all giving programmers more choice.
So yea, huge proliferation of new languages, and the old joke is still true, most of them are just poor imitations of lisp(aside from Clojure, which is basically Lisp).
Meanwhile, there is a trend even in systems programming languages, to incorporate functional programming concepts, lambdas / clojures in C++, Java, Rust, etc.
The 90s saw the rise of a number of high level prototyping languages, ala python, ruby, objective C, perl, but now there is a movement in the opposite direction. Specifically, more safety. Rust, with its memory sanitization, Swift, Typescript, etc.
Looking towards the future, I expect this trend will continue. Dependent types are starting to creep into the mainstream, ATS, Idris, etc, design by contract will start to see some traction.
Given we've reached a kind of plateau for clock speeds, yet moore's law marches on, so we've seen the rise of multicore processors and languages better equipped to program for them. Go is one example, Erlang which is older has seen a spike in popularity. Also the cloud is really starting to take off. Similarly, programmable GPUs are changing the game for some types of programming problems.
Oh, and mobile is everything. We're starting to see the emergence of the 'internet of things' and 'write once, run everywhere' frameworks.
TDLR:
1) Democritization of technology and choice
2) Safety and functional programming
3) Concurrency, GPGPU and the cloud
4) Mobile everything, internet of things
posted by jalitt at 9:51 AM on October 25, 2015 [2 favorites]
Oh, how could I forget,
5) Machine learning, traditional 'general artificial intelligence' has given way to neural nets, bayesian networks, etc.
6) Quantum computing is just starting to become a thing, and not just a fantasy. Not yet production ready however.
posted by jalitt at 10:02 AM on October 25, 2015
5) Machine learning, traditional 'general artificial intelligence' has given way to neural nets, bayesian networks, etc.
6) Quantum computing is just starting to become a thing, and not just a fantasy. Not yet production ready however.
posted by jalitt at 10:02 AM on October 25, 2015
There's a lot of debate between microservices and monoliths (and hybrids) right now. It's not really new, but one thing I didn't see touched on was non-ACID databases where traditional assumptions on what is needed are thrown out.
If you haven't read the Phoenix Project, you should. It won't be anything mind blowing but is one if the definitive DevOps books and is a fast read.
posted by Candleman at 11:36 AM on October 25, 2015 [1 favorite]
If you haven't read the Phoenix Project, you should. It won't be anything mind blowing but is one if the definitive DevOps books and is a fast read.
posted by Candleman at 11:36 AM on October 25, 2015 [1 favorite]
But they seem peripheral to development. Have I missed something?
Well, the "exciting, new idea" is that they're really not. The reason "DevOps" is no longer called "being a sysadmin" because what works managing a handful of machines doesn't work when you've got hundreds, thousands, or if you're in Google territory, millions of physical machines, especially when there are numerous virtual machines running on those physical machines. In order to be able to manage them, a person needs the skills to do what's traditionally been development work while at the same time also being responsible for making sure it's working at the end of the day.
"But it works on my computer" is no longer an acceptable excuse any more (not that it ever really was). If it takes more than a day to set up the build environment for a particular piece of software, then something is horribly wrong. (Though people are still catching up to that.)
Back to the question though, I'll second the Phoenix Project.
posted by fragmede at 1:19 PM on October 25, 2015 [1 favorite]
Well, the "exciting, new idea" is that they're really not. The reason "DevOps" is no longer called "being a sysadmin" because what works managing a handful of machines doesn't work when you've got hundreds, thousands, or if you're in Google territory, millions of physical machines, especially when there are numerous virtual machines running on those physical machines. In order to be able to manage them, a person needs the skills to do what's traditionally been development work while at the same time also being responsible for making sure it's working at the end of the day.
"But it works on my computer" is no longer an acceptable excuse any more (not that it ever really was). If it takes more than a day to set up the build environment for a particular piece of software, then something is horribly wrong. (Though people are still catching up to that.)
Back to the question though, I'll second the Phoenix Project.
posted by fragmede at 1:19 PM on October 25, 2015 [1 favorite]
Response by poster: phoenix project looks kinda fluffy. i haven't read the visible ops handbook which looks more technical. how do they compare?
posted by andrewcooke at 3:23 PM on October 25, 2015
posted by andrewcooke at 3:23 PM on October 25, 2015
Googleable things that are trending:
- Containerization
- Lambda architecture
- Functional programming
- "Streaming" services (e.g. Spark, Samza, Storm)
Also, QA as a discipline is kind of disappearing in some companies. Instead you have test engineers that work on tools that let the developers write their own tests. This often results in weak end-to-end high-level tests, IMHO.
Rise of dev ops. Dev ops people get more respect now and everyone is expected to do a lot more ops-level work at newish companies, compared to the old dinosaurs.
"Data science" is a thing now. Not sure that this profession really existed until ten-ish years ago; though maybe in some bigger companies it did.
posted by deathpanels at 5:19 PM on October 25, 2015 [1 favorite]
- Containerization
- Lambda architecture
- Functional programming
- "Streaming" services (e.g. Spark, Samza, Storm)
Also, QA as a discipline is kind of disappearing in some companies. Instead you have test engineers that work on tools that let the developers write their own tests. This often results in weak end-to-end high-level tests, IMHO.
Rise of dev ops. Dev ops people get more respect now and everyone is expected to do a lot more ops-level work at newish companies, compared to the old dinosaurs.
"Data science" is a thing now. Not sure that this profession really existed until ten-ish years ago; though maybe in some bigger companies it did.
posted by deathpanels at 5:19 PM on October 25, 2015 [1 favorite]
Response by poster: this seems to be the lambda architecture book. has anyone read it?
posted by andrewcooke at 5:25 PM on October 25, 2015
posted by andrewcooke at 5:25 PM on October 25, 2015
The Phoenix Project is entirely non-technical; it's a novel talking about various ideas of the movement. I find it's useful to throw at management to get buy in on why things need to change because it's fluffy and non-technical. I've not read the Visible Ops handbook but they're well respected authors in the field. I'd be a little worried that something technical written in 2005 would be a bit dated at this point. Technology and trends are running fast enough that things change to the degree that by the point books can be written and get to print, they may be out of date enough that they don't get the groundswell to become "the definitive" book in the way that Code Complete was back in the day.
For keeping abreast of what's the new hotness, I find it useful to skim the first page or two of Reddit's programming subreddit and reading an article or two.
posted by Candleman at 6:43 PM on October 25, 2015
For keeping abreast of what's the new hotness, I find it useful to skim the first page or two of Reddit's programming subreddit and reading an article or two.
posted by Candleman at 6:43 PM on October 25, 2015
« Older A couple of questions about Iain M Banks Culture... | Identify this German after dinner shot Newer »
This thread is closed to new comments.
Also, a specific stream-oriented functional reactive programming technique has been bolted onto many non-functional languages in the form of the extremely popular Reactive Extensions. For me, this library has taken writing complex concurrent code from "cold-sweat nightmare" to "tricky but doable." It's also ridiculously useful for complex/dynamic/heavily animated UI.
Finally, those thicker web clients you mention have been made possible by a new generation of web frontend frameworks, many of which tilt toward functional styles, are very-much opinionated, and do A LOT for the developer. The popular ones right now are AngularJS, React, and Ember. An overview can be found at todomvc.
Unfortunately I don't have any book recommendations on these topics. I've been looking myself, but I think the evolution in these areas is currently so fast that it's currently impractical to write authoritative books, although there are plenty of great resources on the web.
posted by tempythethird at 8:35 AM on October 25, 2015 [3 favorites]