Seriously, everyone on the web is scripting nowadays.
June 26, 2015 7:57 AM Subscribe
Why is the web written in scripting languages? It seems most web development today - at least on new-ish websites - is done in scripting languages: PHP, Python (Django/Flask), Ruby (Rails), JavaScript (NodeJS), etc. Is there something about scripting languages that makes them especially well-suited to server-side web development compared to other languages? Or is it just that scripting languages are popular and have useful features, so naturally they're applied to lots of programming tasks today - including web development?
Best answer: Scripting languages have pretty much always been used to build web stuff. The main reason is probably that they're "fast enough" and much easier to quickly write code in. They're also probably mildly more suited to casual programming since most of them do not make you think about pointers or memory management or garbage collection etc.
The slowest parts of most web applications have typically been database access. If the time your web page spends is dominated by something like that, which can't be sped up with a compiled programming language, then it doesn't really matter if the rest of is at maximum efficiency or not. Like if it takes .5s to do all the DB access and .05s to run all the rest, then it doesn't really matter if you can make "the rest" faster or not.
Also, most scripting languages are extensible by writing compiled code. When I first started making web apps in the late 90s, our plan was always to iterate by writing scripting code first, and replacing any parts that were too slow with C++ extensions. Hint: we *almost never* had to write those extensions.
posted by RustyBrooks at 8:26 AM on June 26, 2015 [5 favorites]
The slowest parts of most web applications have typically been database access. If the time your web page spends is dominated by something like that, which can't be sped up with a compiled programming language, then it doesn't really matter if the rest of is at maximum efficiency or not. Like if it takes .5s to do all the DB access and .05s to run all the rest, then it doesn't really matter if you can make "the rest" faster or not.
Also, most scripting languages are extensible by writing compiled code. When I first started making web apps in the late 90s, our plan was always to iterate by writing scripting code first, and replacing any parts that were too slow with C++ extensions. Hint: we *almost never* had to write those extensions.
posted by RustyBrooks at 8:26 AM on June 26, 2015 [5 favorites]
Best answer: So I've started rewriting a content management system for one of my sites in C++. The project is actually pretty far along, and now includes an HTTP server.
The reasons I did so is that I've been doing a lot of professional work in Perl, and I'm finding that the lack of typing and compile time restrictions lead to all sorts of maintainability issues in code. I figured that if I could get to the point in my personal library where I could write the quick one-off apps in C++ rather than Perl or Python or local JavaScript (ie: NodeJS), I wouldn't be tempted to go down the oh-so-attractive dead-ends that those languages provide.
The main things I've noticed:
At least until the system is entrenched and large...
posted by straw at 8:31 AM on June 26, 2015 [3 favorites]
The reasons I did so is that I've been doing a lot of professional work in Perl, and I'm finding that the lack of typing and compile time restrictions lead to all sorts of maintainability issues in code. I figured that if I could get to the point in my personal library where I could write the quick one-off apps in C++ rather than Perl or Python or local JavaScript (ie: NodeJS), I wouldn't be tempted to go down the oh-so-attractive dead-ends that those languages provide.
The main things I've noticed:
- Scripting languages usually impose a set of common stylistic patterns that C++ doesn't. The proponents of a given language will have standard naming conventions that make code that integrates outside projects just look more consistent.
- Similarly, scripting languages also have tools to manage those external libraries. In Perl, CPAN provides a standard way for me to search for tools, and also limits the amount of tools to evaluate. There are a limited number of XML parser bindings, or SOAP tools, or whatever. Sure, I've got to do a little warping to make one of those work rather than finding something that's absolutely the right choice, but I know that my skills will be more transferable, and someone's already made most of the choice for me.
- Scripting languages have more common execution environments. Many shared hosting plans have the ability to run all sorts of languages, but you have to step up a plan or two to compile C++ code, or run Java code, on the server.
- For all of the reasons that I want typing, it is very freeing in the initial stages of development to be able to say "oh, yeah, that needs one of these, and one of those, and...". I mean, just look at how many fields in your average database are "VARCHAR(253)". And, to be fair, this is actually often a good thing: We're just passing data through systems. Back in the day, I remember writing a UI that made damned sure you could only enter a valid phone number. This was right before the rules on 1+ dialing and the format of area codes came into play. Was it a good thing that we enforced those rules on the UI, when really what we needed was a note that got attached to a person that someone else could use to contact them later?
At least until the system is entrenched and large...
posted by straw at 8:31 AM on June 26, 2015 [3 favorites]
I think scripting languages are well-suited for the web because they function as simple templating languages as well, so that PHP/Python/whatever can be intermingled with HTML (for better or worse).
posted by misterbrandt at 8:39 AM on June 26, 2015 [5 favorites]
posted by misterbrandt at 8:39 AM on June 26, 2015 [5 favorites]
The above answers are great. My own two cents: We don't really have "scripting" languages like we used to. We have very-high-level languages for which an interpreter is available; pretty much all of them, when used as web-app server backends, are compiled (JIT or otherwise). How we got there is a longer story (like what misterbrandt points out about templating) but calling them "Scripting" languages just because they may have started as scripting tools is misleading.
posted by Tomorrowful at 8:43 AM on June 26, 2015 [4 favorites]
posted by Tomorrowful at 8:43 AM on June 26, 2015 [4 favorites]
Yeah, my initial response here is "as opposed to... what?"
Scripting languages have always been a preferred option for broad-based general use, back to the early/mid-90s when it was Perl (or even shell!) CGI, and the main restriction was what your local BOFH would permit on one of those new-fangled web servers. They offer portability, extensibility, and some degree of "View Source" that aligns with the basic principles of HTML. You can cobble things together from libraries and modules because, as straw suggests, a lot of dynamic web stuff is shuffling untyped data around.
As Philip Greenspun wrote in his mid-90s guide to dynamic web sites: "How could a lame string-oriented scripting language possibly compete in power with systems programming languages? Well, guess what? The only data type that you can write to a Web browser is a string."
There are certainly rationales for compiled application servers, but they're tied to specific cases.
posted by holgate at 8:43 AM on June 26, 2015 [4 favorites]
Scripting languages have always been a preferred option for broad-based general use, back to the early/mid-90s when it was Perl (or even shell!) CGI, and the main restriction was what your local BOFH would permit on one of those new-fangled web servers. They offer portability, extensibility, and some degree of "View Source" that aligns with the basic principles of HTML. You can cobble things together from libraries and modules because, as straw suggests, a lot of dynamic web stuff is shuffling untyped data around.
As Philip Greenspun wrote in his mid-90s guide to dynamic web sites: "How could a lame string-oriented scripting language possibly compete in power with systems programming languages? Well, guess what? The only data type that you can write to a Web browser is a string."
There are certainly rationales for compiled application servers, but they're tied to specific cases.
posted by holgate at 8:43 AM on June 26, 2015 [4 favorites]
I think scripting languages are well-suited for the web because they function as simple templating languages as well, so that PHP/Python/whatever can be intermingled with HTML (for better or worse).
Yep, and that was more obvious with ASP/IIS on the Windows side back in the day, where it was a kind of progression from server-side includes.
I'd agree with Tomorrowful that the interesting question here is perhaps how we got to high-level MVC frameworks (probably starting from Django and Rails in the mid-00s) with distinct approaches to the HTTP server, which is somewhat different from PHP's long-standing "drop it in public_html" portability, and has influenced PHP frameworks like CakePHP, CodeIgniter and Laravel.
posted by holgate at 8:55 AM on June 26, 2015 [2 favorites]
Yep, and that was more obvious with ASP/IIS on the Windows side back in the day, where it was a kind of progression from server-side includes.
I'd agree with Tomorrowful that the interesting question here is perhaps how we got to high-level MVC frameworks (probably starting from Django and Rails in the mid-00s) with distinct approaches to the HTTP server, which is somewhat different from PHP's long-standing "drop it in public_html" portability, and has influenced PHP frameworks like CakePHP, CodeIgniter and Laravel.
posted by holgate at 8:55 AM on June 26, 2015 [2 favorites]
It's not scripting that has taken over, it's languages with garbage collection. Modern scripting "interpreters" (actually powerful optimizing compilers in their own right), just happen to do this too along with Java, C#, Scala, Go, Swift. With web servers running perpetually, you want to avoid memory leaks and other traditional problems.
posted by nickggully at 9:24 AM on June 26, 2015 [6 favorites]
posted by nickggully at 9:24 AM on June 26, 2015 [6 favorites]
the client, of course, uses javascript. the server, otoh...it's much easier to write shitty code quickly in interpreted languages. weak typing and procedural recipe-based routines seem to have been de facto written to accomodate cheap, fast bullshit. since the bar for profitable web dev is just-un-shitty-enough-to get-paid...what it comes down to is *sufficiency*.
how we got to high-level MVC...
well...MVC was popularized in Design Patterns (GoF) in 1995.
then, an http variation was developed called Model2. Every web variation that followed (Ruby on Rails, Django, Asp.Net MVC) is not actually MVC *at all*, but hijacked the terminolgy of MVC for their individual implementations of Model2. this blurring of definition has definitely cost the industry clarity wrt - for example - implementing actual MVC in a client architectuire.
posted by j_curiouser at 10:00 AM on June 26, 2015 [1 favorite]
how we got to high-level MVC...
well...MVC was popularized in Design Patterns (GoF) in 1995.
then, an http variation was developed called Model2. Every web variation that followed (Ruby on Rails, Django, Asp.Net MVC) is not actually MVC *at all*, but hijacked the terminolgy of MVC for their individual implementations of Model2. this blurring of definition has definitely cost the industry clarity wrt - for example - implementing actual MVC in a client architectuire.
posted by j_curiouser at 10:00 AM on June 26, 2015 [1 favorite]
An even broader factor is the emergence of commodity VPS hosting (Linode, Slicehost, EC2, DigitalOcean, etc.) and cloud PaaS providers like Heroku and various tools that make it relatively easy to deploy and maintain a framework-driven site with a non-AMP stack. If you wanted to run a site on Rails soon after it was released, setting it up on the server side was a right royal pain. Running a (non-PHP) framework-driven site these days is much easier.
posted by holgate at 10:19 AM on June 26, 2015
posted by holgate at 10:19 AM on June 26, 2015
The counterexamples to your question are Java and more recently, Scala and Go (Golang). Also the .NET world, which is more like a compiled system than a scripting system.
I think the reason for scripting being used so much is scripting is more flexible, and simpler to iterate and rapidly develop. Also strong libraries for string processing. The performance question is complicated; I can't tell you how many Ruby on Rails sites I know have been reimplemented in something, anything that runs more efficiently. See also HipHop, Facebook's compiler for PHP.
posted by Nelson at 10:47 AM on June 26, 2015 [3 favorites]
I think the reason for scripting being used so much is scripting is more flexible, and simpler to iterate and rapidly develop. Also strong libraries for string processing. The performance question is complicated; I can't tell you how many Ruby on Rails sites I know have been reimplemented in something, anything that runs more efficiently. See also HipHop, Facebook's compiler for PHP.
posted by Nelson at 10:47 AM on June 26, 2015 [3 favorites]
If the question is why are scripting languages bleeding over from strictly the client side to server side, the answer is there's a fashion trend right now for developers to demand that the same language they use on one side also be used on the other.
Others have already made the point - but there's lots of things standing between the client and performance, such that the relative slowness of scripting language client side didn't make a huge difference. Server side is another story, however - and the rush to use scripting languages server side is a bit perplexing.
posted by NoRelationToLea at 11:00 AM on June 26, 2015
Others have already made the point - but there's lots of things standing between the client and performance, such that the relative slowness of scripting language client side didn't make a huge difference. Server side is another story, however - and the rush to use scripting languages server side is a bit perplexing.
posted by NoRelationToLea at 11:00 AM on June 26, 2015
I think it has mostly to do with startup costs. I can use a text editor to bang out some PHP/Python/Ruby/Perl/Javascript/whatever code, drop it in the public_html folder of a cheap shared web host and away we go. Can you do that with a compiled language? Not without adding a compile step to your deployment process, and very likely not with that cheap shared web host.
posted by Aleyn at 1:02 PM on June 26, 2015 [1 favorite]
posted by Aleyn at 1:02 PM on June 26, 2015 [1 favorite]
RustyBrooks has it – they are plenty fast, and JIT compilers keep getting faster.
As for untyped languages => bad code, that is FUD. I've seen tons of garbage in C++. At this point, we've learned that tests are far more important for ensuring predictability and reliability in programs.
posted by ignignokt at 2:30 PM on June 26, 2015 [3 favorites]
As for untyped languages => bad code, that is FUD. I've seen tons of garbage in C++. At this point, we've learned that tests are far more important for ensuring predictability and reliability in programs.
posted by ignignokt at 2:30 PM on June 26, 2015 [3 favorites]
The dynamic web has always been largely scripting, it just used to be Perl, PHP, and ASP and a few others like Cold Fusion. There's still tons of work done with Java and C# as well.
posted by Candleman at 3:19 PM on June 26, 2015
posted by Candleman at 3:19 PM on June 26, 2015
It really just boils down to: that's what the people who made these sites know. PHP has been around for ages and is easy to pick up, and it already runs on most hosted servers. Ditto with Django and Rails. Ruby and Python are commonly installed by admins. There are also lots of resources online for learning these platforms, and dynamic languages are easier to pick up for those learning to program (quicker feedback cycle, duck typing, etc). Since the barrier to entry is lower, you naturally end up with a lot of sites built on these platforms.
Most web apps are doing pretty simple string processing. So there's no reason to prefer Rails over, say, a custom framework written in Haskell. But Haskell is relatively inaccessible and hard to learn compared to Ruby on Rails.
Other than that, I think it's all just trends. Who's to say if there's really a reason why Node.js became popular versus any other platform? I think it's just because a lot of people already knew JavaScript and the community that formed around that project was good at promoting it and getting it adopted by notable companies.
posted by deathpanels at 8:22 PM on June 26, 2015
Most web apps are doing pretty simple string processing. So there's no reason to prefer Rails over, say, a custom framework written in Haskell. But Haskell is relatively inaccessible and hard to learn compared to Ruby on Rails.
Other than that, I think it's all just trends. Who's to say if there's really a reason why Node.js became popular versus any other platform? I think it's just because a lot of people already knew JavaScript and the community that formed around that project was good at promoting it and getting it adopted by notable companies.
posted by deathpanels at 8:22 PM on June 26, 2015
I tend to agree with deathpanels - another way I would put it is "nothing succeeds like success." Communities grow up around winners, and when something gains momentum, there's a tendency to see it as the hammer that makes everything look like a nail. The second alternative you listed in your question "is it just that scripting languages are popular..." is kind of it.
I'm not sure if this is in your question, but one reason web sites are often set up in CMS systems like Wordpress (which is, of course, written in PHP), even when it seems to be a ridiculous waste of resources (setting up WP to show literally one page) is that it's much faster to stand it up than it would be to write HTML from scratch. In the world of SMALL web sites, server resources are cheap, time is expensive.
Of course, as the site scales up, that dynamic flips a bit - you'll spend time figuring out how to make the site more efficient and faster when a) the bandwidth costs start getting significant and b) you'd lose more in in traffic and sales because your site is slow than the optimization time is going to cost.
posted by randomkeystrike at 7:15 AM on June 27, 2015 [1 favorite]
I'm not sure if this is in your question, but one reason web sites are often set up in CMS systems like Wordpress (which is, of course, written in PHP), even when it seems to be a ridiculous waste of resources (setting up WP to show literally one page) is that it's much faster to stand it up than it would be to write HTML from scratch. In the world of SMALL web sites, server resources are cheap, time is expensive.
Of course, as the site scales up, that dynamic flips a bit - you'll spend time figuring out how to make the site more efficient and faster when a) the bandwidth costs start getting significant and b) you'd lose more in in traffic and sales because your site is slow than the optimization time is going to cost.
posted by randomkeystrike at 7:15 AM on June 27, 2015 [1 favorite]
Re: this part of your question:
Or is it just that scripting languages are popular and have useful features, so naturally they're applied to lots of programming tasks today - including web development?
Node.js is particularly good with concurrency. If you've written multi-threaded code in C++, you know how fraught that undertaking that is. Things are better in Objective-C with GCD, but it's still the most common source of EXC_BAD_ACCESS errors.
Node, instead of making you code threads (although you can if you really, really want to), keeps your program code in a single thread and provides events for I/O. It says, OK, I will do this file read or socket write for you, and you give me a function that I should give the results to when they're ready. That function has the results along with access to whatever context it was defined in.
In the meantime, you keep doing your thing or just go to sleep if there's nothing to do. It saves a lot of resources (it has a very small memory footprint) and keeps a lot of complexity out of your program.
It also provides great support for sharable modules. While individual module quality varies greatly, actually grabbing one though NPM and using it is a seamless process.
posted by ignignokt at 7:58 AM on June 27, 2015 [1 favorite]
Or is it just that scripting languages are popular and have useful features, so naturally they're applied to lots of programming tasks today - including web development?
Node.js is particularly good with concurrency. If you've written multi-threaded code in C++, you know how fraught that undertaking that is. Things are better in Objective-C with GCD, but it's still the most common source of EXC_BAD_ACCESS errors.
Node, instead of making you code threads (although you can if you really, really want to), keeps your program code in a single thread and provides events for I/O. It says, OK, I will do this file read or socket write for you, and you give me a function that I should give the results to when they're ready. That function has the results along with access to whatever context it was defined in.
In the meantime, you keep doing your thing or just go to sleep if there's nothing to do. It saves a lot of resources (it has a very small memory footprint) and keeps a lot of complexity out of your program.
It also provides great support for sharable modules. While individual module quality varies greatly, actually grabbing one though NPM and using it is a seamless process.
posted by ignignokt at 7:58 AM on June 27, 2015 [1 favorite]
Or is it just that scripting languages are popular and have useful features, so naturally they're applied to lots of programming tasks today - including web development?This is definitely the case in some areas - in my experience, at least, a lot of numerical/scientific computing tasks that would have been done in Java or C++ fifteen years ago are being done in Python nowadays. (It's becoming very popular in bioinformatics, for example.)
I can't authoritatively say why. I suspect part of it is simple network effect: there are lots of nice tools and libraries available and pragmatic people will tend to go where these are. On top of that, interpreted languages often have nice REPL shells that reduce the amount of time between thinking up a hypothesis and testing it. That can be more valuable than the savings in execution time that you'd get from optimized compiled code.
posted by Zeinab Badawi's Twenty Hotels at 8:22 AM on June 27, 2015 [1 favorite]
I have feelings on this; it informed my job choices after graduation. Let the computer do the low-level work!
There were studies that say that developers can write a certain number of lines of code per day, and that number doesn't correlate much with the language used. So you may as well use expressive languages.
Compiled languages have a lot of issues that make them a pain. I mean, to point at C and scoff: buffer overflows because you have to declare how long a string could be. You have the opportunity / pain of optimizing for a certain CPU or architecture, and then if you change your server from, say, 32-bit to 64-bit you have to rebuild, maybe unveiling exciting new bugs. Ditto for compilers and *versions* of compilers. And make sure all of the libraries you depend on are installed, which is going to be a pain to keep up-to-date. (Because: root access? or install in a possibly-badly-supported special bin directory?) And working across Mac, Windows, and Linux is not trivial. Even working across different flavors of Linux is likely going to require recompiling and testing.
With those in mind, I'd say, hey, working with many compiled languages on a web server is like reinventing the wheel. They *might* run faster on the server, but beyond a certain point, optimizing for developer time and maintenance tends to have a better payoff re: features.
posted by Pronoiac at 12:46 PM on June 27, 2015 [1 favorite]
There were studies that say that developers can write a certain number of lines of code per day, and that number doesn't correlate much with the language used. So you may as well use expressive languages.
Compiled languages have a lot of issues that make them a pain. I mean, to point at C and scoff: buffer overflows because you have to declare how long a string could be. You have the opportunity / pain of optimizing for a certain CPU or architecture, and then if you change your server from, say, 32-bit to 64-bit you have to rebuild, maybe unveiling exciting new bugs. Ditto for compilers and *versions* of compilers. And make sure all of the libraries you depend on are installed, which is going to be a pain to keep up-to-date. (Because: root access? or install in a possibly-badly-supported special bin directory?) And working across Mac, Windows, and Linux is not trivial. Even working across different flavors of Linux is likely going to require recompiling and testing.
With those in mind, I'd say, hey, working with many compiled languages on a web server is like reinventing the wheel. They *might* run faster on the server, but beyond a certain point, optimizing for developer time and maintenance tends to have a better payoff re: features.
posted by Pronoiac at 12:46 PM on June 27, 2015 [1 favorite]
I was walking with a coworker this afternoon, and he pointed out the obvious:
The best way to figure out if something is a good idea is to implement it.
If it's a good idea, then you get the luxury of the space to remove some of the technical debt which is financing the growth of the organization.
To ignignokt's point about tests, one of the reasons I've never started working on a scripting-language typed system which had sufficient test coverage to compete with the compiled systems which had extensive tests, was that the former were either being tested, or are now migrating from "oh, yeah, this is a good idea" to code bases where that technical debt is being removed and tests are being written, and the compiled language systems have been known good ideas that had the resources necessary to implement them right.
So, yeah: One of the reasons so many things are written in whatever gets the system implemented fast is that there's a bias towards trying out ideas cheaply enough that if they fail, whatever, and then bother to implement them right if it turns out that they're worth doing.
And on MVC, I'd encourage j_curiouser to look back further: MVC was a common pattern in GUI apps since... well... the '70s. Loooong before the Software Design Patterns craze.
posted by straw at 5:23 PM on June 30, 2015
The best way to figure out if something is a good idea is to implement it.
If it's a good idea, then you get the luxury of the space to remove some of the technical debt which is financing the growth of the organization.
To ignignokt's point about tests, one of the reasons I've never started working on a scripting-language typed system which had sufficient test coverage to compete with the compiled systems which had extensive tests, was that the former were either being tested, or are now migrating from "oh, yeah, this is a good idea" to code bases where that technical debt is being removed and tests are being written, and the compiled language systems have been known good ideas that had the resources necessary to implement them right.
So, yeah: One of the reasons so many things are written in whatever gets the system implemented fast is that there's a bias towards trying out ideas cheaply enough that if they fail, whatever, and then bother to implement them right if it turns out that they're worth doing.
And on MVC, I'd encourage j_curiouser to look back further: MVC was a common pattern in GUI apps since... well... the '70s. Loooong before the Software Design Patterns craze.
posted by straw at 5:23 PM on June 30, 2015
Response by poster: Yeah, my initial response here is "as opposed to... what?"
That's a good point, and I specifically didn't include it in the question because I didn't want it to turn into a discussion of scripting language vs not-scripting or interpreted language vs compiled. The difference between interpreted and compiled is getting very blurry, so that's not the distinction I was going for. And while there is a difference between scripting languages (Python, Ruby, JavaScript, etc) and not-scripting languages (C++, Java, etc), we could have really derailed into a discussion of what "scripting language" means instead of answering the higher-level question. So I didn't mention that aspect.
posted by Tehhund at 4:56 AM on August 6, 2015
That's a good point, and I specifically didn't include it in the question because I didn't want it to turn into a discussion of scripting language vs not-scripting or interpreted language vs compiled. The difference between interpreted and compiled is getting very blurry, so that's not the distinction I was going for. And while there is a difference between scripting languages (Python, Ruby, JavaScript, etc) and not-scripting languages (C++, Java, etc), we could have really derailed into a discussion of what "scripting language" means instead of answering the higher-level question. So I didn't mention that aspect.
posted by Tehhund at 4:56 AM on August 6, 2015
« Older Why am I not in love with my amazing friend who... | Ideas for a copyeditor returning to the biz? Newer »
This thread is closed to new comments.
posted by Brian Puccio at 8:20 AM on June 26, 2015