Web development big picture for a non-web programmer?
June 8, 2009 5:48 AM Subscribe
Can you give me any pointers on learning web development? Unlike other similar questions in the archives (1, 2), I have a lot of programming experience. I haven't really followed web programming practices closely, however, because without keeping closely involved, it has been hard to sort out useful stuff from buzzwords and fads. (Non-web programming has enough of those!)
I strongly prefer having a big-picture understanding of systems when working with them, so the various "here's stuff to copy-and-paste to make this happen" sites don't do me much good. I'm not concerned with that level, rather, I want to pick up the underlying skill set, so I can teach myself the surface details as necessary. (FWIW, I recently learned standard practices for HTML, RSS, and have gotten a sense of how to (and not to) use CSS. The stuff about semantic markup makes a lot of sense to me. I suppose Javascript and jQuery come next?)
A good example of a page I find opaque is the wikipedia page on REST. It sounds like it's about managing state in a stateless protocol, and about abstracting away storage so that proxies, replacement servers, etc. can be used transparently, but the terminology is a bit alien, and I wonder if I'm just missing something.
For programming, I'm quite comfortable with Python, Lua, C, OCaml, and Scheme. Of those, Lua is especially noteworthy, both because it would be my first choice (Python second) and because it's semantically quite close to Javascript. I'm also familiar with RDBMSs (SQLite and postgres) and the relational model, and have about a decade's experience with Linux, BSD, and the associated toolchain.
For other languages people tend to suggest: I don't know PHP, and everything I've read about its design makes me wary. I have tried Ruby a bit, but don't care for it, and Lua + Python already cover that niche for me.
I'm not clear if learning a web framework would be a good idea for me or not. It would probably help with picking up the vocabulary I'm missing, but at the same time, the "handle the details for me" aspect would interfere with learning what's actually happening. I'm not worried about ORMs, etc., as I'm fine with setting up abstractions for the database myself. I'm also not concerned with scaling - I know a lot about algorithms, caching, etc. already, but I also know about premature optimization, so I won't worry about that unless something I write needs to scale. Abstracting away browser incompatibilities would be quite welcome, though. (Frameworks also seem to be where most of the buzz is -- advice like "check out _, it's teh new hotness" really doesn't tell me anything...)
I strongly prefer having a big-picture understanding of systems when working with them, so the various "here's stuff to copy-and-paste to make this happen" sites don't do me much good. I'm not concerned with that level, rather, I want to pick up the underlying skill set, so I can teach myself the surface details as necessary. (FWIW, I recently learned standard practices for HTML, RSS, and have gotten a sense of how to (and not to) use CSS. The stuff about semantic markup makes a lot of sense to me. I suppose Javascript and jQuery come next?)
A good example of a page I find opaque is the wikipedia page on REST. It sounds like it's about managing state in a stateless protocol, and about abstracting away storage so that proxies, replacement servers, etc. can be used transparently, but the terminology is a bit alien, and I wonder if I'm just missing something.
For programming, I'm quite comfortable with Python, Lua, C, OCaml, and Scheme. Of those, Lua is especially noteworthy, both because it would be my first choice (Python second) and because it's semantically quite close to Javascript. I'm also familiar with RDBMSs (SQLite and postgres) and the relational model, and have about a decade's experience with Linux, BSD, and the associated toolchain.
For other languages people tend to suggest: I don't know PHP, and everything I've read about its design makes me wary. I have tried Ruby a bit, but don't care for it, and Lua + Python already cover that niche for me.
I'm not clear if learning a web framework would be a good idea for me or not. It would probably help with picking up the vocabulary I'm missing, but at the same time, the "handle the details for me" aspect would interfere with learning what's actually happening. I'm not worried about ORMs, etc., as I'm fine with setting up abstractions for the database myself. I'm also not concerned with scaling - I know a lot about algorithms, caching, etc. already, but I also know about premature optimization, so I won't worry about that unless something I write needs to scale. Abstracting away browser incompatibilities would be quite welcome, though. (Frameworks also seem to be where most of the buzz is -- advice like "check out _, it's teh new hotness" really doesn't tell me anything...)
I can tell you that an obscene amount of what's going on right now in web dev land is all about New Hotnesses. Subscribe to some blogs and ignore anything they're really excited about, just absorb the general ideas. Having said that:
Since you're already into Python, why not take a look at Django? Frameworks are good to work with and know about - although they take control away from you they definitely speed up development time. As a Ruby dev, I recently moved from Rails (heavy heavy framework) to doing a lot of smaller projects in Sinatra (very very light framework), but the practices I picked up working in the structured Rails environment were helpful when recreating modes of action in Sinatra.
I'd also suggest that having something "handle the details" is just plain how web development works these days, and it doesn't mean that you won't learn anything about what's being handled. Stuff breaks, things don't work like you expect, there's a learning curve to a framework that instructs you in what the black box is doing. It's like when you're working with Visual Studio and your libraries are all missing and everything is blowing up and you have to get down and dirty, even though you'd think this great big product should do everything for you.
Oh, and there's always the framework's code. You can read that!
The best way to learn web dev is to just think of a project and do it, however you want. Find a web host that supports mod_python and you can throw up a 2 line python script that'll be a web page. Then try to turn it into a blog (learning basics), then add users (user auth, security), make it three columns (lots of CSS), allow commenting without refreshing the page (REST, jQuery), etc etc. Although at first you might not be using semantic markup or the Proper Methodologies the further you get into learning the more you'll run across things that will make you think "Oh, I'm doing this wrong, this way is better!" I find that more useful than just being told The Right Way and copying and pasting it.
Google early and often about how to do things - take the blogs you find useful and add them to your RSS feeds. Skimming them will give you an idea of what's hype and what isn't, and will probably help you track down more approachable sources for something like REST than a wikipedia entry.
posted by soma lkzx at 6:29 AM on June 8, 2009 [1 favorite]
Since you're already into Python, why not take a look at Django? Frameworks are good to work with and know about - although they take control away from you they definitely speed up development time. As a Ruby dev, I recently moved from Rails (heavy heavy framework) to doing a lot of smaller projects in Sinatra (very very light framework), but the practices I picked up working in the structured Rails environment were helpful when recreating modes of action in Sinatra.
I'd also suggest that having something "handle the details" is just plain how web development works these days, and it doesn't mean that you won't learn anything about what's being handled. Stuff breaks, things don't work like you expect, there's a learning curve to a framework that instructs you in what the black box is doing. It's like when you're working with Visual Studio and your libraries are all missing and everything is blowing up and you have to get down and dirty, even though you'd think this great big product should do everything for you.
Oh, and there's always the framework's code. You can read that!
The best way to learn web dev is to just think of a project and do it, however you want. Find a web host that supports mod_python and you can throw up a 2 line python script that'll be a web page. Then try to turn it into a blog (learning basics), then add users (user auth, security), make it three columns (lots of CSS), allow commenting without refreshing the page (REST, jQuery), etc etc. Although at first you might not be using semantic markup or the Proper Methodologies the further you get into learning the more you'll run across things that will make you think "Oh, I'm doing this wrong, this way is better!" I find that more useful than just being told The Right Way and copying and pasting it.
Google early and often about how to do things - take the blogs you find useful and add them to your RSS feeds. Skimming them will give you an idea of what's hype and what isn't, and will probably help you track down more approachable sources for something like REST than a wikipedia entry.
posted by soma lkzx at 6:29 AM on June 8, 2009 [1 favorite]
Thirding Django, as it's No-Magic approach means you actually understand what it's doing, and can always look at the code for further clarification.
Even if you don't end up using it, learning its web dev pattern will help you devise your own.
Plus, one of the lead developers is a Mefite.
posted by signal at 6:47 AM on June 8, 2009
Even if you don't end up using it, learning its web dev pattern will help you devise your own.
Plus, one of the lead developers is a Mefite.
posted by signal at 6:47 AM on June 8, 2009
Response by poster: scrottreynen: "progressive enhancement" sounds like a useful term, thanks.
soma lkzx: I looked at Django a little bit around Christmas. I'm fine with using something like that once I have a big picture understanding of what's going on underneath, but I would rather learn that first. A more minimal framework may be more helpful (particularly if I dig into the source, e.g. porting it from Python to Lua). I've heard web.py suggested before, but that was a while ago. Any really lightweight Python frameworks anybody would recommend? (Also, the REST page was just quick to grab and suitably buzzword-y. I try to read primary sources.)
posted by silentbicycle at 6:52 AM on June 8, 2009
soma lkzx: I looked at Django a little bit around Christmas. I'm fine with using something like that once I have a big picture understanding of what's going on underneath, but I would rather learn that first. A more minimal framework may be more helpful (particularly if I dig into the source, e.g. porting it from Python to Lua). I've heard web.py suggested before, but that was a while ago. Any really lightweight Python frameworks anybody would recommend? (Also, the REST page was just quick to grab and suitably buzzword-y. I try to read primary sources.)
posted by silentbicycle at 6:52 AM on June 8, 2009
Best answer: There's no real reason to move off of Python, and yeah, a web framework wouldn't be bad to learn. Python is pretty easy to RYO, though, if you feel like you'd be better off starting from scratch. In general, I've had poor luck with frameworks - I usually end up throwing a lot of the stuff away or writing in hacks where I need to because it doesn't do what I want and isn't quite flexible enough to extend without massive work. I've heard Django is a bit better in this regard, but I'd still give some basic programming a shot without it at first.
Also, it's not a very popular option, but if you feel like sticking with OCaml, there's a web server called ocsigen that is actually pretty nice, and an apache extension called mod_caml if you don't want to have another server process running. I've played around with ocsigen and it's actually pretty neat.
posted by devilsbrigade at 6:56 AM on June 8, 2009
Also, it's not a very popular option, but if you feel like sticking with OCaml, there's a web server called ocsigen that is actually pretty nice, and an apache extension called mod_caml if you don't want to have another server process running. I've played around with ocsigen and it's actually pretty neat.
posted by devilsbrigade at 6:56 AM on June 8, 2009
Best answer: There are really only three underlying technologies involved in web design: HTTP, HTML, and JavaScript (on the client). Everything else is built on top of those things, and you can do without them. JQuery, for example, is built on JavaScript. REST is built on HTTP. And most web frameworks fundamentally just spit out HTML and maybe some Javascript if it's a framework that does dynamic stuff.
If you really want to learn how this stuff works at a low level, write your own web server. It's really easy: to start with just write a program that listens on port TCP port 80 and squirts out two blank lines and then some HTML. Read the HTTP specification and implement the basics, they're really simple. (basically HTTP is just a command/response, a few headers in the form 'header: value', then two newlines, then the content.
Of course I'm assuming you know the basics of HTML. I would try to learn the basic Javascript API and then move on to jQuery so you don't have to test on every browser.
So basically what I'm saying is start from scratch and teach yourself the very fundamentals: HTTP, HTML and JavaScript. They're all easy, and then you'll get a good idea of what the software frameworks can do for you.
posted by delmoi at 7:10 AM on June 8, 2009 [3 favorites]
If you really want to learn how this stuff works at a low level, write your own web server. It's really easy: to start with just write a program that listens on port TCP port 80 and squirts out two blank lines and then some HTML. Read the HTTP specification and implement the basics, they're really simple. (basically HTTP is just a command/response, a few headers in the form 'header: value', then two newlines, then the content.
Of course I'm assuming you know the basics of HTML. I would try to learn the basic Javascript API and then move on to jQuery so you don't have to test on every browser.
So basically what I'm saying is start from scratch and teach yourself the very fundamentals: HTTP, HTML and JavaScript. They're all easy, and then you'll get a good idea of what the software frameworks can do for you.
posted by delmoi at 7:10 AM on June 8, 2009 [3 favorites]
Best answer: Any really lightweight Python frameworks anybody would recommend?
I'm not a Python developer, but I read a few blogs of Python developers. One of those is Simon Willison, who recently released a new Python microframework. In that post, he references several other microframeworks. The most popular seem to be web.py and Sinatra.
posted by scottreynen at 7:23 AM on June 8, 2009
I'm not a Python developer, but I read a few blogs of Python developers. One of those is Simon Willison, who recently released a new Python microframework. In that post, he references several other microframeworks. The most popular seem to be web.py and Sinatra.
posted by scottreynen at 7:23 AM on June 8, 2009
Best answer: I don't understand why using a framework and learning the details are mutually exclusive. They might be, if you were lazy. You don't sound lazy. Instead, use the framework as a learning tool.
A well written and documented framework like Django will teach you more about programming for the web in less time than writing things from scratch. Create a simple "Hello world" page in Django and trace how it processes the incoming request to execute your code. Then follow how it uses your code to create a complete response. You can go through a similar process with database objects, different request/response types, etc.
I'd recommend a similar approach with JQuery. Don't shy away from using a JS framework because you're afraid you won't learn. Use JQuery, but make sure you understand what it's doing.
posted by christonabike at 8:25 AM on June 8, 2009
A well written and documented framework like Django will teach you more about programming for the web in less time than writing things from scratch. Create a simple "Hello world" page in Django and trace how it processes the incoming request to execute your code. Then follow how it uses your code to create a complete response. You can go through a similar process with database objects, different request/response types, etc.
I'd recommend a similar approach with JQuery. Don't shy away from using a JS framework because you're afraid you won't learn. Use JQuery, but make sure you understand what it's doing.
posted by christonabike at 8:25 AM on June 8, 2009
Best answer: Over on stackoverflow, there's a very well fleshed out answer about what should a developer know before building a public web site. It looks like it was asked by a beginner, but with all the things they suggest learning, its best viewed as how to be an expert web dev.
As for the architecture, I have some thing like this book bookmarked somewhere: Web Application Architecture: Principles, Protocols and Practices.
posted by philomathoholic at 9:08 AM on June 8, 2009 [1 favorite]
As for the architecture, I have some thing like this book bookmarked somewhere: Web Application Architecture: Principles, Protocols and Practices.
posted by philomathoholic at 9:08 AM on June 8, 2009 [1 favorite]
You may wanna look at WSGI if you really wanna get your hands messy with python on the web. While traditionally mod_python was used for python web development (basically embedding python in the web server ala mod_perl or mod_php), it's pretty stagnant as a project right now.
Anyhoo, WSGI is pretty much the new standard interface* for programming web applications with python, and here's a great article on writing your own framework from the ground up, and here's an intro to wsgi. Good luck FWIW I sometimes wish I had tried them out before I learned ROR and Django -- oh, and Sinatra's really fun if you wanna diversion.
On the frontend *ahem* front, browser quirks will drive you batty as a coder. I'd really recommend using a framework (jquery, prototype, dojo), because they abstract away the half assed job most browsers call their implementaion of the DOM, just fiddle with one and see what you like, they all basically provide the same things: a selector engine, a bunch of relatively silly browser effects, some helper methods for the quirkier areas of JS and implementations of ajaxy type stuff. If you really are itching for deeper knowledge of JS go a head and read this book. JavaScript's actually a pretty cute and quite cool language at it's heart, and that book takes you on a guided tour of the things that actually work in modern browsers, plus it's short.
Any way good luck! Web Development is pretty different than regular app dev, but it sure is fun to tinker with, and experiments are easier to implement IMHO.
* this may be a bit optimistic, it's my impression and I could be wrong about it
posted by Jeff_Larson at 12:58 AM on June 20, 2009
Anyhoo, WSGI is pretty much the new standard interface* for programming web applications with python, and here's a great article on writing your own framework from the ground up, and here's an intro to wsgi. Good luck FWIW I sometimes wish I had tried them out before I learned ROR and Django -- oh, and Sinatra's really fun if you wanna diversion.
On the frontend *ahem* front, browser quirks will drive you batty as a coder. I'd really recommend using a framework (jquery, prototype, dojo), because they abstract away the half assed job most browsers call their implementaion of the DOM, just fiddle with one and see what you like, they all basically provide the same things: a selector engine, a bunch of relatively silly browser effects, some helper methods for the quirkier areas of JS and implementations of ajaxy type stuff. If you really are itching for deeper knowledge of JS go a head and read this book. JavaScript's actually a pretty cute and quite cool language at it's heart, and that book takes you on a guided tour of the things that actually work in modern browsers, plus it's short.
Any way good luck! Web Development is pretty different than regular app dev, but it sure is fun to tinker with, and experiments are easier to implement IMHO.
* this may be a bit optimistic, it's my impression and I could be wrong about it
posted by Jeff_Larson at 12:58 AM on June 20, 2009
Response by poster: Follow-up:
delmoi's suggestion to just read the specs and write my own basic web server from the ground up seemed to be the best approach (for me, at least).
I've implemented a library for the majority of the RFC-2616 spec in Lua, a coroutine & event-based web server, and a minimal web framework (all of which I'll release, once they're more polished).
The web framework turned out kind of like Sinatra, whose approach makes a lot more sense to me than all-in-one frameworks such as Django. I hadn't seen Sinatra until I was already almost done with it, though - My inspiration was Prolog/Erlang-style declarative programming.
posted by silentbicycle at 10:41 AM on March 12, 2010
delmoi's suggestion to just read the specs and write my own basic web server from the ground up seemed to be the best approach (for me, at least).
I've implemented a library for the majority of the RFC-2616 spec in Lua, a coroutine & event-based web server, and a minimal web framework (all of which I'll release, once they're more polished).
The web framework turned out kind of like Sinatra, whose approach makes a lot more sense to me than all-in-one frameworks such as Django. I hadn't seen Sinatra until I was already almost done with it, though - My inspiration was Prolog/Erlang-style declarative programming.
posted by silentbicycle at 10:41 AM on March 12, 2010
This thread is closed to new comments.
Python is a widely-used web dev language, so I'd stick with that.
If you like to understand how things work, learn JavaScript before touching jQuery, and try straight Python before Django (the popular Python framework). Try to stick with progressive enhancement in any JavaScript you do, i.e. it should generally work without JavaScript.
It sounds like you already have enough knowledge to start making things, so you should start making things. You'll quickly discover where you need to learn more.
posted by scottreynen at 6:23 AM on June 8, 2009