Python programming: what next?
September 26, 2010 12:49 PM   Subscribe

I know basic XHTML and CSS, and I'm almost done with my first Python book. I want to build a database-driven website. What should I learn next? Book/website recommendations and general advice welcome.

This is the Python book I've been using, and though I'm not quite finished, I've had absolutely no problems with learning from it so far. Actually, I've been finding programming super fun.

I've been told the next obvious step CS-wise is data structures and algorithms, but everything I've seen at the library has been very C++/Java-centric. Also, right now I feel like my (limited) web development knowledge and my (very limited) programming knowledge are two separate things that I don't know how to fit together. The database-driven website I want to build is entirely my own project, so there are no real deadlines. I'll learn whatever languages/tools I need to, but I gather that Python is appropriate for most web development tasks from previous questions. If I'm wrong, please correct me. (And point me to relevant tutorials for what you do recommend!)

So, here are my questions:

1. Any recommendations for data structures and algorithms materials that use Python? (Or do I really need to learn C++?)

2. I've seen a lot of intro to back-end development books and tutorials that center around PHP/MySQL. Are there comparable ones that use Python? At what point should I start learning Django?

3. How should I go about learning JavaScript/AJAX? To what extent can Python substitute for these, if any? Either way, books/tutorials?

posted by randomname25 to Computers & Internet (29 answers total) 33 users marked this as a favorite
Go and download Turbogears.
posted by sonic meat machine at 12:51 PM on September 26, 2010

Response by poster: That looks like an interesting framework. But right now, I'm not interested in developing at lightning speed--I want to understand exactly what I'm doing, and ideally not be dependent on any framework. Answers to old questions recommend that you know what you're doing in your language before you start taking shortcuts, and that's what I'm going for.
posted by randomname25 at 12:54 PM on September 26, 2010

Best answer: Oh, and because my first answer is too flip and silly:
  1. Most data structures and algorithms are language-agnostic. These are better than language-centric. Try Introduction to Algorithms.
  2. Try Turbogears. It's lower-level than django, and because it's a "best-of-breed" Frankenstein's monster of a framework, you'll learn more about how the underlying systems (SQLAlchemy, etc.) work.
  3. They're completely separate. JavaScript is run at the client level -- the user's browser. Python runs server-side. You want your Python web apps to serve JS, CSS, HTML, etc. to the user, whose browser will render the result. AJAX is a term for technologies used to interface between the user's client-side scripts and the server-side scripts. JavaScript tutorials exist in abundance, but honestly I'd just hack around with jQuery to begin with. You can do "fun" things with it, and it will make the process more interesting.

posted by sonic meat machine at 12:55 PM on September 26, 2010 [1 favorite]

After the author's reply: understanding exactly what you're doing when you're working on the web is a lot of work. You need to understand a ton of technical details which will not help you program more efficiently. If you want more bare-bones than Turbogears, you can try Pylons, which doesn't hold your hand at all... but even Pylons, at least, abstracts the WSGI backbone.

If you want to learn to write web apps "from scratch," well, you're headed down a different road. Never be afraid to use libraries – "not invented here" syndrome is a real nightmare when programming practically.
posted by sonic meat machine at 12:58 PM on September 26, 2010 [1 favorite]

If you like python, Django is the obvious choice if you want to actually, you know, build stuff.
posted by DarlingBri at 1:01 PM on September 26, 2010

Response by poster: Thanks, sonic meat machine. Your Introduction to Algorithms link's not working, though.

I'm not opposed to using libraries or code that other people have written. Someone at work told me about Joomla, which doesn't require that you program anything. I guess I just want to avoid things like that.
posted by randomname25 at 1:05 PM on September 26, 2010

Response by poster: I was thinking no-framework because of answers to this question. Was the advice there misguided? Or just more applicable to PHP/MySQL than to Python?
posted by randomname25 at 1:08 PM on September 26, 2010

I think this is Algorithms book he means, it was the textbook on my (and, I assume, many other's) CS degree.
posted by robertc at 1:17 PM on September 26, 2010

Best answer: Wow. Web app developers who can't make a link work are the best kind. Try this.

DarlingBri, Django is nice but it makes some interesting decisions, not least of which is the use of a "custom" database. SQL + SQLAlchemy will be more generally applicable.

randomname25: PHP hides the "back-end" stuff, so going without a framework is more doable there. Using CGI libraries and the like is going back to the bad old ways, to say the least. I think it's best to use tg2 or Pylons, which will basically just give you a web interface and step out of the way. Ruby on Rails may be limiting/"on rails," but Pylons... is not.
posted by sonic meat machine at 1:19 PM on September 26, 2010

For the web front end stuff I would recommend:
posted by robertc at 1:22 PM on September 26, 2010

Response by poster: Reviews for that book say it's pretty mistake-ridden, and the lack of answers make it inappropriate unless you're taking a class (and I'm not). Any other recommendations?
posted by randomname25 at 1:22 PM on September 26, 2010

Response by poster: and by "that book" I meant introduction to algorithms
posted by randomname25 at 1:23 PM on September 26, 2010

Best answer: I found Algorithms in a Nutshell fairly readable.
posted by robertc at 1:25 PM on September 26, 2010

I saw that question too as I'm trying to pick a framework to use for my web app. I gave Rails and Django a try, but didn't like them. Too much magic and convention for my liking. I just found it confusing and frustrating. Even with Turbogears 2 and Pylons, I just didn't like them. I wanted something simpler.

Check out Flask. It's a Python microframework. It makes it easy to get started making a web app. I prefer it to the aforementioned frameworks because of its simplicity. You only add the libraries that you want.

If you really wanted, you could look through the code and see what's going on behind the scenes pretty easily, though I'm not sure you'd gain much by doing that. Using any of these frameworks, you'll still be programming. It's just that you'll be focusing on your app as opposed to reinventing the wheel.
posted by aloysius on the mixing boards at 1:27 PM on September 26, 2010

Best answer: If you have a conversational (as opposed to fluent) understanding of Python, Django will let you put your good foundation to work quickly with an interesting learning curve. I'm guessing you have the basic programming vocabulary it assumes, unlike the poster in that thread.

My advice in that thread would have been similar: do not pick up, say, Cake until you have a basic conversational grounding in PHP. Fluency is not required; it's in Python and you need to use Python but you can learn as you go.

Or, you know, just go build something in Python if that's what you want to do.
posted by DarlingBri at 1:33 PM on September 26, 2010

Best answer: Seconding Django, it's a lovely and well-maintained framework. Run through the tutorial to get a feel of it.

Someone at work told me about Joomla, which doesn't require that you program anything. I guess I just want to avoid things like that.

Yeah, no, it's nothing like that. It's pure python besides the templating language. You really don't want to build a web app from scratch, trust me. Not as a starter project.
posted by cj_ at 1:44 PM on September 26, 2010

Response by poster: Thanks for clearing up the framework issue, everyone. I'll investigate those further but Django looks good.

So I know that when you're using PHP, it can't communicate directly with the server and you need a go-between like MySQL. Is that the case with Python? Or can Python+whatever framework communicate directly with your web server?
posted by randomname25 at 1:47 PM on September 26, 2010

Response by poster: I see that MySQL is a database management system. Is that just something that organizes stuff on your web server? I realize these are probably obvious noob questions but I'm confused...
posted by randomname25 at 1:48 PM on September 26, 2010

Best answer: That's a slightly confused question, I will try to untangle it for you without bogging you down with MVC definitions.

There are a few components to a web application. You will need a relational database to store stuff. Django (and probably any other framework) can transparently use postgresql, MySQL, sqlite, or Oracle. These all do the same thing with different pros/cons I won't get into here, but yes you will need at least one of these. SQLite is probably installed on your system by default and requires no configuration, so you can use that to get started. It's unsuitable for serving on the web, though.

You need to serve your content with something that speaks HTTP protocol to browsers. This will end up being Apache + mod_wsgi most likely (there are some other alternatives), but you can run a simple server locally for development.

The tutorial covers all of this BTW. Django documentation is pretty good.
posted by cj_ at 1:53 PM on September 26, 2010

MySQL is where you put all your data. If your site is just static html pages, you don't need a framework, you would just write the html. Presumably it will have dynamic data of some kind. That's where it's stored.
posted by cj_ at 1:55 PM on September 26, 2010

So honestly at this point I'd stop with the reading for a bit. Figure out what you want to build, and start coding. You most likely will want to work with a framework like Django which seems to be the defacto standard from what I've seen and has a very large and active community. So getting a book on that would be key.

By starting a project you'll soon see where your lack of knowledge is and you can backfill with further study.
posted by bitdamaged at 2:11 PM on September 26, 2010

Best answer: I'm assuming you want to have a dynamic site that changes it's content based on the conditions you visit it. You have multiple layers to make this work.

For example, let's use this ask-metafilter page.

The page itself is HTML, the browser does not know or care that the comments and timestamps may be different each time you load this page.

When the browser requests this page from the webserver, the webserver is running some kind of interactive backend, like ASP, PHP, Ruby, Python, etc, etc. This is what handles any logic or conditional branching.

The Python would get the actual DATA (how many comments are there, what does each comment contain, who wrote it) from a database... The most common database is MySQL, and most other database systems use a very similar syntax and structure.

Javascript is usually a client side only thing. AJAX is a fancy way of using Javascript to get data to or from the server (and thus the Python code) without having to make the user refresh the whole browser.. A great example is gmail. It can load the message body without having to reload the entire page layout and toolbars.

While i understand you want to get things from a low-level perspective before you build on that, you should know that Python itself is pretty far removed from the real low level operations behind a web server. A framework such as Django will handle most of the housekeeping for you, allowing you to work faster, easier and more importantly, you won't have to pull your hair out trying to debug some little detail of the site's operation.

Your knowledge of Python will allow you to get down into the guts of a framework and understand what's going on, however if you really need to or want to.

(I'm bad at explaining stuff)
posted by kzin602 at 2:19 PM on September 26, 2010 [5 favorites]

Response by poster: kzin602, that explains so much! Thank you!

I think I'm going to try to start coding in Django. If people have more book recommendations/whatever for Django/stuff in my original question, feel free to put them here. I'm going to be checking this thread for a while. But I'm very happy with all the answers here so far.
posted by randomname25 at 2:33 PM on September 26, 2010

Best answer: The Definitive Guide to Django is in extremely heavy rotation as bathroom reading at our house. It is the book my brainbox of a husband selected after very careful consideration and comparison and at considerable expense (we had to import it) when he needed to re-build a Django app with no Python background at all. They have been happily married for more than a year now.

Do the Django tutorial while you're waiting for your book to arrive. Then read through the examples in the book. Then look at these very handy Django apps you can use when building your app. One reason widely supported frameworks are nice is that virtually every app built will need, say, a login and/or a full text search and/or a contact form. There is no need to invent that wheel every time when someone has already done it for you and kindly shared their code so you can bring those services online in under 60 seconds. Additionally, should you require something more sophisticated like I dunno geospatial map stuff, Django has modules you can work with - for pretty much everything, "there's an app for that" and you can mung them together in very tailored ways.

You may consider that cheating but those elements are icing on the cake of the app you want to build. When you go through the work of building what's unique to your app in terms of functionality, slotting those elements in will seem like a nicely earned bonus. There will be blood and sweat and tears here regardless.

Because it was not germane to your original question, this has not been mentioned but you should know: one glorious thing about Django is that as you build the front end, it builds the back end for you. Build a module, and the admin side gets built automatically. Niiiiiice.
posted by DarlingBri at 3:13 PM on September 26, 2010

I agree that Django is a good next step. I'll also agree that, while I completely appreciate your desire to understand every moving piece of your application, there's a lot of moving pieces. (no one's even mentioned all the details of getting Apache set up yet ...) Django will get you going on all of this while insulating you from some of the less-fun details.

In fact, you might even consider Google's AppEngine as a transitional step. It uses Python, and pretty much Django (with a few minor differences) but provides the storage and server for you. Plus you can hook into all of Google's services easily. Then, once you're comfortable with that, set up Django and start learning about databases, configuring servers, and so on.

Also, my go-to recommendation for Python books is Dive Into Python. Readable and fun (and free!), with lots of practical things to do.

I'd hold off on Introduction to Algorithms for now; it's an outstanding book, and one that every computer scientist should own, but it might be a bit too "jumping into the deep end" for you. You can definitely learn data structures and algorithms in Python; here's one book. Finally, if you like solving puzzles and want an excuse to learn all the Python modules, try the Python Challenge.
posted by chbrooks at 3:37 PM on September 26, 2010

Best answer: Hey, Random, since you aren't super-clear on what mysql is, I think SQL really should be your next stop on the learning train. The reason for this is that (bad) decisions made in the database design are hard to revise if you've already built everything that goes on top of it.

A classic example: say you have a table of people such that one of the columns is "phone_number" and you then proceed to build an entire database-driven web site that uses that table. Then you run into somebody who has two phones! D'oh, forgot to consider that possibility. You change the table by adding a place for a second phone number, and then you change all the code that deals with phone numbers, throughout the whole web site. Everything works again, for now.

Along comes another person who wants to supply a work phone. And a couple weeks later, another whose work phone is shared among several people who work in that office, more than one of which is in your table. Each time you change the table, you're having to change the way the web app uses the table; when you start to want to store the phone numbers in a completely separate table and reference the people table from it, the work to change the web app to the new structure is pretty hairy. Not only that, but it's kind of complicated to pull the data out of the old table and make sure it gets into the new phone_numbers table without losing the information about who is using which phone number, in which manner. Along the way, your web app starts to get really slow until you stumble into a more efficient way to write the query it uses to fetch the data.

Gradually, it becomes clear to you could have built several new web sites with the effort it took you to continually adapt this one to your changing understanding of the data it was trying to model. You realize you should have learned about databases in the first place, and built a smarter database to begin with - so you go shopping for a book on SQL and database design. You learn that it doesn't much matter which book you get, because SQL and relational databases have been around since the 1970's and haven't changed that much over the years. So you go check out [$relational_database_book] from the local library and learn all the SQL stuff you should have known at the outset, which turns out to your relief to not be that complicated.

Basically, I'm suggesting a short cut right to that last part, without all the pain that precedes it. I learned from an ancient WROX Press book that came with a 3.5" floppy with some one-off RDBMS on it, and I now do this stuff for a living.
posted by richyoung at 4:01 PM on September 26, 2010 [7 favorites]

Best answer: Seconding richyoung.

Learn how databases work and come up with a good way to store data for any projects you start on, because you’ll tear your hair out trying to rewrite your codebase if you don’t and later realize that some piece of it doesn’t match up with what you actually need (or, if you get lucky, doesn’t scale past your first 10,000 users).

(I’m currently employed at a social-media firm doing exactly the kind of rewrite I speak of — rearchitecting key pieces of the database schema and the store/controller code that turns database rows into runtime objects.)

And my $0.02 on the algorithms-and-data-structures thing: learn them now.
If nothing else, learn the basics. Learn what big-O notation is and why an O(n log n) algorithm will blow the doors off an O(n^2) algorithm for any real-life dataset; learn about trees, graphs, binary search trees, and hashtables, and when to use each of them; and learn how to sort and search efficiently. If you know the standard problems and the standard tools for dealing with them, you’ll avoid 95% of the “this doesn’t run fast enough” headaches.

Any data structures book worth its salt will include enough on algorithms and basic complexity theory to let you postpone reading a real algorithms book until later.

Oh, and learn what an NP-complete problem is and how to tell when you’re dealing with one. Because those are entirely different set of headaches for which we have No Good Solution.
posted by spitefulcrow at 4:16 PM on September 26, 2010 [1 favorite]

What richyoung is saying is important, but it's interesting in another way: it illustrates the fragility of requirements in the early phases of app development. Not saying 'my approach' is the 'best approach', but it seems a lot of experienced architects/developers have been postponing development of the data store until later in the development phases. As a platform for rapidly prototyping concepts, django/sqlite is very good.

Here's a general workflow, but no where near accurate to reality:

Step 1: I have a 'great' idea, so I whip together a django/sqlite app and i start playing around with the 'domain model' and some views. I introduce the concept to a few other co-workers (early adopters), and we begin experimenting. The domain model changes alter the physical storage a lot in this phase (e.g. phone numbers), therefore it's not worth the effort to continuously refine the physical storage.

Step 2: The concepts are beginning to flesh out, and the app is becoming more full featured. I introduce it to a few more people, or maybe a couple different groups of stakeholders. These groups are somewhat tolerant of risk and expect things to fail once in a while. Lots of UX flows are changing. However, I want to start improving the concurrency and performance and long-term data management, so I migrate from django/sqlite to django/mysql and put it on a production server.

Step 3: The 'domain model' is getting pretty solid at this point, so I put the server on a production box and expand it out to a set of 'beta testers'. Beta testers have a lot less tolerance to failures, and they expect moderately-high quality. The beta users will be extremely pissed of data integrity issues. Also, here is where performance and scalability (and avoiding deadlocks and ensuring concurrency) matter. So, at this point I begin converting some of the ORM storage layer into real stored procedures. I also better understand which data is static, which data is partially-static, and which data is completely dynamic. I may introduce caching servers like memcached. I may introduce some file storage schemes. I may introduce real relational tables for some data.

Step 4: I begin to optimize. At this point the 'domain model' is stable enough that a real data storage layer can be introduced. The domain model is mapped into a relational storage layer, usually, with well-defined stored procedures. Some data still stays in memcache, some data stays on the file system. However, I have to be aware of server farm schemes. At this point, django is mostly gone as a full-framework. I'm mainly using it for it's url handlers and for the template system. Actually, the template system may be gone too, in favor of something that's more low-level HTTP.


Django/sqlite is great to prototype concepts with a few users. But, as you scale, you will switch to something else, which may be something you write yourself.
posted by TheOtherSide at 8:29 AM on September 27, 2010 [4 favorites]

Hate to nitpick, but you can't do concurrency with sqlite. You will need to choose a real relational database when you go live. As a veteran of both mysql and postgresql, I recommend the latter. If you have a bunch of money and a DBA, consider Oracle. Otherwise, don't.
posted by cj_ at 2:59 AM on September 29, 2010

« Older Looking for a cheap Austin barber who's good with...   |   What kind of therapy do I need? Newer »
This thread is closed to new comments.