Python and Math

April 14, 2009 3:22 PM

Is it worth learning Python for combinatorial mathematics research?

I need to (re-)learn a programming language for some future research I plan on doing in combinatorics and discrete mathematics, and the elegance of Python is drawing me towards it.

How much efficiency/speed would I lose by not using a lower level language, like C++?

I know this question is very vague, but I do not know exactly what type of problems I plan on tackling. I just want to know if Python can efficiently handle heavy data.

I've read about the excellent math packages for Python, and Sage looks very neat.

Any (discrete) mathematicians out there use Python on a regular basis? Am I better off re-learning C++? (I'd rather not!)

posted by alligatorman to computers & internet (27 comments total)

You would lose a bit of runtime efficiency. But do you plan on spending more time running your programs or programming them? You're likely to save more time overall by learning Python if you're not talking about ridiculously long runtimes where every optimization counts.
posted by zsazsa at 3:24 PM on April 14 [2 favorites has favorites]

Can python efficiently handle heavy data? No. There are extensions that use plugins written in C to massage data and do the heavy lifting, with the higher level logic all done in pythong (numpy comes to mind). Mind you, for REAL heavy data, people use C, with ffi to do the heavy lifting in fortran and assembly because C is too slow, not to mention C++.

Of course heavy is a relative term. Python with or without the numpy extensions may be fast enough, and as said above, if you expect to spend more time writing programs than waiting for them to stop running, python will most likely save you time.
posted by idiopath at 3:34 PM on April 14

In my experience, C is about an order of magnitude faster than Python for basic numerical algorithms. On top of that, Python can't do true multi-threading because of the global interpreter lock.

That said, I'd still go with Python, for the reasons zsazsa mentions. But you might also want to shop around... have you considered OCaml or Scala?
posted by qxntpqbbbqxl at 3:34 PM on April 14

I do scientific programming with Python on datasets of hundreds of gigabytes. The nice thing about Python for this application is that it is so easy to extend it with a fast algorithm written in a lower-level algorithm, either your own, or some of the existing fast methods in NumPy.
posted by grouse at 3:34 PM on April 14 [2 favorites has favorites]

I hit post too soon: with combinatorics, you will quickly find out that python is a RAM hog, and almost completely unoptimized. Numpy is designed for exactly this sort of situation though, and I would definitely check it out.
posted by idiopath at 3:36 PM on April 14

On lack of preview, seconding the Ocaml suggestion above, ocaml has some of the main advantages of python (object oriented, whitespace aware, garbage collected), while having last I heard one of the best optimizing compilers for a garbage collected object oriented language. And it also has bindings to the same assembly/fortran libraries that c/c++ folks use (atlas, blas, etc.) if you want to get into some real heavy numerics.
posted by idiopath at 3:43 PM on April 14

Uh, ocaml isn't white space aware the way python is. No tabbing senetivity. About the most I know of is when you end a function without needing a ;;
posted by pwnguin at 3:51 PM on April 14

In general, you should optimize where you need it, rather than in advance.

So, when you get the problems you're going to work on, start with SPSS or Matlab or what have you. If that's not powerful or fast enough, try Python. If that's not enough, go for C.

And if possible, just optimize the parts you need to optimize.
posted by ignignokt at 4:01 PM on April 14 [1 favorite has favorites]

Use Python until it's too slow.

It's not really 'learning', anyway. As they say, "Python is executable pseudocode. Perl is executable line noise."
posted by katrielalex at 4:16 PM on April 14 [1 favorite has favorites]

You can be effective in Python while dedicating relatively little grey matter to it. C++ will demand huge regions of your brain just to manipulate strings and arrays. Use Python until you decide it isn't cutting it.
posted by chairface at 4:48 PM on April 14

pwnguin:

While the Ocaml usage of whitespace is not identical to that of python, it is normal to use indentation to define blocks, rather than brackets.

I just downloaded a random ocaml program, here is a chunk of source code:

let mlpost ?(pdf = pdf) ?file f =
  let file = match file with
    | None -> next_name ()
    | Some file -> file
  in
  let ext = if pdf then ".mps" else ".1" in
  Mlpost.Metapost.emit file f;
  Latex.includegraphics (Latex.text (file ^ ext))

notice that all the blocks are defined by indentation.
posted by idiopath at 5:02 PM on April 14

In Python indentation is significant. In Ocaml indentation is not significant but is governed by custom. That chunk of Ocaml code rewritten as one long line would mean the same thing as it means the way you posted it.
posted by rdr at 5:16 PM on April 14

OK, I misremembered. I have not actually used OCaml in the last couple of years. I was wrong about the whitespace issue, but the larger point that the code is relatively clutter free, with fewer extraneous brackets and special symbols, may still be a factor in OCaml's favor.
posted by idiopath at 5:31 PM on April 14

Modern C++ using the STL and boost is a lot better than it used to be.

That said, python is extendable as everyone has said, using C/C++ when needed.

It may not fit your requirements particularly well, but the multi language shoot out has python, C++ and various others compared on a number of benchmarks that may give you a better idea of how various languages compare.

Can you ask the people that you'll be working with what they are using?
posted by sien at 6:32 PM on April 14

Functional Programming
posted by phrontist at 6:38 PM on April 14

Python optimizes developer time, C optimizes processor time.
Developer time is at least an order of magnitude more valuable than processor time.
posted by signal at 6:45 PM on April 14

My linear algebra professor uses C and Matlab/Octave and is currently trying to switch to Haskell. I've also heard of Boost (C++) and PDL (Perl + C).
posted by shmooly at 6:55 PM on April 14

C++ is for people who get paid by the hour to write code.

Python is for everyone else.

If you get paid by the hour, and all you have to do to get paid is keep writing code, then learn C++.

Otherwise, learn Python.
posted by koeselitz at 7:36 PM on April 14

For linear algebra type things with big arrays, NumPy should be fast enough. If you start getting speed issues for larger data sizes, look into Pyro or another cluster computing solution.
posted by demiurge at 7:49 PM on April 14

Yeah, I highly recommend Python + numpy/scipy. If you learn to use it well, it will likely be fast enough. And you can always profile your code and rewrite the CPU-hogging sections in C if speed is a problem and numpy can't help you.

Python is a fantastic language for doing complicated things simply and quickly. It is not nearly as fast as C, but it takes much, much, much, much less time to write and can typically be made "fast enough." (Also, a program that works slowly is generally better than a program that doesn't work, but does it quickly.) Plus, as mentioned above, for many scientific-type applications numpy and scipy make it possible to write code that is almost as fast as C because the heavy lifting is done in numpy/scipy, which are written in C.

If you learn Python, stick with 2.5 for now; the major third-party packages haven't yet been updated for 3.0 (and many haven't been updated for 2.6).
posted by musicinmybrain at 8:07 PM on April 14

Thanks everyone.

I think I'm going to do what most suggested, and use Python until it's too slow for whatever I need to do. If I have to, I can crank out some C.
posted by alligatorman at 8:24 PM on April 14

I'd first think about the problems you'll spend your time addressing. If your doing numerical work, you've hundreds of packages for various languages. If your doing number theory, your choices are still quite numerous. If your doing group theory, you'll surely use GAP or Magma.

I see several informative pages about combinatorics software :
http://www.dmoz.org/Science/Math/Combinatorics/Software/
http://www.mat.univie.ac.at/~slc/divers/software.html
http://www.google.com/Top/Science/Math/Combinatorics/Software/

I suggest that you read about Haskell until you understand how algebraic data types work, especially for combinatorics. You should also ask potential coauthors for suggestions.
posted by jeffburdges at 8:49 PM on April 14 [1 favorite has favorites]

Not sure if its appropriate for your needs, but you may find Cython useful. An example.
posted by MetaMonkey at 8:50 PM on April 14

I don't know Python, but recommend Perl and PDL and Inline modules. The Inline modules let you write C/Octave/etc. code directly in your program. So you can easily find the one subroutine that takes the most time and rewrite it in C or whatever within the script itself.
posted by zengargoyle at 10:10 PM on April 14 [1 favorite has favorites]

I'm working with another mathematician on a combinatorial project, and he's big on Haskell.
posted by leahwrenn at 1:02 PM on April 15

I think you should look at F# and Scala.
posted by arimathea at 4:25 PM on April 15

If you're worried about performance, look at this comparison of different speedups.
posted by PueExMachina at 9:10 PM on April 15 [1 favorite has favorites]