Advice in designing a web application for a self-taught developer.
July 11, 2006 11:20 PM Subscribe
I'm going to write a web application that, if it is successful, will need to be "enterprise-level" -- that is, robust, secure, and extensible. Given the limitations of my current skills, what can I do to minimize design errors? Lots more inside.
Here's the deal: I'm writing a web application with a partner that is essentially a tracking and accounting program. My CS skills are primarily self-taught (I've only had the very basics of a CS education in college before I dropped out of the program), and I realize that this will mean that the design of the application will be less than perfect. The project is no-budget, so hiring experienced programmers and designers is not an option at the moment, however, in the future, I'd like to be able to bring professionals in to extend the code.
Simply, I want to build a functional proof-of-concept that will have as much reusable code as possible.
With that in mind, here are the specific questions I have in mind:
1. Is writing such a project in PHP a foolish undertaking? PHP is currently the language I know best. I don't anticipate us needing much more power than what the language is capable of -- since it's mostly storing form data in a DB and recovering it -- but I'm concerned because I've seen other potentially successful web apps fall victim to their own popularity. I'm mainly thinking of Friendster in this case, and how seemed to fall over under the weight of its own codebase (and it's lengthy rewrite), and I want to avoid this. Is it worth the potential code mistakes I'll make writing this in something that other people consider a more "proper" language but with which I am not familar-- like Python, for example? I'm planning to use MySQL as a backend because that's what I'm most familar with (and I don't think we will need more power).
2. Should I consider using existing libraries? For example, in a lot of the work I've done already, I've avoided using stuff in the PEAR repository because it doesn't meet the specific requirements I have. Is this a bad way of doing things? Should I instead base modules on existing libraries and extending them where they do not meet my needs? In short, is hand-rolling everything a mistake?
3. Can you recommend some good books that will help me avoid stupid design mistakes? I've already read a lot of the Joel on Software stuff (which seems more about software PM), and I don't really think I need to read about process (or do I?). Here's what I'm planning to do: get up a barebones proto as fast as possible and have some of the audience who's going to be working with this software evaluate it and give us advice for future iterations. The spec is going to be developed by the person I'm working with who is familar with the industry, so my knowledge of the "what" that the app needs to do is taken care of, but I'll be highly involved in the "how" -- that is, developing a good and very simple (essential for our product) UI.
4. What do I look to for security best practice? I want to take all precautions possible to make sure that the private data remains private. Backup, etc, I'll consider an IT problem, but I want to make sure that do things that are difficult to change later (like user authentication) right there first time. What's the best way to make sure passwords are encrypted in the DB and that cookies are difficult to spoof? How do I make sure my authentication routines are secure AND not sluggish?
5.Besides good documentation and modular structure, what's the best way to make sure that other people can work on code that I've (possibly badly) developed? I don't know if it'll be possible or smart to throw the codebase away after the prototype is completed -- if you think otherwise, please let me know. Is it ever a good idea to just write something as a prototype (that is destined to be thrown away)?
6. Everything is going to live on one server -- should I plan for an eventual distribution of server duties? Is this something a developer even needs to worry about? I've really never had to deal with this before, so I can't elaborate on this question.
Thanks, I'm sure I'll think of tons of questions later as answers come in. Unfortuntely, I can't really entertain any ideas of just having someone else do it because that would pretty much eliminate my participation in this project (which I don't want to do). It's not realistic either that I'll be able to somehow get enough education to meet the skillset of someone with a B.S. either, so although I appreciate responses advising that I gain some more academic experience before undertaking this project, they aren't useful for my current situation.
Anecdotal experience involving your own web application development (and business ventures!) is welcome.
Here's the deal: I'm writing a web application with a partner that is essentially a tracking and accounting program. My CS skills are primarily self-taught (I've only had the very basics of a CS education in college before I dropped out of the program), and I realize that this will mean that the design of the application will be less than perfect. The project is no-budget, so hiring experienced programmers and designers is not an option at the moment, however, in the future, I'd like to be able to bring professionals in to extend the code.
Simply, I want to build a functional proof-of-concept that will have as much reusable code as possible.
With that in mind, here are the specific questions I have in mind:
1. Is writing such a project in PHP a foolish undertaking? PHP is currently the language I know best. I don't anticipate us needing much more power than what the language is capable of -- since it's mostly storing form data in a DB and recovering it -- but I'm concerned because I've seen other potentially successful web apps fall victim to their own popularity. I'm mainly thinking of Friendster in this case, and how seemed to fall over under the weight of its own codebase (and it's lengthy rewrite), and I want to avoid this. Is it worth the potential code mistakes I'll make writing this in something that other people consider a more "proper" language but with which I am not familar-- like Python, for example? I'm planning to use MySQL as a backend because that's what I'm most familar with (and I don't think we will need more power).
2. Should I consider using existing libraries? For example, in a lot of the work I've done already, I've avoided using stuff in the PEAR repository because it doesn't meet the specific requirements I have. Is this a bad way of doing things? Should I instead base modules on existing libraries and extending them where they do not meet my needs? In short, is hand-rolling everything a mistake?
3. Can you recommend some good books that will help me avoid stupid design mistakes? I've already read a lot of the Joel on Software stuff (which seems more about software PM), and I don't really think I need to read about process (or do I?). Here's what I'm planning to do: get up a barebones proto as fast as possible and have some of the audience who's going to be working with this software evaluate it and give us advice for future iterations. The spec is going to be developed by the person I'm working with who is familar with the industry, so my knowledge of the "what" that the app needs to do is taken care of, but I'll be highly involved in the "how" -- that is, developing a good and very simple (essential for our product) UI.
4. What do I look to for security best practice? I want to take all precautions possible to make sure that the private data remains private. Backup, etc, I'll consider an IT problem, but I want to make sure that do things that are difficult to change later (like user authentication) right there first time. What's the best way to make sure passwords are encrypted in the DB and that cookies are difficult to spoof? How do I make sure my authentication routines are secure AND not sluggish?
5.Besides good documentation and modular structure, what's the best way to make sure that other people can work on code that I've (possibly badly) developed? I don't know if it'll be possible or smart to throw the codebase away after the prototype is completed -- if you think otherwise, please let me know. Is it ever a good idea to just write something as a prototype (that is destined to be thrown away)?
6. Everything is going to live on one server -- should I plan for an eventual distribution of server duties? Is this something a developer even needs to worry about? I've really never had to deal with this before, so I can't elaborate on this question.
Thanks, I'm sure I'll think of tons of questions later as answers come in. Unfortuntely, I can't really entertain any ideas of just having someone else do it because that would pretty much eliminate my participation in this project (which I don't want to do). It's not realistic either that I'll be able to somehow get enough education to meet the skillset of someone with a B.S. either, so although I appreciate responses advising that I gain some more academic experience before undertaking this project, they aren't useful for my current situation.
Anecdotal experience involving your own web application development (and business ventures!) is welcome.
1. There's no reason why PHP won't build and scale for your needs. Use PHP 5 and make things class-based and you'll find things are quite modular and capable. I built a rebate management system that's handling 4 M records in MySQL and hundreds of thousands of hits on one machine in LAMP, so bleh. Others will disagree. Or suggest Ruby on Rails.
2. That's up to you. We didn't, but we had the time to build them ourselves. Dealing with the potentially known flaws in other people's elements versus flaws you'll create in your own is an interesting experience, if for no other reason than because "fixing" their flaws will break the system when you go to upgrade next.
3. No. Not off hand. Okay, I take that back. For MySQL at least: High Performance MySQL is brilliant.
4. MD5 is "fine enough" for password encrypting. Others will disagree and suggest seeded hashes using more complex algorithms. They should get over themselves. Store all user passwords as MD5 hashes, and build a password reset system that allows a user to initiate a reset request by entering their email address and perhaps another piece of information. (We use last name + email address.)
Generate a password reset hash that links a random hash to their user id. Send them an email with that has as the argument. (This prevents them from incrementing change_user_password.php?user_id=4 or something as idiotic.) Since you're storing password reset requests in the db, you can also track their generation time and allow them to time out if the user doesn't click them soon enough.
Use a similar idea for cookies. Don't store user ids that will migrate into a session variable in the cookie, since cookies are easily modifed by the end user. Instead, hash them. (Hashing is fun. Others will disagree. That's their way.) Hash a user id, but tack on another piece of information so that users can't just hash any random serial user id in MD5 and change their cookie to that.
5. Seriously, good documentation, good in-code commenting and very modular structure. Try to migrate to classes. They're a bit confusing if you're used to more functional approaches, but they're amazing at their flexibility and they make things *truly* modular.
I can definitely see something being written as a prototype to be tossed. In that case, if you want to go that route, you should definitely make sure you've planned your database out to a t. Spend time with that, since it'll be the hardest part. Also, make sure you document all the use-cases and internal processes scripts and other elements will run through for each aspect of the site.
In short, I think there may be some cases, but if you code with that mindset, you're probably asking for trouble. Especially in PHP.
6. This isn't nearly as big an issue as you'd think. Most of the distribution will be handled by load balancers and by MySQL replication and such. These aren't things that are typically fantastically optimized on the code level, though you'll get down to that level at some point.
posted by disillusioned at 11:41 PM on July 11, 2006 [2 favorites]
2. That's up to you. We didn't, but we had the time to build them ourselves. Dealing with the potentially known flaws in other people's elements versus flaws you'll create in your own is an interesting experience, if for no other reason than because "fixing" their flaws will break the system when you go to upgrade next.
3. No. Not off hand. Okay, I take that back. For MySQL at least: High Performance MySQL is brilliant.
4. MD5 is "fine enough" for password encrypting. Others will disagree and suggest seeded hashes using more complex algorithms. They should get over themselves. Store all user passwords as MD5 hashes, and build a password reset system that allows a user to initiate a reset request by entering their email address and perhaps another piece of information. (We use last name + email address.)
Generate a password reset hash that links a random hash to their user id. Send them an email with that has as the argument. (This prevents them from incrementing change_user_password.php?user_id=4 or something as idiotic.) Since you're storing password reset requests in the db, you can also track their generation time and allow them to time out if the user doesn't click them soon enough.
Use a similar idea for cookies. Don't store user ids that will migrate into a session variable in the cookie, since cookies are easily modifed by the end user. Instead, hash them. (Hashing is fun. Others will disagree. That's their way.) Hash a user id, but tack on another piece of information so that users can't just hash any random serial user id in MD5 and change their cookie to that.
5. Seriously, good documentation, good in-code commenting and very modular structure. Try to migrate to classes. They're a bit confusing if you're used to more functional approaches, but they're amazing at their flexibility and they make things *truly* modular.
I can definitely see something being written as a prototype to be tossed. In that case, if you want to go that route, you should definitely make sure you've planned your database out to a t. Spend time with that, since it'll be the hardest part. Also, make sure you document all the use-cases and internal processes scripts and other elements will run through for each aspect of the site.
In short, I think there may be some cases, but if you code with that mindset, you're probably asking for trouble. Especially in PHP.
6. This isn't nearly as big an issue as you'd think. Most of the distribution will be handled by load balancers and by MySQL replication and such. These aren't things that are typically fantastically optimized on the code level, though you'll get down to that level at some point.
posted by disillusioned at 11:41 PM on July 11, 2006 [2 favorites]
Using an n-tier model will cover most of the possible complications for you. Personally, I wouldn't do it in PHP- I think PHP is a collection of hacks built on top of eachother. Its fine for a front-end to a larger system, but on its own... bleh. Ultimately, what language you choose is largely irrelevant if you can write the application well. PHP is easier than most to write poorly in (see also: perl).
Nitpick: MD5 isn't encryption. Yes, hashing is good and generally won't slow anything down. MD5 is fine as a hash for any website.
Watch out for SQL injection holes. This is the most common type of bug in PHP scripts.
Read up on "Not Invented Here Syndrome." This is a bad thing, despite what Joel says (although he does have a point). You should be using any library that fits your needs. If it doesn't, you should evaluate how long it would take to make it fit your needs, and weigh that against how long it would take you to write it yourself. Chances are, if it doesn't immediately fit your needs, you're better off writing it yourself.
If this were C++, I'd recommend Code Complete (steve mcconnell. It still might be useful for you. I don't know.
Making a prototype and throwing it away is mostly useful for very complicated systems. You probably don't want to do this, unless you're very new to writing large programs. You probably just want to plan out everything & go. Make SURE you nail the DB schema down early. Think of every query you'll need to make, and think of how the relationships work. Think of where the indexes go. If there will be full-text-search indexing, make sure you structure your DB to accommodate the requirements of its FTS system.
PHP does have some OO. Use it. However, don't let yourself get wrapped up in making everything an object. I don't know if PHP supports enough OO to make using design patterns possible, but if so, it might be worth reading up on them (Gamma, et al is the standard DP book, although there may be something more appropriate for you/PHP).
Finally... a lack of education is not remotely a problem. CS education is important for algorithm development and hardware interaction. Business application development is experience all the way. It doesn't need to be fast, so don't spend too much time optimizing ("premature optimization is the root of all evil"). Don't worry about the HTML interface at first - thats what a web designer and CSS is for. Just get the content out there at first.
I'd be glad to help with anything that comes up during development. My emails in my profile.
posted by devilsbrigade at 12:08 AM on July 12, 2006
Nitpick: MD5 isn't encryption. Yes, hashing is good and generally won't slow anything down. MD5 is fine as a hash for any website.
Watch out for SQL injection holes. This is the most common type of bug in PHP scripts.
Read up on "Not Invented Here Syndrome." This is a bad thing, despite what Joel says (although he does have a point). You should be using any library that fits your needs. If it doesn't, you should evaluate how long it would take to make it fit your needs, and weigh that against how long it would take you to write it yourself. Chances are, if it doesn't immediately fit your needs, you're better off writing it yourself.
If this were C++, I'd recommend Code Complete (steve mcconnell. It still might be useful for you. I don't know.
Making a prototype and throwing it away is mostly useful for very complicated systems. You probably don't want to do this, unless you're very new to writing large programs. You probably just want to plan out everything & go. Make SURE you nail the DB schema down early. Think of every query you'll need to make, and think of how the relationships work. Think of where the indexes go. If there will be full-text-search indexing, make sure you structure your DB to accommodate the requirements of its FTS system.
PHP does have some OO. Use it. However, don't let yourself get wrapped up in making everything an object. I don't know if PHP supports enough OO to make using design patterns possible, but if so, it might be worth reading up on them (Gamma, et al is the standard DP book, although there may be something more appropriate for you/PHP).
Finally... a lack of education is not remotely a problem. CS education is important for algorithm development and hardware interaction. Business application development is experience all the way. It doesn't need to be fast, so don't spend too much time optimizing ("premature optimization is the root of all evil"). Don't worry about the HTML interface at first - thats what a web designer and CSS is for. Just get the content out there at first.
I'd be glad to help with anything that comes up during development. My emails in my profile.
posted by devilsbrigade at 12:08 AM on July 12, 2006
do yourself a favour and learn a general purpose language with web framework goodness added. The learning curve might be a bit steep at first, but you'll be grateful for the extra power later. I recommend Catalyst. Interestingly you can use PHP::Interpreter to hook php stuff back into your catalyst application if you want to (for example) integrate legacy code.
PHP has several issues, which tend towards producing apps which are an unmaintainable mess.
BTW self taught dev talking here too.
posted by singingfish at 1:01 AM on July 12, 2006
PHP has several issues, which tend towards producing apps which are an unmaintainable mess.
BTW self taught dev talking here too.
posted by singingfish at 1:01 AM on July 12, 2006
PHP is fine. Many people dislike it, but its popular for a reason. That said, there's good and bad ways to write PHP, and for that reason you should definitely try to leverage third party libraries; the good ones, such as PEAR and Smarty, will encourage you to organize your code the way more experienced coders do, without having to figure those patterns out yourself via trial and error.
I don't keep up with PHP books (the last one I read was the PHP Bible several years ago, and it mediocre), but its generally hard to go wrong with anything from O'Reilly.
You should look in a book for comprehensive PHP security advice, but in general, validate all input from the user before doing anything with it, make sure that the user your code connects to the database as has the minimum db permissions necessary, and make regular, automated backups.
People "write one to throw away" all the time. It just depends on how much time and energy you have. If this is your first web app, you should probably expect to.
I wouldn't worry much about this. Splitting up onto different servers won't be hard; the simplest way is to put the DB on its own machine. Apache also has load balancer modules. Its not hard to do.
Don't worry about not having a CS degree; it probably wouldn't help much. The relevent CS advice I'd pass on is to not let your functions get too long, and don't use "magic numbers" (specific constants embedded deep inside your code).
Also, doing layout in CSS instead of tables takes a little longer, but is worth it. There's lots of sites offering guides on how to do 3 columns, or whatever. And (because I can't help myself) Dean Edwards' IE7 javascript library fucking rocks. If you expect that your users will be using non-ancient machines, use it; it can let you make your html much simpler.
posted by gsteff at 1:53 AM on July 12, 2006
I don't keep up with PHP books (the last one I read was the PHP Bible several years ago, and it mediocre), but its generally hard to go wrong with anything from O'Reilly.
You should look in a book for comprehensive PHP security advice, but in general, validate all input from the user before doing anything with it, make sure that the user your code connects to the database as has the minimum db permissions necessary, and make regular, automated backups.
People "write one to throw away" all the time. It just depends on how much time and energy you have. If this is your first web app, you should probably expect to.
I wouldn't worry much about this. Splitting up onto different servers won't be hard; the simplest way is to put the DB on its own machine. Apache also has load balancer modules. Its not hard to do.
Don't worry about not having a CS degree; it probably wouldn't help much. The relevent CS advice I'd pass on is to not let your functions get too long, and don't use "magic numbers" (specific constants embedded deep inside your code).
Also, doing layout in CSS instead of tables takes a little longer, but is worth it. There's lots of sites offering guides on how to do 3 columns, or whatever. And (because I can't help myself) Dean Edwards' IE7 javascript library fucking rocks. If you expect that your users will be using non-ancient machines, use it; it can let you make your html much simpler.
posted by gsteff at 1:53 AM on July 12, 2006
If you application relies on objects that are beyond trivial, if you database requires more than two or three levels of JOINS to coalesce... do not use PHP. PHP is excellent as a scripting language, but it is absolutely dreadful for relational object mapping. If all you're doing is simple queries, fine, use PHP, but for the love of God categorize your application heiarchy with an iron fist (it is far too easy for a PHP-based project to get out of hand otherwise).
To give a very simple example:
Your objects are Companies. Employees work for a Company. Employees have titles and roles (which may overlap). Companies have a heirarchy based on roles. Accounting has salaries based on titles. Employees also have addresses (some could share the same address).
I'd like to look at the address for all Vice Presidents for the Company 'IBM'.
There's your problem. With PHP, you're either hand-coding SELECT queries and custom-tailoring them to a particular page, or worse (far, far worse), you're basing query criteria on user input, constructing it through a bunch of if...then's and string concatenation. With proper relational mapping, you define the relations between objects and the corresponding tables, and that's it. The O/R system takes care of the rest.
So ideally you want a programming language that can handle this sort of thing (and much more complex objects as well) without requiring you to take care of all the plumbing. If you plan on hiring more competent programmers in the future, it should probably be in a language that is common enough for others to jump into, so it'll have to have a broad, existing base.
I think you see where I'm going with this.
posted by Civil_Disobedient at 4:28 AM on July 12, 2006
To give a very simple example:
Your objects are Companies. Employees work for a Company. Employees have titles and roles (which may overlap). Companies have a heirarchy based on roles. Accounting has salaries based on titles. Employees also have addresses (some could share the same address).
I'd like to look at the address for all Vice Presidents for the Company 'IBM'.
There's your problem. With PHP, you're either hand-coding SELECT queries and custom-tailoring them to a particular page, or worse (far, far worse), you're basing query criteria on user input, constructing it through a bunch of if...then's and string concatenation. With proper relational mapping, you define the relations between objects and the corresponding tables, and that's it. The O/R system takes care of the rest.
So ideally you want a programming language that can handle this sort of thing (and much more complex objects as well) without requiring you to take care of all the plumbing. If you plan on hiring more competent programmers in the future, it should probably be in a language that is common enough for others to jump into, so it'll have to have a broad, existing base.
I think you see where I'm going with this.
posted by Civil_Disobedient at 4:28 AM on July 12, 2006
Try Catalyst - http://catalyst.perl.org . The new release (5.7) has much improved docs, and as a bonus you can make Catalyst use the PHP interpreter in templates if that can help avoiding rewriting.
MVC style web dev is the way to go, and php doesn't really do it for me, because it "promotes messy coding" in my opinion - issues like namespaces etc.
self taught dev too by the way
posted by singingfish at 5:28 AM on July 12, 2006
MVC style web dev is the way to go, and php doesn't really do it for me, because it "promotes messy coding" in my opinion - issues like namespaces etc.
self taught dev too by the way
posted by singingfish at 5:28 AM on July 12, 2006
I wanted to try and clarify some of the advice your being given here. Oddly, that means speaking more generally.
First, in a round about way you're being told to "Follow best practices." I realize you're asking what that are, but I guess the point is, once you know them, follow them.
Second, all of this talk about classes, objects, OO, are meant to say this: "The current programming best practice is use an object oriented approach." Object Oriented Programming, if you're coming from PHP, is probably not something you're in the habit of doing.
I'm not going to go through the what, how, and why of Object Oriented Programming in-depth here, because they are well documented, although sometimes difficult to understand. The benefits are real, I'd say. To describe OO programming briefly, you define logical models of the different pieces that will interact in your application, perhaps user objects, bank objects, account objects, payee objects -each with its own intrinsic limits and abilities as you define them. Once you code the objects, coding the application is just coding the interactions between the objects.
OO is, again a best practice for a reason. PHP 5 is supposed to enable an object oriented approach to programming in PHP. I have not evaluated it, but expect that it would be fine and easy for you to use it to apply OO practices.
There are 2 major camps of object oriented web programmers around the business world right now.
Java programmers and ASP.NET programmers using C#.
In my opinion they are roughly comparable. If you're not using a Windows PC, I'd rule out ASP.NET. Otherwise, you could go either way.
These languages / development platforms were created with OO in mind. There are gazillions of programmers in both these arenas that you could use if the application matures. I think that platofrm partisans on this debate are like Baptists arguing with Pentacostals- they've both come to Jesus, but like their services with more or less snakehandling.
The other major piece of advice I see here is, "Even if you go OO, you're screwed if you don't meticulously design your database up front."
I'd add "And how." I'd also add, your database design should come first and be completed on its own terms, before you design your objects. Then, your objects should be designed on their own terms, not based on how the database had to be laid out.
So, to me your tasks are:
1. Come to terms with best practices, specifically Object Oriented Programming.
To do so, I'd get a "For dummies" book on Java or ASP.Net, or C#. I've just used the dummies C# book to get some VB6 programmers that work for me a leg up on Object Oriented programming. It had a 3 or 4 chapter section on OO that was easy to grasp & very helpful.
2. Determine if PHP 5 sufficiently enables OO techniques. Google is your friend.
3. Learn to wail on database design and normalization. mySQL is even supported natively by Microsoft and Java these days, so I don't think there is a platform problem.
It's tricky finding books & web tutorials on db design because they want to tell you about writing SQL mostly. I think skimming a random sampling of datadase design books & tutorials should get you where you need to be.
posted by putzface_dickman at 5:45 AM on July 12, 2006
First, in a round about way you're being told to "Follow best practices." I realize you're asking what that are, but I guess the point is, once you know them, follow them.
Second, all of this talk about classes, objects, OO, are meant to say this: "The current programming best practice is use an object oriented approach." Object Oriented Programming, if you're coming from PHP, is probably not something you're in the habit of doing.
I'm not going to go through the what, how, and why of Object Oriented Programming in-depth here, because they are well documented, although sometimes difficult to understand. The benefits are real, I'd say. To describe OO programming briefly, you define logical models of the different pieces that will interact in your application, perhaps user objects, bank objects, account objects, payee objects -each with its own intrinsic limits and abilities as you define them. Once you code the objects, coding the application is just coding the interactions between the objects.
OO is, again a best practice for a reason. PHP 5 is supposed to enable an object oriented approach to programming in PHP. I have not evaluated it, but expect that it would be fine and easy for you to use it to apply OO practices.
There are 2 major camps of object oriented web programmers around the business world right now.
Java programmers and ASP.NET programmers using C#.
In my opinion they are roughly comparable. If you're not using a Windows PC, I'd rule out ASP.NET. Otherwise, you could go either way.
These languages / development platforms were created with OO in mind. There are gazillions of programmers in both these arenas that you could use if the application matures. I think that platofrm partisans on this debate are like Baptists arguing with Pentacostals- they've both come to Jesus, but like their services with more or less snakehandling.
The other major piece of advice I see here is, "Even if you go OO, you're screwed if you don't meticulously design your database up front."
I'd add "And how." I'd also add, your database design should come first and be completed on its own terms, before you design your objects. Then, your objects should be designed on their own terms, not based on how the database had to be laid out.
So, to me your tasks are:
1. Come to terms with best practices, specifically Object Oriented Programming.
To do so, I'd get a "For dummies" book on Java or ASP.Net, or C#. I've just used the dummies C# book to get some VB6 programmers that work for me a leg up on Object Oriented programming. It had a 3 or 4 chapter section on OO that was easy to grasp & very helpful.
2. Determine if PHP 5 sufficiently enables OO techniques. Google is your friend.
3. Learn to wail on database design and normalization. mySQL is even supported natively by Microsoft and Java these days, so I don't think there is a platform problem.
It's tricky finding books & web tutorials on db design because they want to tell you about writing SQL mostly. I think skimming a random sampling of datadase design books & tutorials should get you where you need to be.
posted by putzface_dickman at 5:45 AM on July 12, 2006
One last bit. If PHP 5 doesn't cut it, choose java or ASP.Net base on your comfort with the resources and learning tools, and community available. Both have free developer environments. You can find hosting for both. There are lots of learning materials online, and tons of books. I'd limit to the choice to these 2 because they are the biggest with the most. That helps if you get to a point where other programmers are going to be involved. Can you imagine the job search for someone with expert knowledge of an esoteric programming platform?
posted by putzface_dickman at 5:55 AM on July 12, 2006
posted by putzface_dickman at 5:55 AM on July 12, 2006
Advanced PHP Programming by George Schlossnagle covers a lot of what you're looking for.
posted by staggernation at 6:01 AM on July 12, 2006
posted by staggernation at 6:01 AM on July 12, 2006
Perl also has a large user base, tons of libraries, and a significant history of use (lots of documentation, books, etc. out there). It's also open source and multi-platform, and supported on practically every web hosting provider out there. There may be plenty of reasons you aren't into Perl, but reading that Java and ASP.Net are "the biggest with the most" compelled me to mention it.
I'm not prepared to tell you that/why you should use Perl, but felt it should at least be mentioned because it's a major language.
posted by amtho at 6:44 AM on July 12, 2006
I'm not prepared to tell you that/why you should use Perl, but felt it should at least be mentioned because it's a major language.
posted by amtho at 6:44 AM on July 12, 2006
1. Is writing such a project in PHP a foolish undertaking...?
No I've written a fairly large application in PHP. PHP makes lots of things easy, and when a language does that, it makes making mistakes easy. PHP has the advantages of a) being installed in most webhosting environments, b) being easy to use, and c) having lots of built-in functions for common web-related tasks. If you were learning a new language right now for just this purpose, I'd probably suggest java.
2. Should I consider using existing libraries?
Yes. Avoid re-inventing the wheel, you have enough work to do. I used write all my own modules so I "understood how they worked." But spending an hour learning how someone else's well-written module works will always take less time.
4. What do I look to for security best practice?
http://www.zend.com/zend/art/art-oertli.php
6. Everything is going to live on one server -- should I plan for an eventual distribution of server duties? Is this something a developer even needs to worry about? I've really never had to deal with this before, so I can't elaborate on this question.
At least code it so that DB can easily be moved to another server. I'll assume you were doing that already. To work in a load balanced environment you need to be sure that all non-session specific data is stored in the database.
posted by justkevin at 7:50 AM on July 12, 2006
No I've written a fairly large application in PHP. PHP makes lots of things easy, and when a language does that, it makes making mistakes easy. PHP has the advantages of a) being installed in most webhosting environments, b) being easy to use, and c) having lots of built-in functions for common web-related tasks. If you were learning a new language right now for just this purpose, I'd probably suggest java.
2. Should I consider using existing libraries?
Yes. Avoid re-inventing the wheel, you have enough work to do. I used write all my own modules so I "understood how they worked." But spending an hour learning how someone else's well-written module works will always take less time.
4. What do I look to for security best practice?
http://www.zend.com/zend/art/art-oertli.php
6. Everything is going to live on one server -- should I plan for an eventual distribution of server duties? Is this something a developer even needs to worry about? I've really never had to deal with this before, so I can't elaborate on this question.
At least code it so that DB can easily be moved to another server. I'll assume you were doing that already. To work in a load balanced environment you need to be sure that all non-session specific data is stored in the database.
posted by justkevin at 7:50 AM on July 12, 2006
There's one point that lots of respondents here have glossed over, but no one has hit head on -- and as someone who's compiled and maintained *lots* of componentized FOSS packages, it's near and not-dear to my heart, so I will:
"Should I build my application using modules, components or libraries created and maintained by other projects?"
Well, yes, and no. :-)
Understanding *very clearly* *exactly* what the tradeoff is can be pretty important there, particularly if the application is ever going to be "released" (instead of merely "deployed").
Utilizing Other People's Modules pushes the responsibility for keeping the application working properly down from the programmers to the users.
When you take this approach, you have two choices: package up everything they're ever going to need, even if the packages in question are things which distribution maintainers provide, so that they're out of the user OS's update stream, and therefore, don't change out from underneath you, or use the versions that come with people's OSs.
The former solution clutters up people's machines (and in many cases, offends their aesthetics :-), and keeps you *right* on the critical path for things like security updates, while the latter pushes the security update problem to the OS distributor (and the end-user), leaving you only responsible for "which versions of components will my app work with reliably".
It's really a major conundrum, and one which no one's come up with a *really* good solution to, yet, other than "expect smart users" and "link for as wide a range of usable versions of package as possible", which, obviously requires mind reading.
Or scaffolding: if you can test all the interfaces you use, in a program whose only job is to do that testing and complain when something new breaks something old, you're probably golden.
Almost *no one* does this; I suspect there's a market for an automated tool that scans your codebase and constructs such a test program.
Writing that sentence, alas, is the most I can probably do to contribute to that. ;-)
posted by baylink at 8:09 AM on July 12, 2006
"Should I build my application using modules, components or libraries created and maintained by other projects?"
Well, yes, and no. :-)
Understanding *very clearly* *exactly* what the tradeoff is can be pretty important there, particularly if the application is ever going to be "released" (instead of merely "deployed").
Utilizing Other People's Modules pushes the responsibility for keeping the application working properly down from the programmers to the users.
When you take this approach, you have two choices: package up everything they're ever going to need, even if the packages in question are things which distribution maintainers provide, so that they're out of the user OS's update stream, and therefore, don't change out from underneath you, or use the versions that come with people's OSs.
The former solution clutters up people's machines (and in many cases, offends their aesthetics :-), and keeps you *right* on the critical path for things like security updates, while the latter pushes the security update problem to the OS distributor (and the end-user), leaving you only responsible for "which versions of components will my app work with reliably".
It's really a major conundrum, and one which no one's come up with a *really* good solution to, yet, other than "expect smart users" and "link for as wide a range of usable versions of package as possible", which, obviously requires mind reading.
Or scaffolding: if you can test all the interfaces you use, in a program whose only job is to do that testing and complain when something new breaks something old, you're probably golden.
Almost *no one* does this; I suspect there's a market for an automated tool that scans your codebase and constructs such a test program.
Writing that sentence, alas, is the most I can probably do to contribute to that. ;-)
posted by baylink at 8:09 AM on July 12, 2006
I would recommend reading and understanding OWASP's top ten security vulnerabilites.
http://www.owasp.org/index.php/OWASP_Top_Ten_Project
posted by toastchee at 8:20 AM on July 12, 2006
http://www.owasp.org/index.php/OWASP_Top_Ten_Project
posted by toastchee at 8:20 AM on July 12, 2006
5. The second programmer problem is your biggest reason to avoid PHP.
I contributed a few patches to the Wikipedia engine (a PHP flagship app, at least in number of users served) a few years back and eventually I just gave up. Even when I'd figured out the code's basic arrangement (where are the functions? ok, where are the templates? ok, now what the hell is this directory?), making changes would have unpredictable consequences. Unpredictable to everyone but the guy that had written most of the code, who also happened to be a dick.
Anyway, that's my PHP story. Normally I program in Java, but I've also done work in Cocoa (OS X) and I've done the obligatory exploration into Rails. If getting this prototype done quickly is your first priority, PHP is the obvious choice. You know it, it's simple, it's efficient.
But if you intend to pursue Web programming beyond this one app, you'll be doing yourself a favor by learning something more structured as soon as possible. And, let's face it, that means Rails for most people in your position.
posted by Doctor Barnett at 9:10 AM on July 12, 2006
I contributed a few patches to the Wikipedia engine (a PHP flagship app, at least in number of users served) a few years back and eventually I just gave up. Even when I'd figured out the code's basic arrangement (where are the functions? ok, where are the templates? ok, now what the hell is this directory?), making changes would have unpredictable consequences. Unpredictable to everyone but the guy that had written most of the code, who also happened to be a dick.
Anyway, that's my PHP story. Normally I program in Java, but I've also done work in Cocoa (OS X) and I've done the obligatory exploration into Rails. If getting this prototype done quickly is your first priority, PHP is the obvious choice. You know it, it's simple, it's efficient.
But if you intend to pursue Web programming beyond this one app, you'll be doing yourself a favor by learning something more structured as soon as possible. And, let's face it, that means Rails for most people in your position.
posted by Doctor Barnett at 9:10 AM on July 12, 2006
you should certainly look into use and application of design patterns, and specifically you should look into the application of the Model-View-Controller pattern as it relates to PHP/MySQL, or whatever platform you ultimately decide to go with.
prototyping is a good idea, but more often than not, the prototype becomes the armature around which the full-scale application ends up being written. accordingly, flaws in the prototype can wind up being magnified in the end product unless you write your prototype very carefully.
as for using "other people's libraries", there is enough quality open source out there that is simply too good or too useful to ignore. in the web application space, growing your own for the sake of growing your own is probably a mistake. in my experience, in general i find that my time is usually better spent evaluating some open source product or api than by trying to blaze my own path. sometimes i end up writing it myself anyways, but there's almost always something to be learned from examining other people's work.
posted by the painkiller at 10:12 AM on July 12, 2006
prototyping is a good idea, but more often than not, the prototype becomes the armature around which the full-scale application ends up being written. accordingly, flaws in the prototype can wind up being magnified in the end product unless you write your prototype very carefully.
as for using "other people's libraries", there is enough quality open source out there that is simply too good or too useful to ignore. in the web application space, growing your own for the sake of growing your own is probably a mistake. in my experience, in general i find that my time is usually better spent evaluating some open source product or api than by trying to blaze my own path. sometimes i end up writing it myself anyways, but there's almost always something to be learned from examining other people's work.
posted by the painkiller at 10:12 AM on July 12, 2006
Re: security. Here is one area where PHP makes it much harder than other langauges (like Java or Perl.) Unless you use the "mysqli" interface you can't use placeholders in SQL queries. When you use a placeholder you completely and totally eliminate any possibility of a SQL injection, as well as the tedium of having to manually escape every variable that you put in the query. It also is good for performance if you are doing a number of queries. In short, it's almost a no-brainer and it is easily available in things like Perl's DBI. But not so in PHP, where you have to turn to the "mysqli" interface which is very often not included in the default install and has to be added or manually recompiled by hand. Little things like this are why PHP gets a bad name for security.
posted by Rhomboid at 10:57 AM on July 12, 2006
posted by Rhomboid at 10:57 AM on July 12, 2006
Some background, I've been lead dev on quite a few e-commerce and e-government websites.
1. Is writing such a project in PHP a foolish undertaking?
No, there are many successful, enterprise class websites written using basic scripting languages (PHP, ASP, et al). The key to using PHP or any language really is to know the limitations of the language, don't be sloppy, code clean, and comment well.
I would however be hesitant about using MySQL for an accounting system. MySQL insofar as I remember is not ACID compliant which can be very bad news for transactional systems such as you're describing.
2. Should I consider using existing libraries?
That's entirely your call. Just be aware if you don't have the source (either open source, or written by you), then the reliability and uptime of your site depends on your vendors, which could put you in a sticky situation.
Having said that, you're inexperienced, so I would tend to trust tested, released libraries more.
3. Can you recommend some good books that will help me avoid stupid design mistakes?
Pretty much any O'Reilly book for the languages you choose should be fine on the technical side.
I do think you need to think more about the process. The reason most software projects fail, is not the technical acumen of the various programmers involved.
Some valuable books:
The OWASP project (while still relatively uncomplete) is a good start.
Just some common sense guidelines:
Hash passwords
Don't have sequential session keys
Validate all input, both for correctness and against injection attacks
Build security checks into every page
5.Besides good documentation and modular structure, what's the best way to make sure that other people can work on code that I've (possibly badly) developed?
Comment, coment, comment, comment.
Design docs usefulness fades, but it's always the comments that direct what how well others are able to follow you (hopefully).
6. Everything is going to live on one server -- should I plan for an eventual distribution of server duties?
Two things:
Design your interactions with your database so that your database can live on a different machine.
Design your site architecture so that all state is stored in the database.
These two things mean:
You can always split your database off to a different machine if the load gets heavy.
You can always split your webserver into a cluster if the load gets even heavier.
posted by patrickje at 11:32 AM on July 12, 2006
1. Is writing such a project in PHP a foolish undertaking?
No, there are many successful, enterprise class websites written using basic scripting languages (PHP, ASP, et al). The key to using PHP or any language really is to know the limitations of the language, don't be sloppy, code clean, and comment well.
I would however be hesitant about using MySQL for an accounting system. MySQL insofar as I remember is not ACID compliant which can be very bad news for transactional systems such as you're describing.
2. Should I consider using existing libraries?
That's entirely your call. Just be aware if you don't have the source (either open source, or written by you), then the reliability and uptime of your site depends on your vendors, which could put you in a sticky situation.
Having said that, you're inexperienced, so I would tend to trust tested, released libraries more.
3. Can you recommend some good books that will help me avoid stupid design mistakes?
Pretty much any O'Reilly book for the languages you choose should be fine on the technical side.
I do think you need to think more about the process. The reason most software projects fail, is not the technical acumen of the various programmers involved.
- Inability to foresee the risks involved
- Inability to set a project scope
- Poor development practices
- Unrealistic project estimates
Some valuable books:
- Code Complete - Steve McConnell
- Rapid Development - Steve McConnell
- Peopleware - Tom DeMarco
- User Interface Design for Programmers - Joel Spolsky
- The Pragmatic Programmer - Andrew Hunt
The OWASP project (while still relatively uncomplete) is a good start.
Just some common sense guidelines:
Hash passwords
Don't have sequential session keys
Validate all input, both for correctness and against injection attacks
Build security checks into every page
5.Besides good documentation and modular structure, what's the best way to make sure that other people can work on code that I've (possibly badly) developed?
Comment, coment, comment, comment.
Design docs usefulness fades, but it's always the comments that direct what how well others are able to follow you (hopefully).
6. Everything is going to live on one server -- should I plan for an eventual distribution of server duties?
Two things:
Design your interactions with your database so that your database can live on a different machine.
Design your site architecture so that all state is stored in the database.
These two things mean:
You can always split your database off to a different machine if the load gets heavy.
You can always split your webserver into a cluster if the load gets even heavier.
posted by patrickje at 11:32 AM on July 12, 2006
Response by poster: These answers are great! To clarify a couple things:
Thanks for all the confirmation on doing stuff the OO way -- I am familar with OOP (fortunately, this was one of the rudiments I learned before I dropped out of CS), and I actually *much* prefer it to procedural stuff. I am using PHP5 and primarily build classes, although I do use simple procedural scripts to process user input, etc. I do plan on using smarty (I've been developing with it for the last few months or so) and have fully bought into MVC programming (or as much as you can do so with PHP).
I should mention that I do web dev full-time (primarily PHP) at my day job, primarily patching and adding features to someone else's code that was written waaaaaaay back in 2001, which is mostly why I'm concerned about not having an unmaintainable mess like what I'm working with.
I think the big thing is that even the stuff I'm writing now -- using PHP5 objects and errorhandling -- *still* seems as though it's gonna take awhile for someone else to realize what I'm doing. I'm not yet convinced that switching to another language will, in the short term, result in better code, but I'm willing to start out in Ruby or Python if necessary. However, if writing better code is just a matter of sticking to a few principles (I'm already hugely anal about using K&R syntax style, for example) , then I might as well stick to PHP.
I guess I should post some code excerpts that I've recently written and have people look through them for bad practices. Let me see if I can find something.
posted by fishfucker at 11:35 AM on July 12, 2006
Thanks for all the confirmation on doing stuff the OO way -- I am familar with OOP (fortunately, this was one of the rudiments I learned before I dropped out of CS), and I actually *much* prefer it to procedural stuff. I am using PHP5 and primarily build classes, although I do use simple procedural scripts to process user input, etc. I do plan on using smarty (I've been developing with it for the last few months or so) and have fully bought into MVC programming (or as much as you can do so with PHP).
I should mention that I do web dev full-time (primarily PHP) at my day job, primarily patching and adding features to someone else's code that was written waaaaaaay back in 2001, which is mostly why I'm concerned about not having an unmaintainable mess like what I'm working with.
I think the big thing is that even the stuff I'm writing now -- using PHP5 objects and errorhandling -- *still* seems as though it's gonna take awhile for someone else to realize what I'm doing. I'm not yet convinced that switching to another language will, in the short term, result in better code, but I'm willing to start out in Ruby or Python if necessary. However, if writing better code is just a matter of sticking to a few principles (I'm already hugely anal about using K&R syntax style, for example) , then I might as well stick to PHP.
I guess I should post some code excerpts that I've recently written and have people look through them for bad practices. Let me see if I can find something.
posted by fishfucker at 11:35 AM on July 12, 2006
Response by poster: Oh, I should mention that I am familar with MySQL injection prevention (I use mysqli now, and prepared queries as much as possible, even though it's occasionally a PITA), and I'm wondering if there's some other major security issue that I might be ignoring.
posted by fishfucker at 11:36 AM on July 12, 2006
posted by fishfucker at 11:36 AM on July 12, 2006
Response by poster: Ok. Here's a code sample
I'm especially concerned about some of the methods like "Kind()" and "Author()" -- I occasionally feel that there's. got. to. be. a. better. way., and it's my lack of CS education that's keeping me from getting there. *
* You should note that a lot of the DB this code has to interact with is legacy, so some stuff that seems poorly designed on the DB side isn't really under my control. This is particularly true of the author section.
posted by fishfucker at 11:42 AM on July 12, 2006
I'm especially concerned about some of the methods like "Kind()" and "Author()" -- I occasionally feel that there's. got. to. be. a. better. way., and it's my lack of CS education that's keeping me from getting there. *
* You should note that a lot of the DB this code has to interact with is legacy, so some stuff that seems poorly designed on the DB side isn't really under my control. This is particularly true of the author section.
posted by fishfucker at 11:42 AM on July 12, 2006
There's obviously no right answer to the question of what programming language to use, but since no one has discussed python so far, and you seem to be interested in it, I'll put a word in. People that use it love it; its clean, and object orientation was built in from the beginning, unlike PHP. It has great support for modular programming (PHP's autoload hack is a joke). For web programming in particular, it has a wide variety of templating systems, covering a wide variety of design styles, along with excellent object relational mappers, form validation toolkits, etc.
If you want to test it out, I'd download python, go through the tutorial (should take an hour, maybe a bit more), and then download and experiment with cherrypy. Its the best of python's "simple" web frameworks. I'm not saying that you should use python, just that trying it out is the only way to make a decision about any language. And feel free to email if you have any questions.
posted by gsteff at 8:59 PM on July 12, 2006
If you want to test it out, I'd download python, go through the tutorial (should take an hour, maybe a bit more), and then download and experiment with cherrypy. Its the best of python's "simple" web frameworks. I'm not saying that you should use python, just that trying it out is the only way to make a decision about any language. And feel free to email if you have any questions.
posted by gsteff at 8:59 PM on July 12, 2006
some stuff that seems poorly designed on the DB side isn't really under my control. This is particularly true of the author section
Without seeing the schema, it's hard to tell if that's a bad thing, or a not-as-bad-as-you-think thing.
The code you linked to, well... It's obvious that whomever wrote it took great care to make it as clean as possible, and that's commendable. That said... Holy mother of God.
Just for example, you must include the DB
As far as naming conventions go: I tend to use reflection idioms even when the language is not reflection-based. It just keeps the code readable. So, for instance,
I hate seeing methods with their first letter capitalized. PHP already has syntax to distinguish between variables and function calls ($), so there's no need to make methods "stand out" by capitalizing the first letters. Also, while comments are nice to have, that code suffers from over-comment-itis. It looks like a code generator created those comments for the properties. Which is easier (and faster) to read:
Save the multi-line comment definitions for your functions.
Anyway, I'm rambling. The code is very decent compared to most PHP you see in the wild. I paricularly like seeing universal methods like GetClassName implemented. That's a nice touch, and most Cowboy Coders would omit it for the sake of brevity.
Unfortunately, the problem is that good coding practice is going to hurt your execution performance, since you're likely to be running this as a script. This is one of the reasons I tend to code my JavaScript routines by hand, even though there are some nice libraries available. They concentrate too much on "good coding practice" when, unfortunately, that good practice is going to negatively impact your code efficiency. It's a necessary juggling act as long as you stick with a runtime language.
posted by Civil_Disobedient at 10:06 PM on July 12, 2006
Without seeing the schema, it's hard to tell if that's a bad thing, or a not-as-bad-as-you-think thing.
The code you linked to, well... It's obvious that whomever wrote it took great care to make it as clean as possible, and that's commendable. That said... Holy mother of God.
Just for example, you must include the DB
.inc a gazillion times. Now, I understand that the concept is not very different from Java's import syntax, except for one huge difference: with Java, that shit is compiled, so you have the benefits of compile-time optimization. That PHP script, being script, is going to have to keep dynamically inserting the same chunks over and over again. What about connection pooling? What about caching? Threading? How are concurrent transactions handled?As far as naming conventions go: I tend to use reflection idioms even when the language is not reflection-based. It just keeps the code readable. So, for instance,
getters and setters. Properties and methods start with lowercase letters. Objects start with uppercase letters. No abbreviating. CamelCase, not _under_score. Boolean methods follow is/has naming convention (ValidAuthor() becomes isValidAuthor()).I hate seeing methods with their first letter capitalized. PHP already has syntax to distinguish between variables and function calls ($), so there's no need to make methods "stand out" by capitalizing the first letters. Also, while comments are nice to have, that code suffers from over-comment-itis. It looks like a code generator created those comments for the properties. Which is easier (and faster) to read:
/**
* User submitted item variables
*
* @var unknown_type
*/
public $item_variables;
/**
* this is the id of the current logged in user.
*
* @var integer
*/
private $user_id;
---snip---
// @variant : User submitted variables
public $item_variables;
// @int : ID of the current logged in user
private $user_id;
Save the multi-line comment definitions for your functions.
Anyway, I'm rambling. The code is very decent compared to most PHP you see in the wild. I paricularly like seeing universal methods like GetClassName implemented. That's a nice touch, and most Cowboy Coders would omit it for the sake of brevity.
Unfortunately, the problem is that good coding practice is going to hurt your execution performance, since you're likely to be running this as a script. This is one of the reasons I tend to code my JavaScript routines by hand, even though there are some nice libraries available. They concentrate too much on "good coding practice" when, unfortunately, that good practice is going to negatively impact your code efficiency. It's a necessary juggling act as long as you stick with a runtime language.
posted by Civil_Disobedient at 10:06 PM on July 12, 2006
Response by poster: Everyone! Thanks for your help -- nearly every answer in this thread has been useful to me, so consider yourself all marked as best answer.
posted by fishfucker at 12:41 PM on July 13, 2006
posted by fishfucker at 12:41 PM on July 13, 2006
This thread is closed to new comments.
posted by fishfucker at 11:27 PM on July 11, 2006