Help me figure out tech stack for a social network + app

March 20, 2017 9:41 AM

I will be developing a social network for an NGO during this year. I'm thinking of using Django (with Channels ) for the backend, standard HTML + jQuery for the web frontend and the same plus PhoneGap for the mobile apps (android and iOS). Please tell me why I'm wrong.

Some background:

1) The NGO is a micro-finance bank with over 100.000 clients, mostly low-income women of all ages with varying levels of tech ability, with Android phones and not-so-great data plans. Most don't have web connected computers at home.

2) The app will be a sort of Facebook-lite for the NGO's users, mostly focusing on messaging and buying/selling goods and services. There will be a strong locational aspect, but it'll be set by a location field in the users' accounts, not realtime GPS.

3) Most of my experience is with Django + Bootstrap + jQuery. We've done one super simple app using PhoneGap, and haven't done any native app development. I don't want to hire somebody or subcontract the project out.

4) Users should be able to take and upload pictures, maybe video, receive notifications, etc.

I'm mostly worried about: A) giving a good experience to users with old, slow phones on slow dataplans and B) not crapping out if all 100.000 users connect at the same time.

So, the actual questions are: Does the Stack (django, channels, bootstrap, jquery, phonegap) sound OK? Should I use one of the super fast python microframeworks instead? Or side-by-side with Django? Should I use a frontend framework like React or Meteor, with or without the Python backend? Any specific tips, advice or war stories doing something similar?

posted by signal to Computers & Internet (13 answers total)

IMHO only:

1. 100k users is small, so I wouldn't worry too much about scale stuff.
2. Assuming the business goal (rather than learning technology) is the point, use whatever tech stack will allow you to achieve the goal in a reasonable amount of time. Most tech stacks will work for most situations, even if they're not the new hotness. Don't go off and spend weeks or months learning something new just because it's trendy.
3. Also take into account the ability to find other developers to update and maintain the code after the fact (or what your teammates know how to work with, if you're not solo). You probably don't want to be the only person who can work on it.
4. Think long and hard about whether building a brand new social network is actually the best way to achieve your business objectives. I've consulted with a number of people who have started out saying they want to "create a social network" and it rarely turns out to be the best approach.

posted by primethyme at 9:50 AM on March 20, 2017 [10 favorites]

I think that stack sounds fine. I don't think you would reap much benefit from something superfast because your users' hardware is probably going to be your limiting factor, not anything on the backend. And even "100,000 users connecting at the same time" is still going to be limited by the speed of their phones (and fingers, for that matter).

I do agree with everything primethyme says though.
posted by mskyle at 9:57 AM on March 20, 2017

I'd really consider whether build is a better strategy than buy in this case.
posted by chesty_a_arthur at 9:59 AM on March 20, 2017 [5 favorites]

Please please please go where these people already are... WhatsApp or Facebook or whatever.
posted by k8t at 10:29 AM on March 20, 2017

The stack is fine, but I'd highly recommend a componentized, 'reactive' framework like Angular, React or Vue for the frontend. It's just tons cleaner than using straight jQuery. Vue.js is my current jam, but any of them are fine. For the backend, servers are (relatively) cheap, and you won't need that much horsepower to support 100k users unless all 100,000 are on the site at the exact same time. Django is fine.
posted by dis_integration at 10:37 AM on March 20, 2017

Response by poster: Hi, thanks for all the answers. A clarification: I've specifically been hired to develop a social network for my client, so while it might hypothetically not be the right path for them, it is what they're paying me for.
posted by signal at 10:38 AM on March 20, 2017 [3 favorites]

hmmm...this smells funny.

i think you could hit it out of the park by putting on a consulting hat and advocating for yammer. i used it at a medium sized firm. it's pretty good.

it also has a self-hosting licensing option.

this is a textbook buy-it-don't-build-it scenario. without having details, i'd bet you are setting yourself up for a reputational hit. greenfield coding is going to take a lot of tima and money, and will not keep up with the features or reliability of the big dogs.

you can still make your scratch by facilitating the license purchase, doing the portfolio management, and administering the system.

otoh...'reactive framework' +1
posted by j_curiouser at 11:07 AM on March 20, 2017 [2 favorites]

The overall stack seems fine, but since you sound like a solo developer with a limited-sophistication client, I’d avoid all forms of client-side code wherever possible. Plain HTML and a server-side technology stack is easy to comprehensively unit test, behaves predictably in most environments, and saves you a wide range of headaches on people’s weird local devices. The need to support old, slow phones particularly underscores this. In my experience attempting to find a point of stability in client-side code, most JS testing approaches like Browserstack or Selenium are unreliable and slow. Mobile apps are even worse, because you can't dynamically upgrade them; a friend who works as an Android developer for a large company says that every single historical version of the app he works on is being used by someone out there at all times. There are no take-backs with mobile apps.

Assuming that one of the buy-not-build suggestions in this thread isn’t a good enough answer, stick with what you know and do as little new work as possible: Django and Bootstrap, and jQuery only where absolutely necessary while adhering to progressive enhancement. Photo uploads don’t require client-side code.
posted by migurski at 11:12 AM on March 20, 2017 [2 favorites]

Don't don't don't develop a social network from scratch, especially for an NGO. Please convince your client that it's a bad idea. Your money and time is so much better spent going with Yammer or Ning or Slack and doing a whole bunch of custom styling / scripts / integrations / bots, rather than building everything from the ground up.

I'm sure the NGO thinks it's sexier and a better sell internally to make a 'custom-created' web site. Tell them that there are certain things you buy and certain things you wrangle together. As an analogy, they may want a private chef to 'custom-cook' a meal for them, but that doesn't mean that they need the private chef to grow all of their produce from scratch.

And how long do you see yourself maintaining this? When the NGO calls in 3 years because Django has a long-running bug that affects something, and you run a maintenance update, and it breaks something .. are they happy for being billed for that work?
posted by suedehead at 11:28 AM on March 20, 2017 [6 favorites]

There's obviously no technical reason why the stack you suggested shouldn't work, and I'm going to skip over the question of whether this is a good idea in the first place. I'm going to be the devil's advocate who suggests a more modern set of tools.

My own preference (with my own technical knowledge) would be to use a combo of Preact (a 3kb fast React clone) & Redux for the front-end of the web, and React Native & Redux for the mobile apps. This allows a lot of code reuse, and allows you to structure and reason about your app very similarly across platforms.

A reactive front-end framework like React is especially useful for more app-like projects like the one you're talking about. It becomes obvious to build it in components, which ends up encouraging consistency and modularity. It's very widely used, so there's a lot of documentation/discussion online. It's also reasonable to set up what's called isomorphic rendering, which basically means that you let the server use React to render the same pages
as the browser would, again increasing code resuse.

React Native will work well online/offline, and you'll be able to have a very large degree of code reuse across iOS (if you want it) and Android. It lets you build apps with the same patterns as React for the web, but it renders into native UI elements instead of the DOM. You can also use Microsoft's Code Push service to issue core app updates without waiting for the users to do so.

If using the app is wholly dependent on JavaScript functioning, then reducing your bundle size/execution time should be a priority. (See Alex Russell's talk on this). It will also allow you to trim down the real life data usage of your users dramatically, as will using something like Webpack's code splitting.

Alternatively, if I anticipated Android phones accounting for the vast majority of users, then I'd do a bit of research into the possibility of building the kind of progressive web apps that Google's been pushing a lot recently. There's a Preact-based starting point, and a very impressive Polymer-based example. Using service workers on the front-end should help make offline behaviour more elegant too.

All of the above suggestions may sound scary to you, but if executed well, should be more empathetic to your users restrictions. Obviously, time/budget/etc all apply - but I figure it's worth rattling your comfort zone a little. If it sounds like it'll be a lot of work - well, it will be!
posted by Magnakai at 12:02 PM on March 20, 2017 [3 favorites]

A clarification: I've specifically been hired to develop a social network for my client

Well not really, it sounds more like you are being asked to develop mutliple mobile apps for a social network that doesn't exist yet, and some kind of API for the data (maybe?), and possibly all of the backend data infrastructure as well. That concerns me because your clarification seems to actually lack clarity on exactly how large a project this is going to be, or even what a "social network" is.

A social network is created by the people who use the application. The only thing developers can do is make it easier for people to send and receive data through the social network. The heart of the social network is all the data that is shared by the users, not the UI or any of the features. The data is the social network.

But you haven't said where that data will live. You seem pretty focused on the front end, when an application like this is so much about the backend. Facebook does not maintain warehouses full of servers to run the UI (the UI runs in your phone just fine), they need those servers to serve data to the UI. It's the data that is the real challenge, and I get the sense that it's no more than an afterthought in your description of the "stack". I don't see a single mention of any kind of database in this entire post. That should concern you.

Where are you going to store and serve all those photos from? Is that going to be the same database that stores messages between users? Who is going to maintain the backend as the social network grows from 10 users to 1000 users? What is your backup solution for all this data? What are your reliability constraints?

These data maintenance and data scaling issues will not happen "automagically" with a purpose built custom solution. This is a project that requires active maintenance, and a set of feature release cycles and growth stages that could be measured on the order of many months *if* you had a team of experienced people working on it.

As other people have mentioned, I would seriously reconsider your approach. I understand you want to bill as a developer, not a consultant, and so you have a bias towards building, not buying, but you also have a responsibility to your client to give them the best solution possible. Unfortunately I think your suitability for this project as a builder is called into question from the apparent lack of understanding of the scale of this project. It's not really about the stack here at all, it's about scope. And the scope is crazy.

If you could find a way, as someone else suggested, to put on your "consultant hat" I think everyone would be happier in the long run. I apologize if I'm selling your development abilities short based on the limited information in your post, and there's nothing wrong with taking on big projects and shooting for the stars, but I think you are seriously underestimating how difficult a project like this will be, especially for a solo developer, and I believe it will be beneficial to your long term mental and professional health to take a step back for a minute here and reconsider your strategy for success.
posted by grog at 12:02 PM on March 20, 2017 [2 favorites]

I believe it will be beneficial to your long term mental and professional health to take a step back for a minute
this.

it would be really easy to walk this back, too. after some initial soundings from experienced peers, this is a pretty high-risk approach. we are much more likely to satisfy the organization with lower risk, lower expense, and shorter timeline if we implement off-the-shelf-product. and i can spend development hours fine tuning and customizing.

do *that* right, and they become solid customers for life.
posted by j_curiouser at 12:36 PM on March 20, 2017 [2 favorites]

If you choose to go the dangerous route of questioning your client’s assumptions, here’s an in-depth guide on how to talk about software risk. It assumes you’re staff rather than a contractor, so one outcome may be to talk yourself out of a job.
posted by migurski at 12:54 PM on March 20, 2017