How is Twitter still running?
August 14, 2024 12:20 PM   Subscribe

When Musk fired like 90% of Twitter's staff, the *best*-case scenarios that I saw said that it would stop functioning within 2 months. Now here we are 16 months later, and it seems like it's running fine? Leaving the politics of it aside, how is this possible? Is it really possible to run an app that large with a skeleton crew?
posted by nushustu to Technology (41 answers total) 11 users marked this as a favorite
 
This is pretty unanswerable, but ask yourself the opposite question:

If it worked today, what would make it break tomorrow? What would make it break in 6 months? What would make it break in a year?

In technology, things move fast, but not that fast. In the short and medium term, your now-gone talented staff left behind something they worked really hard to make good. But eventually the world will pass you by and you're stuck trying to run a coldfusion site way past the point where people would ever choose to use coldfusion or learn coldfusion or have even heard of coldfusion...
posted by cmm at 12:30 PM on August 14 [18 favorites]


It doesn't seem very surprising to me that you'd need fewer staff to keep something running than it took to design and build that system in the first place.

There's also a lot of motivated reasoning out there that takes the form of "Elon Musk is bad, and therefore every single decision of his is wrong." He might simply have been right that Twitter was heavily overstaffed.
posted by kickingtheground at 12:41 PM on August 14 [9 favorites]


Yes. I have worked in many different tech companies from small to 1000s of employees. Here's what is happening based on surviving layoffs and tech reporting:

1. Less security and content moderation: He hated moderation and cut it to the bone. Now dodgy accounts and posts that could be screened by expensive staff are visible.
Many companies outsource this to developing countries fro more savings. In the US these people are very expensive!

He can quietly hire contractors who scramble around there for a year who are cheaper than a full employee with benefits.

2. Automation: You can automate a lot of social media processes, especially around account creation, etc. This is very true if you don't care about fixing bugs and making a consistent user experience. Many people find that features that are paid on Twitter are free if they use an old version or the phone or whatever.

3. Possible funding from powerful friends: Elon was on at least 1 group chat of powerful dudes right before he bought it. The dudes discussed operations and a court case over his "funding secured" was going to involve these dudes. It is likely these guys have provided funding or something as he technically cut costs and staff. Think low interest loans or other things Walter Isaacson will discover in 10 years.

4. It is basically on pause, by that I mean they don't have a lot of growth in free accounts, paid accounts have been disappointing, and that means it will not grow or thrive .


You are looking at the PT Cruiser of media
: yes it exists and runs but fewer people admit to wanting it, so repairing it or selling it will be almost impossible in a few years. Or consider it like a big dead mall full of smoothie shops and cell phone stores, waiting for someone to buy the land and knock it down. Here are some predictions that seem pretty accurate so far...
posted by Freecola at 12:44 PM on August 14 [40 favorites]


To build upon the above, if cut lots of people actively making changes and developing new features, you perversely increase system stability because change is often a root cause of problems.

Secondarily, while the site works for one definition of "works" it is definitely not making money and is in fact loosing quite a bit of money. The active Trust and Safety measures that were going on played a big role in making Twitter a place that advertisers wanted to be or at least tolerated. Nowadays, most big name-brand advertising people won't touch Twitter with a 100 foot pole. I can't remember the last time I got an ad for a mainstream, blue-chip Fortune 50 company.
posted by mmascolino at 12:50 PM on August 14 [8 favorites]


Social media is unique because many leaders are obsessed with "free speech" but also wants ad dollars. Advertising is specialized speech, so interacting with them means you will eventually say something that offends a Fortune 500. A smaller company can happily accept ad dollars from "Plastic Pete's Pieces of Crap" but I don't think Twitter can for long....

Truly "free" venues and media turn into Gab, Truth Social, or anything related to the dark web. "Anything goes" is not an attractive message for most users or advertisers, strangely enough.

Allow me another analogy: The creepiest dude at the party will affect how long people will stay there!

I'm curious how long it will last, but assume it may slowly become a "cleaner version" of Pornhub and Gab combined. Not sure if the CEO can turn it around.
posted by Freecola at 1:03 PM on August 14 [4 favorites]


All the personnel talked with keeping civility and truthfulness were let go, no?
posted by citygirl at 1:05 PM on August 14


> Remember how the Y2K bug was supposed to cause a worldwide apocalypse?

Sorry, I can't let this one go. As someone whose first programming job was fixing Oracle stored procedures so they wouldn't break beyond the year 1999, I can tell you definitively that the reason there was no worldwide apocalypse is because the industry spent hundreds of thousands (millions?) of person hours and well over $100B in 1990s money to address the problem in advance.

I would also say that "seems like it's running fine" is generous. In the early days post Elon there were widespread outages of subsystems. Much of what you see in the web UI or on mobile Twitter apps also depends on data that is "eventually consistent." One of the upshots of this eventual consistency is that it can take awhile for broken things to become evident to an end user. I still hear high-volume X users complain about things like retweet counts, like counts, engagement reports, etc., having nonsensical numbers, thus indicating that behind the scenes things aren't working well.
posted by davybyrne at 1:10 PM on August 14 [76 favorites]


If "running fine" includes wholly imaginary hour-long DDOS attacks that stop high-profile Spaces conversations from happening, then I guess it's running fine.
posted by flabdablet at 1:34 PM on August 14 [24 favorites]


I'm not sure about the X platform, but 'Twitter is still running fine and they are maintaining and even got rid of a bunch of porno spammers, or at least shunted them off elsewhere.

And like it or not, Twitter is still a big deal where Joe Biden first announced he was no longer running for president.

Contractors, slower rollouts of new functionality, putting a pause on research and development, and potentially less content moderation, though I've not seen any real difference there.
posted by The_Vegetables at 1:38 PM on August 14 [1 favorite]


I see at least three dimensions to this....

1. security. Twitter was not great about security even before it got cut to the bone. It may be that Twitter is getting absolutely owned by black hats right now. Skilled black hats will do their work silently and without disruption, and Musk is probably not the most forthright guy when it comes to disclosing security incidents anyway.

2. uptime. Like the OP I'm surprised that Twitter hasn't seen a lot of downtime since it got musked. I explain this to myself by saying that uptime is an obsession of devops staff and Twitter had some good devops staff who may have engineered for sustainability. Also, Twitter is dead in terms of new feature work, and feature work is a big risk to uptime. Less change to the platform = a more technically stable platform. Also, note that both times they've tried to do giant ambitious political chats with Desantis and Trump, they had serious technical issues, suggesting that their ability to do stuff at that scale is limited.

3. as a business. +1 to "Twitter is a PT Cruiser" which is a super apt way of putting it.
posted by Sauce Trough at 1:38 PM on August 14 [1 favorite]


oh, yeah, and content moderation is a fourth dimension to the "is twitter working?" question ... which they've solved by basically ignoring it .... except to red-flag "cisgender"
posted by Sauce Trough at 1:41 PM on August 14


BTW the ad change is comically obvious. the ads are not from major corporations anymore, but fly by night scammers and super small time outfits..
posted by The_Vegetables at 1:44 PM on August 14 [5 favorites]


Aside from the fact there's a lot that is bugging out in terms of app UI, search functionality, "spaces" etc - he just tried to interview Trump and it bugged out before it could even get started properly! - keep in mind that the giant social media companies do have ad sales, marketing, PR/public affairs, media production, public policy, trust and safety/moderation teams which accounts for a large percentage of non-engineers.

Some of these people are pivitol in helping the app run better and with a better user experience, and some are more involved in external stuff like marketing/sales/growth/gov affairs but I imagine it actually doesn't take much to keep the lights on if the goal is to keep a shitty, spammy, glitchy version of the tool running if none of that is top of mind.

But it still definitely fulfills its original utility as an app for mass conversation. How many people does it take to run BlueSky? I imagine not nearly as many as Twitter even post-Elon cuts, and if BlueSky had no interest in keeping the user experience safe, spam-free, and smooth as possible if's active user base were to begin growing into the mass millions it would also be a similarly shitty, buggy spammy but still somewhat useable app.
posted by windbox at 1:44 PM on August 14


1. Twitter had gone wild with hiring — e.g. going from 5.5k to 7.5k headcount in one year. It's just not possible to hire at that rate and have everyone doing genuinely meaningful business-critical tasks. It had acquired huge amounts of froth.

2. Without in any way defending Musk myself — a percentage of society, and in particular people who identify as sensible reasonable centrist expert types who are very active on social media, have genuinely lost their minds on the topic of Elon and cannot deal with his rockets that successfully fly into space, electric cars people want to buy, etc. So there was a huge echo chamber lapping up hot takes in late 2022 along the lines of "I was in charge of the Doohickey Server at Twitter until I got fired and I can tell you for a fact the whole site will collapse in 72 hours without my essential contribution". These takes were all wildly wrong and yet you will never hear someone recant them.

3. The staff who are left are the ones who are insanely committed to the cause and working all hours.
posted by Klipspringer at 1:53 PM on August 14 [5 favorites]


> Remember how the Y2K bug was supposed to cause a worldwide apocalypse?

Seriously, how does this keep getting brought up as an example. It's like saying "gee, Covid was supposed to infect everyone but nobody I know's been infected" in a country with continuous lockdowns and mask protocols and contact tracing and air filtering. Or "gee, I got a cancer diagnosis but the tumor is gone and I'm fine" after you've gotten months of chemo and radiation therapy.

The worst thing is people keep bringing up the fact that a big Y2K disaster didn't happen as an argument that there's no reason to invest resources in addressing some problem, when in actual truth, not mythical truth, it's an argument for the exact opposite.


Anyway, I too have been wondering about twitter, most recently because the reporting about the recent Trump interview mentioned that there were significant service disruptions for over 40 minutes (which also happened during the DeSantis interview).

I wonder if the fact that the basic service is still up is helped by the fact that the service itself has been cut back. For example, I don't have a twitter account; I used to be able to nonetheless view both individual tweets and tweet threads, but that's been impossible almost since Musk took over. These days it's individual tweets only, if you're not logged in. I'm guessing that's cut down a lot on server loads and traffic. Also it's hard to find user activity numbers (apparently since "X" stopped reporting them?) but there are some articles I can't verify about how growth has stalled.

And there's this: Twitter is dead in terms of new feature work, and feature work is a big risk to uptime. When he renamed it X, Musk was talking big about how new features would be coming and how he would be turning X into an "everything" app. And even before Musk, I got the sense that existing Twitter features were actively evolving and new ones were being developed. But if there's anyone working on new features, the results are not yet public. It seems at least as likely that that work isn't happening because the remaining engineers have instead been working on keeping the core functionality going. It feels (though again, I'm just an occasional tweet viewer) like a place that's hunkered down and been treading water.
posted by trig at 1:54 PM on August 14 [7 favorites]


Things that *aren't* working:
Revenue is down 80%
There are almost no major-name-brand advertisers any more.
99% of the accounts that have followed me since the takeover have been weird sex-bots.
I don't see a lot of the tweets from people I follow, because the algorithm is busted.
I keep getting sent to the "For You" feed (as opposed to "Following"), for no reason - I literally NEVER want to see that.
People I know are getting more engagement on Bluesky, a site with a fraction of the user accounts that X has.
I got suspended the other day for making a wrestling reference to jumping off the top rope, so moderation is automated and broken, too.
Images stopped appearing for me last week, just showing the URL instead of the graphic.
Blocking no longer seems to work right - I see tons of content from users I've blocked.
They completely fell over with the Elon/Trump thing.
posted by bashos_frog at 2:17 PM on August 14 [25 favorites]


As has been pointed out, Twitter was regarded as overstaffed before it was musked, so I'm sure a lot of superfluous people were let go. I'm amazed at how "well" it's working, or that it's working at all.

I think there's a lot to be said for a solidly engineered technology base, but unless it's maintained (and properly upgraded), it'll fail eventually. In 2012 and 2013 I worked for a small company that was using business software so old they couldn't upgrade beyond Windows server 2000-something or Windows XP on the workstation side. It took some work, but it was still keeping the company in business and making a profit. (I left as soon as I could.)

What disappoints me is that its still regarded as an important platform. A lot of people on the progressive side of things still have Twitter accounts, and reference other people who still have Twitter accounts, and I just can't see why. I'm mainly on Mastodon, but at this point I'd consider following some of these people over to Bluesky or Threads if they would just quit directing me to Twitter.
posted by lhauser at 2:21 PM on August 14 [5 favorites]


a lot of the people let go were the ones doing the business parts of running a business, like being in compliance with laws and paying invoices to vendors, and upkeep on physical locations. Musk, life Trump, famously doesn’t pay his bills or follow laws and regulations, and while these are not things that are gonna immediately bring down the website, they are compounding risk for when eventually this catches up to them.
posted by Jon_Evil at 2:23 PM on August 14 [2 favorites]


> I don't see a lot of the tweets from people I follow, because the algorithm is busted.

I followed about two hundred people at the high of my twitterdom. Only one still appears to be tweeting -- a very chatty San Francisco academic bizdev type. My timeline is literally just that one guy now.
posted by Sauce Trough at 2:40 PM on August 14


> The staff who are left are the ones who are insanely committed to the cause and working all hours.

I wonder how many of the remaining staffers are on H1-B visas.

H1-B ain't indentured servitude but you can see it from there. Tech leaders love H1-Bs.
posted by Sauce Trough at 2:43 PM on August 14 [15 favorites]


Another perspective I'd add is that it takes a lot more people and time to design and build new infrastructure, and less to just operate it. So you can lay off huge numbers and the system keeps running fine.

Once it is operating, with all the usual design margins and redundancies in place, it takes a while for the internal breakdowns to become evident on the outside. But then when it does break down, you discover the whole thing's now rotten on the inside.

Also, adding to what Jon_Evil said above, legal problems take a long time to percolate to the fore.
posted by intermod at 4:01 PM on August 14 [2 favorites]


lhauser its still regarded as an important platform

That's why I continue to use it. I stay as it's the best source for African ecologists, people researching Christian Dominionism, angry useful political musicians, ecologists and eco-activists in general including lawyers.

I live in NZ where twit use is widespread and into farm sector, so what, it is a very imperfect world.
posted by unearthed at 4:31 PM on August 14 [2 favorites]


Working in FinTech for 48 years I was once called upon to go to a remote conference room and to have a very private conversation with a gentleman who was retiring about a sensitive security related program. I was a person in the corporation who both knew the language and the applications in which the program was used. He asked me what I thought of the program. I said it was a joke, it does nothing. Right he said, can you infer what that means? Yes I said and smiled. I left that company several years later after it was bought out. I never had a meeting about that program with someone else before I left.

There is a corollary to this. Young programmers will say "Why did my program stop working? It must be that last change I put in?" No Clyde, many things change: input data, number of days in the year, number of transactions or memo items may have caused an array to overflow, a subroutine could have been changed by someone else. As was pointed out above, stable systems tend to remain stable, stable systems with changes added? Watch out.
posted by forthright at 4:40 PM on August 14 [2 favorites]


> That's why I continue to use it. I stay as it's the best source for African ecologists, people researching Christian Dominionism, angry useful political musicians, ecologists and eco-activists in general including lawyers.

I like it for watercolors by Japanese artists, and crosstalk with western artists.

And every time I click links to tweets, chrome on android brings me to the front page and not the tweet. It's a 1 year old bug.
posted by sebastienbailard at 5:15 PM on August 14 [1 favorite]


Twitter is "still running" in a very loose sense; the site is more or less a walking corpse, with a staggering amount of its functionality more or less inconsistently working or not at all. Even basic features like search break a large portion of the time with cryptic error messages, let alone things like spam filtering (which is borderline nonfunctional if not worse; most of the time it filters real tweets while letting oodles of spam through).

On Japanese Twitter, for example, almost every tweet that goes viral gets literally hundreds or thousands of bot-responses, often in other languages, attempting to farm views (they're known as "impression zombies"). In addition, popular tweets often get copy-pasted hundreds of times (also by bots), resulting in the "trending topic" system more or less being completely broken.

This is before we even get to intentional failures of the site, like the absolute mess caused by prioritizing blue check replies.

A large portion of the site has left in the past 18 months or so as well; while it looks vaguely populated, what you see is merely what remains; you don't see everyone who disappeared.
posted by etealuear_crushue at 5:40 PM on August 14 [1 favorite]


Not to derail, but:

He asked me what I thought of the program. I said it was a joke, it does nothing. Right he said, can you infer what that means? Yes I said and smiled.

Come onnnnnn. What does it mean?
posted by maupuia at 6:54 PM on August 14 [14 favorites]


sebastienbailard
I use Anthony Restaino's Android Lightning-Browser
https://github.com/anthonycr

It's very basic but all I need for on the phone.
posted by unearthed at 7:53 PM on August 14 [1 favorite]


It depends on the way you define "how" and "working", imo.

If working = allowing people to post and comment, then we'd generally can assume it will simply keep going until something breaks it, be that someone changing its code (maliciously or accidentally), some kind of limitation being met (storage getting full), a hardware issue, or something else.

If working = continuing as a viable, healthy endeavor, with effective response to things like reports of threats and harassment, well... it's not. At all. For a while I was finally getting responses to reports from 6+ months earlier, which is utterly ridiculous. Now, they've completely stopped even stopped that pretense, from what I can tell.

In that second case, the answer to How? is Poorly.
In the first, well... remember all those reports from people about just how many hours beyond reasonable Musk expected the retained employees to devote to work? Those poor souls that remain may all be too afraid or trapped financially to tell him to stuff it.
posted by stormyteal at 9:15 PM on August 14


Now here we are 16 months later, and it seems like it's running fine

In my experience, it is not running fine. I often have to manually reload links to Twitter feeds or posts, in order for the site to load that specific asset. Otherwise, I get a rotating indicator that spins indefinitely. This was not a problem before Musk showed up and fired a bunch of engineers for looking at him sideways.
posted by They sucked his brains out! at 9:29 PM on August 14 [3 favorites]


it seems like it's running fine?

Prior to the purchase, I could flag spam,
hate speech, and porn-spam (as in an account replying to non-adult posts with hardcore porn) and it would usually get deleted in a few days.

Now the flag is almost always rejected immediately and the posts stay up forever.

It's not impossible to run a high traffic website with a very small staff, as long as you don't try to do too much beyond allowing users to post. craigslist, for instance, has a staff of around 50 people.
posted by zippy at 11:35 PM on August 14


One reason Twitter has not died yet - and may not do so imminently - is that it sits at the nexus of a symbiotic relationship between those who broadcast news and those who want to be in it. Users such as politicians, PR departments, and journalists - unlike advertisers - don't really give a stuff about the ethics/feature set/user base or technical reliability of the platform. They care only that it is a cheap, quick, global, key-word targetted, de-facto standard for getting the word out there: an announcement, a hot take, an event, a meme. Such users were delighted if their tweets were being seen and reacted to by you - in the days before en-Musk-ification - these days you may only be aware of something they said because of a newspaper, Youtuber, or Mefite who still scans the platform - but you are still aware of these messages, no?
posted by rongorongo at 4:19 AM on August 15 [1 favorite]


Yeah. There has been a tiny bit of movement this week in the UK, with some Labour politicians leaving X or scaling back (hopefully for good, and not just to make a temporary statement) because of its role in the recent hate riots. Better than nothing, but really too little, too late.
posted by trig at 4:41 AM on August 15 [1 favorite]


Mod note: This interesting question and answers has been added (Tweeted? Xeeted?) to the sidebar and Best Of blog!
posted by Brandon Blatcher (staff) at 6:02 AM on August 15


Tweeted? Xeeted?

Xat.
posted by flabdablet at 6:11 AM on August 15 [3 favorites]


Works for me, but I only use it to while waiting in a line for a moment or two, it does seem to refresh better in low internet connectivity than threads or blue sky. Certainly fewer of my old standards and more rightwing crazy but I hardly notice those as I seem to be crazy-post-blind.
posted by sammyo at 6:37 AM on August 15


I think that not showing threads to those not logged in is because they can't handle the load any more; that probably doesn't help bring new people on board. I've seen issues with embeds elsewhere being slow or broken, and even when logged in, video can hang midway.

I think it'll be a "gradually then suddenly" sort of thing.
posted by Pronoiac at 7:18 AM on August 15 [3 favorites]


That is something I definitely noticed as well. There seems to a minimum level of network connectivity where below, Threads does not but once you have it, Threads works seemingly as expected.

Twitter for all I said about it "working" above, is very much frequently never showing me images.
posted by mmascolino at 7:19 AM on August 15


It's not impossible to run a high traffic website with a very small staff, as long as you don't try to do too much beyond allowing users to post. craigslist, for instance, has a staff of around 50 people.

This is basically the answer I came in to post.

The truth is that we don't actually know the number of people that it takes to keep Twitter (or any other large-scale web property) operational. Twitter's staffing numbers always seemed on the high side to me, so it wasn't entirely surprising that it didn't fall over as soon as Elon started firing people, but as others have noted it's really only "working" in the loosest technical sense anymore; its tech stack might as well be frozen in amber. It follows he must be closing in on the magic number.

It probably helps that a lot of the technology that Twitter used when it originally scaled up (and it was terribly unreliable in the "failwhale" era) is now reasonably commonplace, or at least the concepts are. And it's possible they had a lot of good DevOps automation that's still chugging along, There Will Come Soft Rains style, without which it wouldn't be possible to operate at the scale they're still running at.
posted by Kadin2048 at 9:46 AM on August 15 [4 favorites]


Twitter is now operating much like my landline - instead of several calls a day, from people I need or want to talk to, I am getting robocall scam callers. The scammers and spammers are eager to use my landline because, while I'm not profitable to them yet, never having fallen for a scam, who knows when or if I might deteriorate mentally enough to become one of their victims. All their other victims were once happy and confident landline users originally. On my side, I'm still keeping the landline in case I need it, but I'm not keeping it because I want to. I'm keeping it because I haven't found an alternative for a couple of types of calls from people I still need to communicate with.

My phone company is running their land lines at a loss. In their case they are not allowed to stop supporting land lines because of their contracts with the government. They don't want any new landline customers and would be happy if their current ones all disappeared. They are ripping out and selling the copper as fast as they can get away with it. But they have enough funding from their other income streams that they don't have to declare bankruptcy, and they are not trying to wriggle out of providing land line services because of how bad they would look if they did.

The spammers and scammers are paying bulk rates to put their garbage out on Twitter, so Twitter is not hemorrhaging money. Twitter might even stay in business for awhile, as long as the spammers and scammers think they can find dupes on Twitter.

Twitter is still in business the way a landlord who is renting to a meth lab and crack den is still in business. The operators of the crack den faithfully pay the rent, but you need a hazmat suit to safely enter the premises, and even then you might get shot, or savaged by an abused pit bull. Two thirds of the fixtures inside have been smashed, and the building itself is a write off. The neighbors are desperately looking for new apartments, and the landlord is conscientious about claiming about having good tenants and not investigating the actual condition of the edifice.
posted by Jane the Brown at 2:58 PM on August 15 [2 favorites]


"twitter got musked". pure gold.
posted by bluesky43 at 12:04 PM on August 17


I think much if the doom-saying was simply wishful thinking by haters.
posted by davidmsc at 11:01 AM on August 19


« Older How to accept having to move due to migraines?   |   Quiet hybrid vehicle Newer »

You are not logged in, either login or create an account to post comments