Software experiments and dumpster fires

I still haven't fully delved into your previous post about social media, but I wanted to highlight an article I ran across recently talking about how software experiments may factor into how platforms have tended to slowly morph into the shit-shows they are today.

The post, Experiments Aren't Expressions Of Customer Desire talks about how experiments are mistaken as "letting the users decide" which is how I've always viewed them, but that's clearly not what they are.

But the rhetorical flourish it uses - “Let the users decide” - is wrong, in a serious way: the outcome of an experiment is not a customer’s decision.

• The customer does not get to choose whether they get to use A or B: we do.
• The customer doesn't get to choose what A or B are: we do.
• The customer doesn't get to choose what metric determines whether A or B wins: we do.

It's always a little troubling when something like this is presented to me and seems so glaringly obvious but yet I'd never thought of it that way. To me instrumenting an app in such a way as to slowly evolve it over time based purely on customer usage was the height of system improvement. I know from experience that my own intuition about what people prefer is often wrong so experiments are the gold standard, right?

By itself this insight would be very interesting to me, but the author's tying this methodology to social media's descent to the dumpster fire it is today is what really blew my mind. I guess I'd given "the algorithm" most of the credit–or rather blame–here, but maybe it's just as much these experiments; thousands of them over time slowly being chosen for driving engagement and retention.

At the end of the day, “Experimental outcomes are customer sentiment” isn’t a philosophy we buy into because it actually serves the interests of our customers. It is a philosophy we buy into because it lets us believe we are serving the interests of our customers, even as we are not. And in the long run, it’s a damaging philosophy that eventually depletes the trust our customers place in us.

Ouch.. This article definitely has me rethinking one of my favorite software "holy grails". It also makes think that there's really no replacement for a good UX engineer doing the hard work of regular usability testing–watching users navigate your system and see where they get stuck. It strikes me now that the flashy "culture of experimentation" in some ways is trying to automate away this important role. Watching what someone does and hearing what they say while they use your system is actually "letting users decide", whereas the other is, as the author states, just an illusion of serving the customer.