No excuses. I’m busy and tired. Let’s get to the good stuff.

Five Things

1. On accuracy

A lot of the stuff I have been reading recently is around prediction, benchmarks and accuracy. Gabriel Grill’s work was a real breakthrough here.

A more productive way of engaging with this kind of accuracy would be to understand it as a marker or proxy for problematic inequalities, such as racism, in a society and then the goal should be to actually reduce this accuracy to increase the agency of marginalized people to have different futures. (Grill, 2022, p. 10)

This nicely capped off a line of argument in which the pursuit of accuracy as a key benchmark in the efficacy of predictive systems is only important so long as you believe in the existing world view. As Wang et al put it:

what seems to have happened is that in all the hoopla around big data and AI, the shaky intellectual foundations of predictive optimization were not sufficiently widely recognized, and developers’ claims, especially accuracy, went unchallenged for too long, even by critics. This let predictive optimization proliferate so quickly that it appears to have become a part of the new social order, and hence normalized, to the point where challenging the entire category seems almost unthinkable. (Wang et al., 2023, p. 18)

The risk of bungling out a load of predictive systems into the world (which includes generative AI as it predicts the next token) is that they cement the underlying logic and statistics that have gone into prediction. This may seem obvious on the face of it, but I was trying to understand why ‘accuracy’ felt so pernicious, almost like targeting as a military thing. Then there’s the way that, as these predictions are instantiated in products, they further cement that world and push demands for that version of accuracy. This paper predates our current AI boom (and in fact dismisses the chances of anything like the AI form ‘Her’) but indicates that both the speculative product in the film and the ambition to build those products in real life point to the desire, the worldview being locked into object, devices and services.

I suggest that we might understand Theodore’s situation less in terms of a heartbroken subject searching for consolation in devices and more in terms of a generalization of prediction of utterances and actions that progressively interpolates and interpellates subjects. This desire to predict desire has epistemic implications; it is power-saturated and also materializes in complex technological–cultural commodities that are beginning to stabilize in aggregate forms. (Mackenzie, 2015, p. 431)

So what? Basically the world as imagined by those building AI and the world-model predicted by AI get closer and closer. Every advance in ‘accuracy’ is simply the closing of this gap to a point where the model exactly replicated the version of the world in Altman or Zuck’s head. When someone says to you that something is increasingly ‘accurate’ you should be asking ‘accurate to what?’

2. AI as Normal Technology

I was going to say ‘I only just got round to reading AI as Normal Technology‘ but then I realised it was only published in mid-April and you can give me a break, I have other things to do than frantically scroll through the latest bit of AI reckons. Anyway, this bit of AI reckons is actually my kind of thing. From the folks that brought you AI Snake Oil and Air Street Press comes an essay that continues much of the same vein, namely that AI is just another technology, and is subject to all the same up-and-downstream whims and proclivities of previous technologies. As many have argued, the focus on the speculative and spectacular so-called misalignment problems of AI distracts from the well-established existing models we have for how we know new technologies are abused to entrench power:

While the [existential] risks discussed above have the potential to be catastrophic or existential, there is a long list of AI risks that are below this level but which are nonetheless large-scale and systemic, transcending the immediate effects of any particular AI system. These include the systemic entrenchment of bias and discrimination, massive job losses in specific occupations, worsening labor conditions, increasing inequality, concentration of power, erosion of social trust, pollution of the information ecosystem, decline of the free press, democratic backsliding, mass surveillance, and enabling authoritarianism.
If AI is normal technology, these risks become far more important than the catastrophic ones discussed above. That is because these risks arise from people and organizations using AI to advance their own interests, with AI merely serving as an amplifier of existing instabilities in our society.

Their thesis is that restrictive policies on AI development aren’t appropriate. Partly because the most effective risk mitigation measure are downstream (e.g. in the case of phishing – better spam filters and training people to spot spam rather than restricting what can be sent at the server which is both ineffective and harmful to normal activity) and because resilience is a better approach to the unknown and speculative impacts of AI.

They suggest that many of the safety concerns, their probabilities and impacts are unknown so it’s better to enhance the capacity of society and organisations as whole to mitigate potential risks by allowing the proliferation of systems. Read another way, this could be framed as socialising risk – spreading the impact of risk over a wider section of society – but their argument is that capitalism tends towards power imbalance and giving more people wider access to more powerful AI will mitigate the risk of a highly-motivated and nefariously-motivated smaller group having access.

So what? Their point is that the ‘worldview’ that AI is a normal technology is probably the most common one but, unlike the proliferation of manifestos and essays about some world-ending/enhancing superintelligence, no one has ever bothered to write it down. So it’s good to see it written down. The focus of the piece is very much on safety and encourages regulating AI just like any other technology as it proliferated with callouts to a handful of unique circumstances (e.g. ‘bioterrorism is not an AI risk any more than it’s an Internet risk, it’s a pre-existing risk that may be multiplied by AI’). Also reminds me a little bit of this piece that analyses AI purely as digital product that largely doesn’t work as advertised and that it can be approached under the rubric of consumer rights rather than high-minded ethical frameworks.

3. Things sit inside things, inside other things inside things now

I was reading Matt’s great writeup of building a multi-bot chatroom. What he describes is a problem of exponential scale. In a one-to-one conversation whether with an AI or a person the rules are relatively easy to follow; it’s all turn-based. You write something and wait for the response, then the other party responds and you wait and so on. There’s no pressure to intervene, speed up, show enthusiasm or anything. As soon as you add more parties into the mix it explodes in complexity; how do you decide who should respond to what and in what order? This is when conversations get quicker and more fractious; people posting single words to indicate acknowledgement without necessarily needing to contribute don’t demand a response but single words that indicate a ‘hold for a question’ might. And then with bots, how do they get instructed on which one replies to what sort of question?

The solutions Matt walks through are elegant in that vibey/London/Blue Peter way that he’s great at – none of that Californian glamour, just gluesticks and tape but goddamnit it works and has potential to work. But it made me think about a logical shift in the way we process information I’ve been thinking about at work. I’ve been trying to integrate MS Loops into our team’s organisation and communication; I make no advocacy for it as a thing and much like other sudden Microsoft integrations it’s mostly a pain but it exists and so I thought; why not? I find myself describing it in the same way I talk about Obsidian, like weaving a wiki, embedding, looping and linking snippets of information to organically weave an information ecosystem. This has been easy for me to bend my head around because I’ve already got Obsidian in my brain but it’s hard for folks who have grown up with, let’s say, a ‘filing cabinet’ model of information; things go linearly and are grouped into containers which sit in an order.

Reading how Matt is tackling some of these LLM issues, it’s somewhat similar; weaving a web of snippets that reference each other, loop around, check and link. I think the way LLMs work also introduces a new epistemology of information that has been somewhat previewed with Note-taking 3.0 platforms like Obsidian that work in this new, rhizomatic way.

So what: This was a tricky thing for me to get my head around in starting to use Obsidian; that the way it was organised don’t have to be immediate apparent and visible to me because it would emerge intuitively as I built it.

4. Education is a national security project

Education and national security have always been entwined. Whether it’s people building weapons literally also designing technologies that are used in education, or national security priorities defining education. Audrey Watters highlights how the new agreement between OpenAI and the UK government includes provision for the company to work with education ‘problems.’ This is while their parent company is building spyware for the IDF.

In the 1980s, Neoliberals adopted the slogan There Is No Alternative as a a way of framing the idea that the only way to stand up to communism was through their policies. Including, in the UK, the 1988 Education Reform act which introduced standardised testing for the UK and formalised this worldview in what ‘good’ education looked like. As a project of national security, we have SATs.

So what? Education as a national project has always been a thing but it’s a return to the use of it as a national security project that’s interesting here.

5. PhD update

It’s been quiet here because I am really, really focussing on some PhD work right now. I am on a deadline every two months to complete another section which means all my time is taken up with that pursuit. At the end of June I finished up a bit about how the anthropomorphic design heuristics of Replika left users unequipped to deal with the update and scrabbling for new metaphors to explain the sociopolitical shift that had resulted in their bots having ‘personality’ changes or ‘forgetting’ them. All that just in time for there to be a new, well-advertised podcast about the event.

Now I’m onto the next section which is about the use of rhetorics of scale by Cambridge Analytica to bamboozle both the press and their alleged clients as to their capabilities (hence all the reading on prediction). The exact flow of these arguments changes as a I go but I’ve got a pretty good process. Right now I’m on looking back over notes and re-reading core articles, pulling stuff into groups and starting to pull together an argument. I usually do this by pulling together quotes, images and articles that ‘feel’ like they sit together and it’s only once I actually sit down to write prose that a substantial and rational argument starts to form about why those things are connected. After going over and over and over it again half a dozen times, it reads like it started that way and I get to pretend I know what I’m doing.

Even that quite straightforward-sounding idea above – that anthropomorphic design heuristics Replika chose to implement in the design of the chatbots shaped the way users were able to respond to and reconcile the update – took 6 weeks to emerge from notes and strings of arguments.

So what? Just giving you excuses.

Recents

As some folks have noted, a lot of talks recently and I’ve put some up on the website. Including:

• This conversation on the PhD with Niklas Egberts at TU Dresden.
• Positioning AI with Maria Luce
• Interview with the Re-Pete project
• I started a podcast. Sorry.

Short Stuff

• Space can still be weird which is great news, this thing emits both x-ray and radio pulses on the order of minutes.
• Murmurings over the efficacy of scaling laws have abounded since the 80s. Perhaps this is the nail in the coffin.
• Microsoft’s Mattergen materials generator.
• Against Ai Everything, Everywhere, All at Once by Judy Estrin.
• AI ads think you’re a moron.
• Betti has a sick new book coming out. There’s a chapter called ‘Metic Wayfinding’ for lordy’s sake.
• You probably saw that you’re stupider with ChatGPT.
• Fascinating history of the Simulmatics Project and how it sat at the turn of computers and behaviouralism after the Second World War. You can probably imagine how I came across it.
• I think this is basically three-way fertilisation.
• I first read this paper in which computer scientists went ‘woah woah woah, lads, what if… what we’re doing has a negative social impact. Here’s an idea; you should think about that, ok?’ when it came out in 2018. It’s interesting to reflect now on everything that’s happened since. It’s still highly cited. I wonder if we’d be worse off without it.
• I like the argument here that a focus on ethics in AI precludes discussion on whether AI is actually useful or functional which may be a more helpful way of invoking critical thinking.
• Every dollar spent mitigating climate change now saves ten later.
• How about batteries made from sand, crisp flavouring or bricks.
• Dark Visitors is a database of bots, scrapers, AI agents and everything else creepy-crawling around your browser.
• You can, if you wanted to, store data in a bird’s song.
• Ride the solar tidal wave.
• Chimps have memes that often involve sticking grass in places on their bodies.
• LeBron James is not pregnant no matter what AI says.
• Greenhushing.
• I too have been reading about the reproducibility crisis and the essential need for metascience
• Now exporting cognitive debt to rural Colombia. That’s nice of you, Meta.
• China produces 70% of the world’s lab grown diamonds.
• Jo Barnard is on the Crit podcast.

It gets harder to publish the longer I leave it cos I’ve lost the thread on what the hell I’m doing so it no longer makes any sense to talk about these things. I reckon this one just about slid under the mat.