There’s a veil of grey-blue over my window. Everything through it is tinted by it. It’s not hazy out there, the lights and shapes are clear but they’ve all been cooled and muted by the grey-blue veil.
I feel like quite a lot happened last week but as usual I can’t remember much of it. The time seems to slip away doens’t it? I get to my desk around 0610 after figuring out whether the morning demands the big coffee or small coffee and then before I know it, it’s 1700 and I still have a couple of things to wrap up before tomorrow. And there are some things in the email that will take more time that are still unread and remain so for weeks often – I’m sorry. 0610-0900 is my favourite time. It’s quiet and not many other are awake, there are no emails or updates on Teams™ until about 0830. I usually use this time to read and write something or Zwift. At 0900 the fire-fighting starts and goes on into the late afternoon. There’s no point making plans to do anything during the day.
This all means that I am firmly ready to leave my desk by 1700. But I find my evenings now are in a half-daze of tiredness. I can’t concentrate because I’ve front-loaded everything, I tidy up around the house, do my daily Duolingo and maybe play a video game for an hour or so.
My video game time has nose-dived recently. My relationship with video games has always been strained. At times they have been a dangerous ghostly presence in my life, calling me to finish up work and play and sinking dozens of hours a week. Half of me feels guilty when I get into these binges but the other part feels that if something makes you happy, you should do it. I suppose that’s the dilemma of capitalism amirite.
Either way at the moment I’m not playing many. I got the Star Wars spaceships one and wasn’t too impressed so dropped it quickly. Wes got me into Rimworld which is brilliant fun but requires a strategic capacity to properly understand that I can’t offer it at the moment. I’m probably not going to get a PS5 for a while. Not until they’re cheaper and there’s a couple of good games to play, maybe around Easter I can settle into Cyberpunk on one. But by then the weather will start to pick up and I’ll want to be out on the bike all the time.
So cycling takes up around 15 hours a week. I want to get stronger, at the moment my FTP (Functional threshold power) which is a baseline measure of strength is around 230-250 watts at a push. This is very good for my weight but not ‘A’ material. So I’m doing another big training package on Zwift over the winter to try and build it up more.
Why did I tell you all this? I don’t know. Taking stock I suppose. A year ago I was at the pub four nights a week, smoking a lot and barely sleeping to try and cram everything in.
AI: Good new bad news
Seems AI was in the news again a lot this week. DeepMind has developed AlphaFold which successfully predicts the way protein will fold. How do they come u p with these terrible names? Modelling protein folding is one of the original problem propositions for AI from the 1960s, before the fantasy of human-like intelligence took over and before it was even really called AI. The implications for health are significant as a lot of the trickier analytical work of medical and bio-science is difficult to do because of its unpredictability as well as being prone to human error and misinterpretation. I remember reading about how protein folding and early cancer detection were like the golden ticket for machine learning researchers way back when.
It’s nice to see a conversation around AI grounded in something meaningful rather than robots taking over the world or whatever. A lot of the reading I’ve been doing recently around the sociology of expectations and innovation talks about how often things and products that make it to the popular press are spin-offs of longer, harder and less glamorous projects the aim of which is to keep public interest and investment in the science.
On the other hand, Google decided to go along and fire Timnit Gebru for apparently not following Google’s paper publication process. Gebru’s paper (which isn’t published but is summarised here) addresses the risks and issues of large data sets for natural language processing machine learning. These are the type of sets that go into thing like GPT-3 which had everyone all in a tizz the other week. Gebru points out that their extreme size and weight means that they are prohibitively expensive to work with. This makes them not only environmentally unsound but also secures a monopoly for those with the resources to work with them. Lots of the data on the financial and carbon cost are in the summary there.
Secondly she points out that these sets are un-curated and un-documented. They are most often just massive scrapes of what is out there. This means that they have the tendency to homogenise languages by – for instance – including racist tracts of text with the same weighting as the developing anti-racist language being articulated by Black Lives Matter and others. And without documentation there’s no accountability nor ease of editing or curating for someone working with it.
Finally, and what I found most interesting was the idea that these language models created ‘illusions of meaning’ – that, as discussed previously we have a habit of mistaking statistical likelihood for meaning. ‘Illusion of meaning’ seems like such an obvious way to put it but for some reason, in writing and thinking about meaning-making practices in AI I hadn’t. In the same way that Facebook continues to insist it is not a media organisation – partly for reasons of regulation and tax, partly because it puts an existential burden on them that they don’t want to have to deal with – Google insists that it is not in the habit of bending reality. Because these type of projects are spin-offs of who knows what that end up in the public domain there’s a reluctance to engage in the same type of scrutiny that something as prestigious, significant and genuinely useful as predicting protein folding get after decades of intense work.
Google suggested that Gebru’s paper didn’t meet their criteria because it “disregarded subsequent research showing much greater [ecological] efficiencies. Similarly, it raised concerns about bias in language models, but didn’t take into account recent research to mitigate these issues.” However, one of the co-authors points out that they reference 128 sources including several that demonstrate attempts to remediate the social and ecological risks of these data sets. In the same article, a former PR person for Google also suggests that this is not normally the way this process would go and implies it is an extreme application of procedure where normally the vetting process is only to protect corporate secrets rather than offer insight into quality which is done through peer review anyway.
The contents of the paper as reported are significant – Gebru is a titan in the field – but not altogether mind-blowing. The fact that data sets can be biased in their collection and application and ecologically destructive is no earth-shattering revelation. That plus the heavy-handed application of procedure seems to indicate something deeper and more nefarious in her firing rather than errors of procedure. Seems the fight is still on for the ‘soul’ of machine learning.
Short Stuff
- One way of responding to the very real problem of water scarcity, particularly somewhere like California is to make it a tradable commodity like gold or oil. (This was sarcasm)
- I’ve been very much enjoying a bunch of writing up on the Walker Art Centre blog. Not least because of the peers and colleagues it features. Defuturing the Image of the Future by Andrew Blauvelt was particularly cool.
- Are we still in ‘uncertain times’? Benque, make a diagram.
- Real Life mag, (which I didn’t realise until the same Benque told me) is related to Snapchat is a great magazine full stop with some of the most interesting articles I’ve seen in recent times. BUT, did you know they also have a podcast version of some of the articles?
- I know I’m just resharing what I twittered about or Instagrammed last week and that’s a bit lazy but this paper by Maya Indira Ganesh – The ironies of autonomy – was awesome. It reframes autonomous technologies and human as a heteromy in which both are component parts of a system as well as addressing the way we misconceive autonomous systems.
Well I opened more tabs this morning than I closed which tells you something. Until next time, write me back would you? Love you as always.