Jamal Awil

← Doppelganger

Algorithms amplify identity confusion through learned pattern reinforcement [causal]

I felt blood rush to my face. In that moment, nine months into living on the rock at the end of the road, thirteen months of remote teaching and indoor masking, an eternity since seeing my closest friends, it suddenly hit me: the confusion was now so frequent that Twitter's algorithm was prompting it, helpfully filling in the mistake for its users, to save them precious time. This is how machine learning works—the algorithm imitates, learning from patterns. So, if my name is repeatedly mixed up with Wolf's, even in jokes, then my name would start being suggested instead of hers, leading to even more mix-ups. Which also meant that anything I did to correct the record—or state my own position on what had become her pet topics—would just train the algorithm to confuse us even more.

QUESTION: The feedback loop described here — where corrections train the algorithm to further conflate two people — is a disturbing implication worth exploring. How does this play out for people with similar names or identities more broadly? SEED: Could be an essay angle: how algorithmic feedback loops can trap individuals in misidentification cycles that resist correction.

Naomi Klein, Doppelganger, loc. 125