[[The stereotypes are true and harmful|Stereotypes are useful (yet harmful) heuristics]], and of course AI has noticed+adopted them. ### Background: word embeddings + directions Modern AI uses "word embeddings", which take a word and map it into a long vector. In this vector space, with similar words being placed together. For instance, "cat" and "dog" might be close to each other, while "car" would be far from both. Another fascinating feature is that directions in the embedding space have meaning. This is captured by "king - man + woman = queen", which looks like: ![[Vector Arithmetic.png]] With this, you can actually take a direction in the embedding space, and find its meaning, so King-Man = Royalness, and Woman-Man = Femaleness ### Let's look at what the AI thinks: So you can actually take these "directions" and for each word see "how much" they have this quality. For instance, let's find out the direction of "more", and then rank the different words on how much they embody that. ![[AI Bias 1.png]] Kind of makes sense? ### AI is stereotyping Now, and we're getting to the main point of this note, let's find the female vector, and see how much the AI thinks that different words are "female". ![[Images/AI Bias.png]] This is a sad state of affairs! The AI concurs with the human bias of thinking that a nurse is female, and a doctor is male, and it for sure doesn't put president as feminine. This both a correct description of the world for sensemaking (in the US we *had* no female presidents, and most programmers *are* male), but it will continue perpetuating the stereotypes as people will consult AI. So AI will be as biased as us, because it has our shared context. [Colab Notebook](https://colab.research.google.com/drive/1W-qH9mf5Xw04bPu7nqc2r4g7QiHNk--z#scrollTo=ay_Aa8h-mZ0R) #published 2025-04-12 #process_next