Lectures

• All the AIs We Have (Not) Seen
• Teachable Machine (Lecture)
• Teachable Machine - Google
• ml5.js
• Tech Examples

Readings

• Machine Learning Explained: A Guide to ML, AI, & Deep Learning, IBM Technology
• Excavating AI: The Politics of Images in Machine Learning Training Sets, Kate Crawford and Trevor Paglen
• A Critical Field Guide for Working with Machine Learning Datasets, Knowing Machines
• ImageNet Roulette, Trevor Paglen and Kate Crawford

Tutorials

• A Beginner’s Guide to Machine Learning in JavaScript, Daniel Shiffman / The Coding Train
• Train Your Own Neural Network: Classification, Daniel Shiffman / The Coding Train
• Pose Regression with PoseNet and ml5.js, Daniel Shiffman / The Coding Train
• ml5.js Neural Network
• TensorFlow Playground
• Learning to See, Memo Akten

Assignment 7

This assignment is part 2 of the project Uncertain Archive.

Step 1: Teach

Return to your Uncertain Archive and put it in front of a machine that learns from examples, the way a child is raised by a very strange family: shown many things, told what each one is, and never asked whether the names were right. You can use the pre-defined classification and regression models in ml5.js, train your own with Teachable Machine, or build something stranger outside these tools, which would be impressive in its own right.

If you use a pre-defined model, feed it the examples from your Uncertain Archive directly or through the webcam. If they’re not images, try different models that can classify your archive.

If you train your own model, decide what it should learn and which examples it will grow up on, and notice that every label you give it is a small verdict made by you. Pay attention to the labels you hesitated over, the examples you almost left out, and the parts of your archive you could not describe in a word or a number, because the model will be asked to describe them anyway.

You can continue to grow your archive and add new examples this week, try to create one with 500-1000 examples, and reorganize it using different categories or groupings.

Step 2: Look for Failures

Do not only look for what the model gets right. Look for where it is confidently, absurdly, or tenderly wrong.

The items it mistakes for something else, the ones it refuses to settle on, the ones it calls certain when you never were, a prediction that lands somewhere you did not expect, or a pattern it finds that you cannot see. Feed it things that do not belong, things that belong too well, and things that sit between the groups of your map, and keep a record of every failure, because a failure is where the logic of the model becomes visible.

You do not need to fix the model. Try changing what it was taught by removing examples, adding strange ones, relabeling, or shifting the numbers, and watch how its idea of your archive changes with its upbringing.

Step 3: Argue With It

For each failure, ask who is mistaken. The model may be blind to something you can see, or it may be showing you a pattern you were trained not to notice, or it may simply be repeating what you taught it without ever knowing why. Return to your Week 6 map and look at where your uncertainty appears in the machine’s mistakes, and where the machine becomes sure of something that you intentionally left open.

What is the model’s reasoning for its mistakes? What assumptions is it making about your data? How does its perspective differ from yours?

Research into the model you picked a little bit more, read its documentation, do some experiments with it, and try to understand its limitations and biases.

Bring Next Week

• your model or demo, however strange
• a collection of its failures, with your notes on what they reveal
• how the machine’s way of seeing differs from yours, and who or what taught it to see this way