AI's Geography problem
Waddah Hago:
Consider a simple example. Flat concrete roofs are common throughout the Caribbean because they are designed to withstand hurricane-force winds. Yet AI models trained largely on North American imagery have sometimes classified these homes as parking lots because they do not resemble the residential structures the model learned to recognize. The model isn’t broken. It simply learned from a dataset where homes generally have pitched roofs, asphalt shingles, and suburban street layouts. When confronted with something different, it makes the best prediction it can based on what it knows.
The same problem appears across much of the world. A neighborhood in Lilongwe doesn’t look like a suburb in Virginia. Traditional homes in Malawi don’t resemble the structures that dominate most Western training datasets. Informal settlements in Abuja don’t follow the same patterns as planned communities in Arizona. The buildings are different. The materials are different. The settlement patterns, densities, vegetation, and infrastructure are different. When AI encounters environments it has never seen before, performance often begins to deteriorate.
Without local knowledge, you cannot produce an accurate map. This is a problem now, as it was in the days of ERDAS Imagine, when we still called it image classification rather than AI. What has changed is how we view AI output today. The marketing of leading AI companies and a litany of one-sentence-per-paragraph influencer stories on LinkedIn suggest that the models get better and better — which they do — and that they never fail; let’s go all in, trust the output and move on.
One of my first jobs was to sit in an office, stare at a screen for eight hours, draw rectangles over satellite imagery of Hungary, and classify each rectangle. We had training and extensive examples to draw from. But in the end, a team still went to Hungary to verify on the ground what we mapped. If you can’t trust a human to make correct assumptions about the state of the environment in a remote location, how can you trust the output of a statistical model when you don’t know what data it has been trained on?
I’m not arguing not to use AI at all. These tools are efficient and useful. But the model outputs include errors, and cognitive surrender is real. You need a human to control and own the output, with all its consequences. The model output is your output. If the model output is wrong, then you are.