On the measurable divergence between human attention and model attention
In the preceding text, What matters and what is relevant, I argued that aesthetic misunderstanding—between human cultures and between humans and machines—can be described as a misalignment of relevance regimes: distinct hierarchies of what counts as a significant signal. That text operated primarily on a conceptual plane. This one operates on an empirical one, and it does so to answer a specific question: when a human and an artificial intelligence system are presented with the same image, do they look at the same places?
La respuesta de la evidencia disponible es clara y matizada a la vez. No miran los mismos lugares; la divergencia es medible, repetida y sistemática; y se concentra precisamente donde la relevancia humana deja de ser formal y se vuelve social, semántica o vital. Pero esa divergencia no es prueba de que la máquina «vea peor»: es efecto del objetivo para el que fue entrenada. Cuando el objetivo cambia, la alineación con la mirada humana puede aumentar mucho, sin llegar nunca a ser identidad.
Antes de entrar, una precisión que el texto anterior dejó gruesa y que aquí es central. Hablar de «la atención de la máquina» agrupa, de forma engañosa, al menos cuatro cosas distintas: la fijación ocular humana medida con eye-tracking, el mapa de saliencia de un modelo entrenado específicamente para predecir esas fijaciones, los pesos internos de un módulo de atención de un modelo entrenado para otra tarea, y los mapas de explicación tipo Grad-CAM que intentan reconstruir qué pesó en una decisión. Son cuatro niveles que la conversación pública mezcla y que conviene separar, porque buena parte de la confusión sobre «si la IA ve como nosotros» nace de compararlos como si fueran lo mismo.
1. Four things called attention
La distinción importa porque la respuesta a «¿miran lo mismo humanos y máquinas?» cambia radicalmente según de cuál de los cuatro niveles hablemos. Confundirlos produce tanto el optimismo ingenuo —«la IA ya ve como nosotros»— como el catastrofismo fácil —«la máquina no ve nada»—. La verdad es más fina y está en los datos.
The first level: observed behaviour
The first level is observed behaviour: where a person's eyes rest when viewing an image, recorded via eye-tracking. It is behavioural data, directly measurable. It is not a hypothesis about anything; it is what occurs.
The second level: saliency model
The second level is the saliency model: a system trained specifically to predict those human fixations. Its stated objective is to imitate the gaze. The DeepGaze family is the reference example. Here, the model performs no other task: its task is to resemble us while looking.
The third level: internal attention weights
The third level consists of the internal attention weights of a model trained for a different task—classifying, answering questions about images, generating. Those weights indicate, in an approximate manner, which parts of the input the model weighs when operating. They were not designed to resemble the human gaze; they are a byproduct of the architecture.
The fourth level: explanation maps
The fourth level is explanation maps, such as Grad-CAM, which attempt to reconstruct a posteriori which regions influenced a model's decision. These are interpretations of the machine produced by external tools, not the machine's gaze itself.
2. Do they look at the same places? The cleanest case
The study that addresses the question directly is that of Das and collaborators on visual question answering. The authors constructed a dataset of human attention—they asked people to indicate which regions of an image were relevant for answering a question about it—and compared it with the attention maps of the models that solved the same task.
The result is compelling. The best model of the time achieved a mean rank correlation of 0.26 with human maps. The agreement between different humans was 0.623. And in the evaluation corrected for central bias—the tendency to look at the centre of any image—the models fell to correlations of between 0.038 and 0.114. The authors' conclusion was explicit: the attention models for this task do not appear to look at the same regions as humans.
It is worth reading those figures carefully. They do not state that the model fails in its task—many answered the questions correctly. They state that they arrived at the correct answer by looking at something else. Success did not imply coincidence of gaze. This is the first empirical crack: one can solve the problem without sharing the relevance regime with the one who poses it.
Un segundo estudio, de van Dyck y colaboradores, comparó eye-tracking humano y mapas Grad-CAM en reconocimiento de objetos sobre los mismos estímulos. Su valor doble es que mide la divergencia y, a la vez, advierte de una trampa metodológica: las visualizaciones humanas y maquínicas tienen resoluciones distintas, y compararlas sin corregir eso distorsiona el resultado. Incluso tras corregirlo, el acuerdo seguía siendo bajo: una arquitectura coincidía con los bloques de atención humana en el 17% de los casos, otra en el 12%. El estudio deja una lección que conviene retener: la pregunta correcta no es «¿se parecen?», sino «¿qué arquitectura, qué tarea y qué métrica de comparación?».
3. When the goal is to imitate the gaze
The decisive nuance appears when the model is specifically trained to predict human fixations. There, the alignment increases significantly. DeepGaze II, built upon features from an object recognition network, manages to explain approximately 87% of the explainable information in human fixations, with very high metrics in benchmark test sets.
However, it does not achieve identity. In direct comparisons, the model falls below the level of agreement between humans. It performs particularly well in predicting that we will look at faces and text—because it has learned that this is where we look—but a low-level, purely physical model outperforms it on around 10% of images. That is to say: even the system designed to resemble our gaze leaves a remainder uncaptured, and that remainder is not random noise; it is the structure of the human gaze that the model fails to reproduce.
Hay un hallazgo todavía más revelador, de Fel y colaboradores. Al comparar 84 redes profundas con datos humanos de qué rasgos son diagnósticos para reconocer objetos, encontraron una compensación sistemática: a medida que los modelos mejoran su precisión en la tarea, muchos se vuelven menos alineados con las estrategias visuales humanas. Más exactitud, menos parecido a nosotros. Esto es exactamente lo contrario de lo que el sentido común esperaría —«cuanto mejor la IA, más como nosotros»— y es central para la tesis: la máquina no se aleja de nuestra mirada por defecto técnico, sino, a veces, por exceso de optimización. Se vuelve más eficaz en su tarea separándose de cómo un humano establece relevancias.
4. Where it breaks down: the social, the semantic, the vital
The divergence is not distributed equally across the image. It is concentrated in specific zones, and those zones are revealing.
Human gaze is not distributed solely by physical salience—contrast, brightness, edges. Cerf and colleagues demonstrated that, in free-viewing conditions, individuals look at faces and text significantly more often than at comparable regions matched for size and position: approximately sixteen times more for faces, eleven times more for text. Furthermore, a purely physical model requires the addition of semantic information regarding faces and text to approach predicting gaze. Social and symbolic elements carry weight for us, even when they do not dominate the visual field in physical terms.
Rösler, End, and Gamer confirmed this with briefly presented naturalistic scenes: initial saccades are oriented towards social features more than would be expected by chance, and social content predicts the direction of the first gaze better than physical salience. The act of looking at social stimuli is, in their terms, reflexive: it occurs prior to any deliberate decision and is directed towards areas that are not those of greatest physical salience. We looked at them for what they signify, not for how they stand out.
Del lado de la máquina, el régimen es distinto. Geirhos y colaboradores mostraron que las redes convolucionales estándar entrenadas para clasificar están fuertemente sesgadas hacia la textura, mientras los humanos privilegian la forma. Ante un conflicto entre forma y textura —la silueta de un gato con piel de elefante—, humano y máquina deciden cosas distintas, y por motivos distintos. Crucialmente, al reentrenar la misma arquitectura con un conjunto de datos diseñado para forzar atención a la forma, el modelo adoptaba una estrategia más parecida a la humana. La divergencia no era un destino de «la máquina»: era un efecto del entrenamiento.
El análisis interno de los modelos de saliencia afina aún más el cuadro. He y colaboradores encontraron que algunas categorías semánticas —cabezas, texto, símbolos, vehículos— ya están codificadas en redes preentrenadas, pero que el ajuste fino para predecir saliencia hace que el modelo favorezca unas sobre otras, por ejemplo la cabeza sobre el texto. Más aún: el modelo «aprende a dispararse ante caras» con relativa independencia de si esa cara es realmente la región más relevante en el contexto. Y aunque estos modelos dominan en escenas naturales, fracasan ante estímulos sintéticos de pop-out que los modelos clásicos resuelven con facilidad. Han aprendido muy bien ciertos sesgos del mundo natural y muy mal otras regularidades de la atención humana.
El cuadro que emerge no es «la máquina es ciega». Es más incómodo y más interesante: la máquina aprende algunos de nuestros invariantes —reconoce caras, texto— pero los pondera de otro modo, y produce con ello una cartografía de relevancia que se parece a la nuestra en la superficie y diverge en la estructura.
5. The case of art: a gap that is data
One might expect art to be the field where this comparison has been most studied, given the weight of eye-tracking in aesthetic research. This is not the case, and it is worth stating this precisely.
There is sound evidence regarding how a human views a work of art. Quiroga and colleagues recorded gaze towards figurative and abstract paintings and found that, despite individual variability, common fixation patterns exist: in one work, for example, the attention of all subjects was drawn to the area of greatest sharpness. Massaro and colleagues showed that gaze towards painting results from the interaction between bottom-up and top-down processes, and that correlations between aesthetic assessment and gaze patterns appeared precisely in the areas of the faces. Even in art, the social and the affective retain special weight.
There is also reasonable evidence that part of that gaze can be modelled. Le Meur and colleagues constructed a set of 150 paintings from five movements, found that the deployment of the gaze before paintings resembles that of natural scenes, and demonstrated that a saliency model specifically adjusted to the art domain predicts better than a general one. Art is not excluded from modelling, but generalisation from natural images is limited: it requires retraining.
Lo que no encontré —y conviene afirmarlo como límite honesto— es una línea consolidada que compare de forma sistemática los mapas de eye-tracking de espectadores de arte con la atención interna de modelos generales de visión aplicados a esas mismas obras. En arte, la comparación computacional adopta sobre todo la forma de «modelo de saliencia que predice la mirada», no la de «esto es lo que un clasificador consideró importante en el cuadro». Esa asimetría no es un inconveniente para mi planteamiento: es, en sí misma, un dato. El territorio donde la divergencia entre relevancia humana y maquínica sería más significativa —el arte, donde el peso vital y el contexto histórico son máximos— es precisamente el menos estudiado en estos términos. Ahí hay un vacío de investigación que el programa que esbocé en el texto anterior —el archivo de importancias— está diseñado para empezar a llenar.
6. Why a map is not a reading of what mattered
There remains a level that should be treated with caution, as it is where popularisation most often falters. Even if we possessed the internal attention map of a model, would it tell us what it truly found important?
The research response is that it does not, or at least not reliably. Jain and Wallace, in a paper with the eloquent title *Attention is not Explanation*, demonstrated that a model's attention weights correlate weakly with other measures of feature importance, and that it is possible to construct entirely different attention distributions that yield the same prediction. If two opposing maps produce the same result, the map cannot be the explanation for that result.
In computer vision, the issue recurs. Chefer and colleagues maintain that reducing a transformer to its attention weights is shortsighted: it ignores other operations that participate in the decision, and some methods based solely on attention highlight irrelevant elements or produce maps that do not even distinguish between classes. More recently, Wu and colleagues demonstrated that certain standard metrics for explanation fidelity fail to distinguish effectively between sophisticated methods and purely random attribution.
La consecuencia para esta línea de trabajo es importante y conviene formularla sin exagerar: ni siquiera desde dentro tenemos hoy acceso plenamente fiable a «lo que importó» para el modelo. Tenemos aproximaciones, algunas útiles, otras engañosas. Esto refuerza, no debilita, la tesis del régimen de relevancia: no solo el régimen de la máquina difiere del humano; ese régimen es además opaco, y lo es incluso para quien dispone del modelo entero y sus pesos. La caja no se lee a sí misma con transparencia.
7. What this demonstrates and what it does not
It is prudent to close the door on three misinterpretations.
It does not demonstrate that the machine sees less effectively
In many tasks, models perform as well as or better than humans. The evidence shows not an inferiority of machine vision, but a difference in criteria: it arrives at often correct results by attending to different regularities. Looking differently is not looking poorly.
It does not demonstrate that the divergence is fixed or essential
Es lo contrario. Cuando se cambia el objetivo de entrenamiento —de clasificar a predecir fijaciones, de textura a forma—, la alineación con la mirada humana se mueve, a veces mucho. La divergencia es un efecto del objetivo, la arquitectura y los datos, no una propiedad metafísica de «la máquina». Esto importa porque desactiva tanto el catastrofismo como el determinismo.
It does not yet demonstrate the self-referential part of my thesis
La evidencia sólida cubre la divergencia en lo social y lo semántico —caras, texto, forma frente a textura—. El salto desde ahí hasta «el peso vital, lo que toca al sí mismo» sigue siendo, por ahora, extensión interpretativa mía, no resultado empírico cerrado. Lo social está demostrado; lo autorreferencial es inferencia bien fundada pero abierta. Marcar esa diferencia es parte del rigor que esta línea exige, y es también una de las preguntas que el programa de investigación tendría que abordar de frente.
8. Conclusion
Three conclusions.
The first, empirical
When humans and machines attend to the same stimulus, the coincidence is partial, conditional, and measurable. Studies comparing human and machine attention maps on the same stimuli find low correlations, well below the level of inter-human agreement, except when the model is specifically trained to mimic gaze, and even then, a remainder remains uncaptured.
The second, analytical
The divergence is concentrated where human relevance ceases to be formal and becomes social, semantic, or vital. We look at faces, text, and meaningful content even if they are not the most physically salient; models, depending on their objective, rely on other regularities. And the divergence is not fixed: it depends on the objective, the architecture, and the data, which confirms that we are discussing configurable relevance regimes, not an essential blindness.
The third, methodological and programmatic
Ninguno de nosotros tiene hoy acceso fiable desde dentro a «lo que importó» al modelo. Eso hace que su régimen de relevancia sea doblemente distinto del nuestro: diverge y es opaco. Y el territorio donde esta divergencia sería más significativa, el arte, es precisamente el menos estudiado en estos términos. Esa laguna no es un obstáculo para la línea de investigación que vengo trazando: es su justificación.
The machine does not look less. It looks differently, and it looks differently for reasons we can measure, name, and, in part, modify. Knowing where it looks—and where it ceases to coincide with us—is the first empirical step toward understanding what would need to change in its architecture of relevance for its vision to approach comprehension, rather than merely a correct result. That is the distance between solving a problem and sharing a world.
About the open conversation
This text is the second in the series on relevance regimes that I opened with *What Matters and What is Relevant*, as part of my work in *NeuroArt: Cognitive Surplus*. If that piece established the thesis on a conceptual level, this one subjects it to empirical evidence regarding comparative attention. The planned next steps address the effect of self-relevance on aesthetic judgement and the limits of model interpretability.
If anyone wishes to intervene from the fields of perception psychology, computer vision, neuroaesthetics, model interpretability, or artistic practice, this notebook remains open.
Sources
Das, A., Agrawal, H., Zitnick, L., Parikh, D., & Batra, D. (2017). Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions? Computer Vision and Image Understanding, 163, 90–100. https://doi.org/10.1016/j.cviu.2017.10.001
van Dyck, L. E., Kwitt, R., Denzler, S. J., & Gruber, W. R. (2021). Comparing Object Recognition in Humans and Deep Convolutional Neural Networks—An Eye Tracking Study. Frontiers in Neuroscience, 15, 750639. https://doi.org/10.3389/fnins.2021.750639
Müller, R., Dürschmidt, M., Ullrich, J., Knoll, C., Weber, S., & Seitz, S. (2024). Do Humans and Convolutional Neural Networks Attend to Similar Areas during Scene Classification: Effects of Task and Image Type. Applied Sciences, 14(6), 2648. https://doi.org/10.3390/app14062648
Kümmerer, M., Wallis, T. S. A., & Bethge, M. (2016). DeepGaze II: Reading fixations from deep features trained on object recognition. arXiv:1610.01563. https://arxiv.org/abs/1610.01563
Fel, T., et al. (2022). Harmonizing the object recognition strategies of deep neural networks with humans. NeurIPS 2022. https://arxiv.org/abs/2211.04533
Cerf, M., Frady, E. P., & Koch, C. (2009). Faces and text attract gaze independent of the task: Experimental data and computer model. Journal of Vision, 9(12):10. https://doi.org/10.1167/9.12.10
Rösler, L., End, A., & Gamer, M. (2017). Orienting towards social features in naturalistic scenes is reflexive. PLoS ONE, 12(7), e0182037. https://doi.org/10.1371/journal.pone.0182037
Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., & Brendel, W. (2019). ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. ICLR 2019. https://arxiv.org/abs/1811.12231
He, S., et al. (2019). Understanding and Visualizing Deep Visual Saliency Models. CVPR 2019. https://arxiv.org/abs/1903.02501
Quiroga, R. Q., & Pedreira, C. (2011). How Do We See Art: An Eye-Tracker Study. Frontiers in Human Neuroscience, 5, 98. https://doi.org/10.3389/fnhum.2011.00098
Massaro, D., Savazzi, F., Di Dio, C., Freedberg, D., Gallese, V., Gilli, G., & Marchetti, A. (2012). When Art Moves the Eyes: A Behavioral and Eye-Tracking Study. PLoS ONE, 7(5), e37285. https://doi.org/10.1371/journal.pone.0037285
Le Meur, O., Le Pen, T., & Cozot, R. (2020). Can we accurately predict where we look at paintings? PLoS ONE, 15(10), e0239980. https://doi.org/10.1371/journal.pone.0239980
Jain, S., & Wallace, B. C. (2019). Attention is not Explanation. NAACL 2019. https://arxiv.org/abs/1902.10186
Chefer, H., Gur, S., & Wolf, L. (2021). Transformer Interpretability Beyond Attention Visualization. CVPR 2021. https://arxiv.org/abs/2012.09838
Wu, J., et al. (2024). On the Faithfulness of Vision Transformer Explanations (SaCo). CVPR 2024. https://arxiv.org/abs/2404.01415
Esteban Ruiz, J. A. What matters and what is relevant. Public notebook at juanesteban.art, 2026.
Esteban Ruiz, J. A. Art as Structural Surplus: Toward a Relational Ontology Beyond Human Authorship (V2.3). PhilArchive and Zenodo, 2026.