Sketches of Thought

Sketches of Thought:

Inside the Black Box of AI


Gabriele Mirra + Alberto Pugnale, University of Melbourne

‘Sketches of Thought’ is a human-machine collaborative design system based on Artificial Intelligence

The first aim of ‘Sketches of Thought’ is illustrating an approach to AI-integration within the designer’s creative workflow. The system translates a hand-drawn architecture sketch into a photorealistic image that suggests a possible evolution of the design idea. The dialogue with the system happens through a visual interface whereby the designer communicates by sketching directly on a drawing monitor, while the system responds by showing the results of the image translation process on a second monitor. Interaction with the system does not end after a first iteration. Instead, the designer is encouraged to adjust the initial sketch – or even make new sketches – for several times to explore, with the aid of the machine’s feedback, different elaborations of an idea. This system does not require particular drawing skills, and therefore anyone can experience a proficient ‘exchange of ideas’ with the AI model.

The second aim of ‘Sketches of Thought’ is helping the designer familiarise with AI technology. This is achieved by unveiling the black box of the AI model functioning, that is, through a representation of its internal processes. Moreover, as the AI model simulates some aspects of human cognition, a look inside the black box of AI also means visualising a simplified version of the human mental processes. Therefore, learning about AI is an opportunity for humans to learn more about themselves.

The relevance of this virtual prototype is twofold. First, it promotes the view of AI as a means to augment rather than replace the human cognitive capabilities. Second, it challenges the current beliefs and prejudices on AI-technology by making the AI internal processes explicit through a visual representation.


Gabriele Mirra_ is a computational designer and PhD student at the University of Melbourne. His research focuses on acoustic optimisation and the integration of AI within the architectural design process.

Alberto Pugnale_ is a Senior Lecturer in Architectural Design at the University of Melbourne. His areas of expertise are in the computational morphogenesis of structures and acoustic spaces, as well as in the history of construction.

Image: Diagram explaining the architecture of the AI model, as well as the human-machine collaborative system.