WorkRed Paper Heart
Culture Cloth
Generative AI walls that mimic what you're wearing
- Role
- Lead Developer
- Year
- 2022
- Studio
- Red Paper Heart
- Project
- Full case study
- Technology
- Stable Diffusion, Image description model, Kinect, Cinder (C++), Unreal Engine, Real-time cloth simulation
Overview
Culture Cloth was a Red Paper Heart prototype and an experiment in using generative AI for interactive experiences: walls that mimic what you're wearing, in delightfully imperfect ways.
A Kinect camera found each visitor's body and captured an image of their clothing. That image went to a description model first, to get an accurate text description of the shirt's pattern or design, and then the description and the image together went to Stable Diffusion to generate a new, seamless repeating pattern. The pattern was sent into Unreal and applied to a real-time fabric simulation that rippled and reacted as people moved in front of it.
Clothing marks changes in mood, weather, style, season and occasion. With enough people passing through, the room ended up colored by the collective choices of whoever was in it. The patterns were interpretations rather than copies, and that looseness was the point.
My role
I developed the tracking and capture system that read what each visitor was wearing, and integrated Stable Diffusion into the studio's workflow.
- Built the Kinect tracking and capture app in Cinder (C++) that found each visitor and captured their clothing
- Integrated the Stable Diffusion API into the pipeline, from captured image to finished pattern
- Developed the Unreal application and its UI