GaoDe — Game of Design

Projects / GaoDe — Game of Design

GaoDe: A Multimodal Architectural Design Serious Game

GaoDe custom user interface with pop-up guidance and real-time feedback

Summary

GaoDe, Game of Design, is an open-ended educational serious game that lets architecture students practice design through self-assessment inside a familiar CAD workflow. Instead of keyboard/mouse-only input, it uses a multi-modal natural user interface that combines hand gestures, machine vision, and offline speech commands to support playful exploration without “right or wrong” solutions.

Rhino Grasshopper Python Leap Motion Machine vision Offline speech HCI Game-based learning

Read the paper (open access)

Project at a glance

Context

Master’s project (2020) • Educational technology • Parametric CAD environment

Domain: early architectural design education
Supervisor: Dr. Ali Andaji
Core idea: students compete with themselves, not each other
Outcome: immediate feedback for self-assessment and iteration

What students do

Select one of five iconic-building game modes (each with different rules)
Model and navigate in 3D using gestures + finger-count actions + short voice commands
Compare outcomes with reference targets, then iterate
Finish a session and reflect on choices (self-assessment)

Problem and goal

In early studio education, students often learn design concepts but struggle to apply them confidently in open-ended tasks. GaoDe reframes practice as a playful, low-risk environment: learners explore alternatives, receive feedback, and steer their own progress through self-assessment rather than external grading.

The goal is to strengthen the hands–eyes–mind loop central to design thinking, while staying embedded inside the CAD ecosystem students already use.

System architecture

Leap Motion (gesture recognition)

3D hand/finger tracking mapped to modeling actions and camera control
Example pattern: clench one hand; move the other around it to navigate

Webcam + CNN (finger-count recognition)

Simple, high-confidence actions using number of visible fingers
Example mapping: one finger = save; five fingers = finish

Offline speech (keyword spotting)

Short commands (e.g., “yes”, “up/down/left/right”, “stop”)
Intentionally gated (e.g., a yellow marker) to reduce accidental triggers

Processing framework flowchart of GaoDe — Processing framework: multimodal inputs feed game logic + parametric modeling actions inside Rhino/Grasshopper.

Interaction design (examples)

Click any image to open full size.

Camera control using an open hand moving around a fist — Gesture-based camera control (Leap Motion).

Finger count gestures: one finger saves, five fingers finishes — Finger-count actions (webcam + CNN): save / finish.

Voice command activation using yellow color detection — Offline voice commands with color-gated activation.

Five game modes

The initial GaoDe prototype includes five games based on iconic buildings. Each mode targets a different aspect of design perception and uses distinct interaction rules to sustain motivation and avoid repetition.

1) Brick Country House (Mies van der Rohe)

Brick Country House game guide — Volumetric ordering and spatial partitioning using guided walls and block placement.

Learning focus: volumetric reasoning + ordered iteration
Key interaction: palm orientation and pointing to place blocks

2) Fallingwater (Frank Lloyd Wright)

Fallingwater game guide — Target style is not fully revealed during play; students control proportions and orientation, then “reveal” the style at the end.

Learning focus: visualization under uncertainty + proportion control
Key interaction: stack elements, then transform into a Fallingwater-like composition

3) Dancing House (Frank Gehry)

Dancing House game guide — Two-hand gestures control soft curved towers; finger spacing influences curvature.

Learning focus: expressive form-making + continuous control
Key interaction: two-hand finger spacing shapes curvature

4) House III (Peter Eisenman)

House III game guide — Rule-based transformations and non-orthogonal compositions (including 45° logic) using ordered two-fist generation.

Learning focus: transformation rules + non-orthogonal composition
Key interaction: two-fist ordering controls cube generation

5) Capsule Tower (Kisho Kurokawa)

Capsule Tower game guide — Fast-paced vertical aggregation; rhythmic fist movement drives modular capsule placement.

Learning focus: vertical composition + modular aggregation
Key interaction: rhythmic hand movement drives generative placement

Project video

If it doesn’t load, use the direct link below.

Direct link: Open video on Google Drive

One-page portfolio

GaoDe one-page project composite with abstract, render, and workflow diagram — Tip: open the full-size version to read the diagrams and captions.

Talk with my AI assistant

Hold the button to record. Release to send. The reply will play aloud and appear in the chat.

Amir Goli