← Back to Main

How to well-design semantical interaction - which users actually care??

2/6/2025, 10:00:13 PM | Jeanyoon Choi

Original Notes (Pre-LLM)

Semantical yet Smooth Interaction

Most of the existing interactions: are non-semantical (Bodily movement, Rafael Lozano Hemmer, Just simple scrolling)

Why?

Semantical interaction (i.e. interaction where the user provides meaning/interacts within context) makes it hard to expect active user engagement within an exhibition context

User 쭈뼛쭈뼛 혹은 비협조적/Dummy Text Input

Ex. Chunk of Text Input, Voice Interaction

Possible workaround:

Make user's non-active/passive interaction itself as part of the semantical interaction itself (Non-active/passive interaction itself is an interaction)

As technology advances, Face Tracking is a strong example of such interaction

On-boarding 1: Strong Intro (V-Effekt): Immediately grabs user's attention &

On-boarding 2: Gradual Intro: Make the user relax, Smooth Incremental Engagement

On-boarding 3: Strong Background Storytelling/Pre-given context

Dadaism or Interaction-Dimensional Transformation: Input A, Output AAAAAAAA some kind of dadaistic exaggeration or fun/unexpected input-output relationship (dimensional transformation)

Can be for example transforming non-semantical interaction (low information input) into very highly semantical output (Amplifying/Exaggerating information)

Can be used for VUI-based interaction: Simple voice input, Passive murmuring → Gives a hell lot of context and transforms into strong/affirmative output

Queuing experience: Have a long queue outside (People will interact harder as they want to compensate for their queuing - and know that thereis a value in the interaction)

In this case: Making non-active/passive interaction as the part of the interaction, as well as amplifying the user's voice input interaction (Normally LLM-VUI situation: user leads the context, but in this case, LLM leads the context!), as well as giving strong intro - user doesn’t know there will be interaction when they are trying to take some rest, but apparently, the interaction starts unexpectedly as user lies down on the bed!

English Version (LLM-Generated)

Creating compelling and meaningful interactions in art and technology can pose a challenge, as many existing types of engagements are non-semantical in nature. Take, for example, bodily movement-based interactions like those found in Rafael Lozano-Hemmer's works or simple scrolling on digital devices. The difficulty with semantical interactions, where users provide meaning through contextual engagement, lies in the unpredictability of active user participation, especially within the confines of an exhibition. Often, users find themselves either hesitant or uncooperative, akin to providing "dummy" text input when prompted.

To address this issue, a potential strategy is to integrate the user’s non-active or passive participation as a meaningful part of the interaction. This concept is well-illustrated with face tracking technology, which facilitates engagement without relying on overt user actions. Effective user onboarding can further enhance this process – starting with a strong introduction that immediately captures attention, followed by a gradual introduction to ease the user into a comfortable state of engagement. Complementing this with rich storytelling or a pre-established context can deepen the user’s connection to the interaction.

Incorporating elements of Dadaism, or interaction-dimensional transformation, adds an element of unpredictability and fun to the experience. Here, non-semantical interactions—like low-information inputs—are exaggerated into highly semantical outputs, amplifying the meaning beyond the user's initial actions. This approach can be particularly effective in voice-user interface (VUI)-based interactions, where even passive murmuring can be transformed into assertive outputs, thus creating a dynamic and engaging experience.

Moreover, leveraging the psychology of queuing can enhance user engagement. When users endure a long queue, they often engage more deeply with the interaction to compensate for their wait time, recognizing the value embedded within the experience. By considering these tactics, the interaction narrative transforms: users become part of an unexpected experience that draws them in just when they least anticipate it, even when they are simply taking a moment to rest, such as lying down on a bed. Through such innovative approaches, interaction is redefined as a seamless blend of passive and active input that captivates and rewards user engagement.

Korean Version (LLM-Generated)

의미 있는 매끄러운 상호작용

대부분의 기존 상호작용은 의미 없는 방식으로 이루어집니다. 이는 신체적 움직임이나 라파엘 로자노 헤머의 작업, 단순한 스크롤 등과 같은 형태로 나타납니다. 그 이유는 무엇일까요? 전시회와 같은 문맥 안에서 사용자가 의미를 부여하고 상호작용하는 '의미 있는 상호작용'은 사용자의 적극적인 참여를 예상하기 어렵게 만듭니다. 사용자가 쭈뼛쭈뼛 거리거나 비협조적일 수도 있고, 더미 텍스트 입력처럼 될 수 있습니다. 예를 들면 텍스트 묶음 입력이나 음성 상호작용 등이 있습니다.

이 문제를 해결하기 위한 가능한 방안으로는 비활성적이거나 수동적인 사용자의 상호작용 자체를 의미 있는 상호작용의 일부로 만드는 것입니다. 즉, 비활성적/수동적인 상호작용 자체도 상호작용이 될 수 있습니다. 기술이 발전함에 따라 얼굴 추적과 같은 강력한 상호작용 예시가 생겨나고 있습니다.

도입 단계 1: 강렬한 도입부(V-Effekt)를 통해 사용자의 관심을 즉시 끌어 냅니다.

도입 단계 2: 점진적인 도입을 통해 사용자가 긴장을 풀고 점차 참여하게 만듭니다.

도입 단계 3: 강력한 배경 스토리텔링과 사전 제공된 문맥이 제공됩니다.

다다이즘이나 상호작용 차원의 변환도 적용됩니다. 예를 들어, 입력 A가 과장되거나 예상치 못한 입력-출력 관계로 증폭되는 방식입니다. 즉, 정보가 적은 비의미적 상호작용을 매우 의미 있는 출력으로 전환할 수 있습니다. 음성 사용자 인터페이스(VUI) 기반의 상호작용에서는 단순 음성 입력이나 수동적인 중얼거림이 강력하고 긍정적인 출력으로 변환될 수 있습니다.

대기 경험 역시 중요합니다. 사람들이 길게 늘어선 줄을 서게 되면, 더 열심히 상호작용하게 되며 그 상호작용에 가치가 있다는 것을 알게 됩니다.

이 경우, 비활성적/수동적인 상호작용을 상호작용의 일부로 만들고 사용자의 음성 입력 상호작용을 증폭하며(일반적으로 사용자 주도의 문맥이지만, 이번 경우에는 LLM이 문맥을 주도합니다), 강렬한 도입부를 제공하는 것이 중요합니다. 사용자가 쉬려고 할 때 예상치 못하게 상호작용이 시작되며, 침대에 누웠을 때 상호작용이 시작될 수 있습니다.