Art · Secondary 4 · Digital Frontiers and New Media · Semester 2

Sound and Performance in Time-Based Media

Investigating the integration of sound design, music, and performance elements in video and new media art.

• MOE Syllabus Outcomes
• MOE: Time-Based Media and Narrative - S4

About This Topic

Sound and Performance in Time-Based Media explores how sound design, music, and performance elements shape video and new media art. Secondary 4 students analyze how audio layers influence viewer perception, distinguishing diegetic sounds, which originate within the film's world, from non-diegetic ones added externally for effect. They examine real works where sound drives narrative, manipulates emotion, or enhances visual storytelling, aligning with MOE standards for Time-Based Media and Narrative.

This topic connects to the Digital Frontiers unit by building skills in multimedia production and critical analysis. Students learn to construct short videos where sound leads the story, fostering creativity and technical proficiency with tools like editing software. It encourages understanding performance as an integrated element, where live actions sync with audio to create immersive experiences.

Active learning shines here because students actively experiment with sound manipulation through collaborative editing and performance trials. Hands-on creation makes abstract concepts concrete, as peers critique and refine pieces together, deepening analysis of sound's narrative power and boosting confidence in new media production.

Key Questions

1. Analyze how sound can manipulate the viewer's interpretation of visual information.
2. Differentiate between diegetic and non-diegetic sound in video art.
3. Construct a short video piece where sound is the primary narrative driver.

Learning Objectives

• Analyze how specific sound design choices, such as reverb or distortion, alter a viewer's emotional response to visual content.
• Differentiate between diegetic and non-diegetic sound by identifying examples in provided video art clips.
• Construct a 60-second video piece where sound elements are the primary drivers of narrative progression.
• Critique the effectiveness of sound and performance integration in a peer's time-based media artwork.
• Explain the relationship between synchronized performance actions and audio cues in creating a cohesive new media piece.

Before You Start

Introduction to Video Editing

Why: Students need basic skills in editing software to assemble and manipulate video footage and audio tracks.

Elements of Visual Storytelling

Why: Understanding how visual components contribute to narrative is foundational before exploring how sound amplifies these elements.

Key Vocabulary

Diegetic Sound	Sound that originates from within the narrative world of the video, such as dialogue spoken by characters or the sound of a car engine.
Non-Diegetic Sound	Sound that is added to the video from outside the narrative world, like a musical score or a narrator's voice-over, used for mood or emphasis.
Soundscape	The complete auditory environment of a place or a piece of media, encompassing all sounds, both natural and artificial.
Foley Artistry	The reproduction of everyday sound effects that are added in post-production to enhance audio quality, such as footsteps or the rustling of clothes.
Performance Art	An art form where the artist's actions or the artist's own body becomes the medium, often presented live and integrated with other media.

Watch Out for These Misconceptions

Common Misconception

All sounds in videos come from the on-screen action.

What to Teach Instead

Diegetic sounds belong to the story world, while non-diegetic enhance from outside. Pair analysis of clips helps students identify and debate examples, clarifying distinctions through shared evidence and discussion.

Common Misconception

Sound only supports visuals and never leads the narrative.

What to Teach Instead

Sound can drive interpretation and emotion independently. Group editing tasks reveal this as students test audio-first builds, observing peer reactions that confirm sound's primary role.

Common Misconception

Music in art is always background filler.

What to Teach Instead

Music shapes mood and structure actively. Performance recordings let students experiment with layers, using class critiques to see how it transforms viewer experience beyond mere accompaniment.

Active Learning Ideas

Pair Analysis: Diegetic vs Non-Diegetic Clips

Pairs watch three short video art clips and log diegetic sounds, like footsteps in scene, versus non-diegetic, such as added music. They discuss how each alters visual meaning and present one example to the class. Extend by remixing a clip's audio.

30 min·Pairs

Small Group: Sound-Driven Narrative Build

Groups film a 30-second silent visual sequence, then layer sounds to shift its story, prioritizing audio as driver. Record performances for sync, edit in software, and screen for class feedback on interpretation changes.

50 min·Small Groups

Whole Class: Live Performance Soundscape

Class performs a collective scene with live sounds and music; half generate diegetic effects, half non-diegetic. Record, review playback to analyze impact, then vote on most effective elements for redesign.

45 min·Whole Class

Individual: Personal Sound Remix

Students select a familiar visual clip, compose original sound design emphasizing performance elements, and export a new version. Submit with reflection on narrative shifts caused by audio choices.

40 min·Individual

Real-World Connections

• Film sound designers at studios like Lucasfilm use sophisticated software to layer diegetic and non-diegetic sounds, creating immersive auditory experiences for blockbuster movies such as the Star Wars franchise.
• Video game developers employ sound artists to craft dynamic soundscapes that respond to player actions, enhancing immersion in games like 'The Last of Us' where environmental sounds build tension.
• Live performance artists, such as those at the Singapore International Festival of Arts, often integrate pre-recorded audio, live music, and synchronized movement to convey complex themes and narratives.

Assessment Ideas

Exit Ticket

Provide students with two short video clips. Ask them to write on an exit ticket: 1. Identify one instance of diegetic sound and one of non-diegetic sound in Clip A. 2. Describe how the sound in Clip B influenced their emotional response.

Peer Assessment

Students present their short video pieces to a small group. Peers use a checklist to assess: Is the sound clearly driving the narrative? Are there at least two distinct sound elements used effectively? Is the performance element integrated with the sound?

Quick Check

Display a still image from a film or video art piece. Ask students to brainstorm in pairs: 'What diegetic sounds might be present in this scene?' and 'What non-diegetic music or sound effects could enhance its mood?' Have pairs share one idea each.

Frequently Asked Questions

What is the difference between diegetic and non-diegetic sound in video art?

Diegetic sound exists within the film's narrative space, audible to characters, like dialogue or ambient noise. Non-diegetic sound, such as voiceover or score, is for the audience only. Teaching this through clip dissections helps students grasp how each manipulates perception, building analytical depth for MOE narrative standards.

How can active learning help students understand sound in time-based media?

Active approaches like group sound layering and live performances give students direct control over audio's impact. They experiment, iterate based on peer feedback, and reflect on changes in narrative drive. This hands-on process makes theoretical concepts experiential, aligning with Secondary 4 skills in multimedia critique and creation.

How does sound manipulate viewer interpretation of visuals?

Sound cues emotion, pace, and meaning; a tense score turns neutral visuals ominous. Students analyze examples to see shifts, then apply in edits. This reveals sound's power, essential for constructing pieces where audio leads, per unit key questions.

What tools support sound design in new media art projects?

Free software like Audacity for editing, GarageBand for music, and DaVinci Resolve for video sync work well in classrooms. Start with simple recordings via phone mics. Guide students to layer tracks, ensuring focus on performance integration for narrative strength.