Memorandum on Multimedia Applications & Virtual Reality

Available on zenodo – DOI : 10.5281/zenodo.17504792 Memorandum on Multimedia Applications & Virtual Reality VR Applications & Constraints Author: Abdelkrim Mebarki Master’s Program in Computer Science – Artificial Intelligence & Applications Department of Computer Science – Faculty of Mathematics and Computer Science University of Science and Technology of Oran November 2, 2025 1. Where Virtual Reality Becomes Essential Virtual Reality is present whenever there is a need to be present in an artificial world or to experience immersion in a virtual environment. A VR-based solution can be applied in entertainment contexts— such as video games or virtual museums—as well as in professional and serious fields, like engineering and medicine. It enables a unique interactive presence with the machine, as in a simulation, or a collaborative presence, such as in multiplayer games or shared creative environments. 1.1. When Does a Computer Solution Qualify as Virtual Reality? 1. When the System Builds or Represents a 3D Spatial Environment ➔ The problem and the solution is modeled in 3D, not only through rough data or 2D interfaces. ➔ Entities have spatial properties: Position, Dimension, Scale, and Behavior in a simulated space. ➔ The user’s point of perception can be placed inside or within this environment. Example – 3D Simulation of a factory to study ergonomics, instead of showing 2D diagrams or tables. 2. When the Solution Involves User-Centered Immersion ➔ The end-user has a presence in (becomes part of) the model, not just an external observer. ➔ The system computes visuals and feedback relative to the user’s position and orientation (head tracking, viewpoint control). ➔ The goal is to make the user “present” in the system, not merely to visualize results. Example – Training simulator: Operator looks around virtual cockpit, instead of viewing 2D flat screen. 3. When interaction is Continuous and Spatial ➔ The user acts directly without intermediate stage within the simulated world, using gestures, movement, or devices that correspond to human natural actions. ➔ Interaction happens in real time and affects naturally the state of the virtual environment. ➔ The user’s actions and inputs produce contextual and spatial feedback (visual, auditory, haptic). Example – Grabbing and moving a virtual object to test its weight or balance. 4. When the Solution Core Is a Simulation, Not only Visualization ➔ The system reproduces dynamic behavior: physics, motion, or procedural changes in real time. ➔ VR is not just static model, it’s a real system that responds to modeled laws or predefined scenarios. ➔ A mere viewer or 3D animation is not VR unless it reacts to the presence and input of the user. Example – Simulating airflow around aircraft model in VR, beyond displaying the shape only. 5. The Objective Is Experiential Understanding or Skill Transfer ➔ The purpose of VR is User experience or training through immersion (Beyond observe & compute): ✓ Training and learning (pilot training, surgery rehearsal) ✓ Design and evaluation (architectural walkthrough, ergonomics study) ✓ Empathy and exploration (psychological exposure therapy, cultural immersion) Example – Feeling what it’s like to operate a crane, not just reading the manual. 6. The System Architecture Integrates the Real-Time Feedback Loop ➔ The system continuously processes in a loop: User act → Sensor in → Env. update → Display out ➔ This “action–reaction” cycle is seamlessly and continuously. ➔ It transforms the digital system into a medium that is interactive and experiential not only static tool. Example – Turning your head immediately changes what you see — with no lag or predefined camera Memorandum on Multimedia Applications & Virtual Reality (A. Mebarki, 2025) Page 2 of 5 1.2. Core Criteria of a VR Computer Solution Condition Description Distinguishes VR from… 3D Spatial Modeling Representation of a world with depth and volume 2D simulation or GUI systems User Immersion The user is “inside” the model, not outside it Data visualization tools Spatial Interaction Direct manipulation in space Mouse–keyboard interfaces Dynamic Simulation Real-time changes based on laws or models Pre-rendered animations Experiential Goal Learn, feel, train, or explore through presence Analytical or computational tools Feedback Loop Continuous real-time input/output cycle Static viewers or recordings A computer solution becomes VR when it transforms a computer computational model into 3D real-time, interactive, immersive experience, enabling the user to be part of the system rather than simply using it. 2. Application Domains 2.1. Art and Attraction Applications (Imagination – Symbolism – Synthesis) This family of VR applications includes creative, cultural, emotional, and symbolic usages of immersion in Computer applications. It emphasizes experience, aesthetic immersion, and emotional engagement rather than physical accuracy. 🖼️1. Entertainment and Gaming ➔ Video games in VR (first-person adventures, role-playing, strategy) ➔ Theme park attractions with immersive rides and interactive stories ➔ VR eSports and tournaments offering full-body engagement 🏛️ 2. Culture and Heritage ➔ Virtual museums and art galleries with 3D navigation and guided tours ➔ Reconstruction of historical sites (ancient cities, monuments, ruins) ➔ Immersive storytelling bringing history or mythology to life 🎭 3. Performing Arts ➔ VR theater and dance performances viewed from inside the stage ➔ Immersive concerts and 360° music experiences ➔ Virtual choreography for dance creation and education 🎬 4. Cinema and Visual Media ➔ 360° immersive films and VR documentaries ➔ Virtual film sets for previsualization and scene design ➔ Artistic VR installations in digital exhibitions  5. Wellness, Tourism, and Emotional Experiences ➔ Virtual tourism (exploration of exotic or inaccessible places) ➔ Meditation and relaxation environments in VR ➔ Art therapy and creative expression through virtual painting/sculpture 2.2. Engineering Applications (Real – Reproduction – Simulation) This family of VR Applications focuses on precision, performance, training, and problem-solving emphasizing realistic modeling, data-driven reproduction, and interactive simulation of real systems. 🏗️ 1. Industry and Manufacturing ➔ Virtual prototyping (designing and testing before production) ➔ Assembly and maintenance simulation for workers ➔ Ergonomics and safety training in realistic environments 🚗 2. Transport and Mobility ➔ Driving and flight simulators for training pilots and drivers ➔ Navigation system testing and vehicle design validation ➔ Maritime and space operation simulators 🧠 3. Medicine and Healthcare ➔ Surgical training simulators and operation planning ➔ Physical and cognitive rehabilitation in immersive settings ➔ Exposure therapy for phobias, PTSD, or anxiety Memorandum on Multimedia Applications & Virtual Reality (A. Mebarki, 2025) Page 3 of 5 ➔ Patient education through 3D anatomical exploration 🏗️ 4. Architecture and Construction ➔ Virtual walkthroughs of buildings before construction ➔ Urban planning simulations with lighting, noise, and traffic models ➔ Collaborative design reviews in shared VR spaces 🔬 5. Science and Research ➔ Visualization of complex data (molecules, galaxies, climate models) ➔ Experimental simulations for physics, biology, or engineering ➔ Virtual laboratories for remote scientific collaboration 🎓 6. Training and learning ➔ Learning in remote and distant locations ➔ Explanation of hard concept using virtual simulations and builind ➔ Training in tough situations and risk terrains 🎨 Art & Attraction → Imagination, Emotion, Expression → Experience, Culture, Creativity ⚙️ Engineering → Realism, Precision, Efficiency →Simulation, Training, Optimization 3. 🕰️ Evolution of VR – From Interfaces to Immersive Worlds 1960s – Birth of Human–Machine Interaction The Mouse – First Behavioral Interface (1963): Invented by D. Engelbart, computer mouse introduced a new mode of human-computer interaction precursor to interactive systems. Around the same time, I. Sutherland developed the “Ultimate Display” concept — a theoretical framework describing a room where computer controls the existence of matter — an early vision of VR. In 1968, Sutherland created the first head-mounted display (HMD), nicknamed The Sword of Damocles, offering primitive visual immersion using wireframe graphics. 1970s – Foundations of Visual Immersion Immersive Helmets and Head-Tracking Displays appeared in military/research labs. These early devices used cathode-ray screens and mechanical tracking adjusting visuals to head movement. Concepts like telepresence and augmented reality began to emerge in writings. The focus was still on scientific visualization and simulation, not entertainment. 1980s – The Age of Interaction and Sensory Expansion Introducing Data Gloves (VPL DataGlove by J. Lanier in 1984) for gesture-based haptic interaction. HMD improved in quality and became available commercially (VPL EyePhone, NASA View). J. Lanier popularized “Virtual Reality” via VPL Research, 1st company commercializing VR devices. Parallel advancements in 3D graphics and simulation software improve interactive virtual worlds. 1990s – From Research Labs to Industrial Prototypes CAVE Systems (Cave Automatic Virtual Environment, 1992) – Developed at Illinois University, immersive rooms – multiple projection walls surrounding users in real-scale VR spaces. VR found applications in aerospace, automotive design, and scientific visualization. However, VR systems were expensive, bulky, and limited in realism slowing down public adoption. Meanwhile, the gaming industry began experimenting with early VR headsets (Sega VR, Virtual Boy), though the technology wasn’t ready for mass use. 2000s – Digital Graphics and Popularization With the rise of powerful GPUs and 3D engines, environments became more realistic and accessible. Consumer-grade headsets and 3D environments (Google Earth VR, Second Life, 3D cinemas). The concept of the “Metaverse” re-emerged, combining social interaction and shared virtual spaces. VR moved gradually from research/defense to engineering and entertainment. Memorandum on Multimedia Applications & Virtual Reality (A. Mebarki, 2025) Page 4 of 5 2010s – The Renaissance of Virtual Reality Breakthroughs in display resolution, motion tracking, and affordability reignited VR development. Key innovations: Oculus Rift (2012 Kickstarter revolution), HTC Vive, PlayStation VR (2015–2016), Google Cardboard (mobile VR democratization) VR ecosystems expanded: Development platforms (Unity, Unreal) and motion capture systems. The idea of social VR and shared immersive spaces gained power. 2020s – Everyday VR and Convergence with AR Mixed Reality (MR) and Augmented Reality (AR) merge with VR creating Extended Reality (XR). VR becomes a tool for healthcare, education, architecture, and remote collaboration. Consumer devices (Meta Quest, Pico, Apple Vision Pro) make VR wireless, lighter, more integrated. VR content enters media mainstream — interactive cinema, virtual tourism, immersive journalism. Platforms (VirtualBox, video glasses, immersive browsers) make VR part of everyday digital life. 4. The Virtuality Continuum: From Reality to Virtuality The Virtuality Continuum (VC) is a conceptual framework that defines the spectrum of lived environments ranging from the completely physical real to the completely artificial virtual. It was first formalized by Milgram & Kishino (1994) to classify different kinds of mixed and mediated realities according to the degree of artificial content integration. 4.1. Physic Real Environment (RE) At the left end of the continuum lies the Physic Environment (called Real Environment), where perception is based solely on natural sensory inputs — what we perceive (see, hear or touch) in the physical world without any digital mediation. ➔ Example – Observing a real surgical operation or interacting with physical machinery. • Characteristics: No artificial components, full natural sensory fidelity, and direct interaction. 4.2. Augmented Reality (AR) Moving rightward, Augmented Reality introduces virtual entities (text, 3D objects, data overlays) into the physical real world. The user still perceives the physical real environment, but his perception is “augmented” with computer-generated information that enhances perception and actions. ➔ Example –AR-based maintenance instructions in engineering, Mobile AR apps like Pokémon Go. • Scientific goal: Integrate virtual data seamlessly with physical real-world perception while preserving spatial and temporal semantic coherence. 4.3. Mixed Reality (MR) Mixed Reality represents the middle zone of the continuum, both real and virtual entities coexist allowing two-way dynamic interaction: physical objects can influence virtual ones and vice versa. ➔ Examples – Collaborative design systems blending physical prototypes with virtual extensions. • Key feature: Continuous blending between physical and digital spaces, supported by advanced spatial mapping and sensor fusion. 4.4. 4. Augmented Virtuality (AV) As we move further to the virtual side, Augmented Virtuality describes environments being mostly virtual incorporating real-world elements — such as live video feeds, scanned 3D objects, or real users represented inside a virtual space. ➔ Examples – Telepresence systems embedding live camera feeds into virtual environments. • Scientific interest: Preserving realism and human presence inside synthetic worlds. 4.5. 5. Virtual Environment (VE) / Virtual Reality (VR) At the far right of the continuum lies Virtual Reality, a fully synthetic artificial environment generated digitally. All sensory input is replaced by virtual stimuli, which produces the illusion of being physically present in a computer-generated world. ➔ Examples: Memorandum on Multimedia Applications & Virtual Reality (A. Mebarki, 2025) Page 5 of 5 ✓ Immersive simulators for flight, surgery, or education, 3D virtual worlds and games using HMD. • Core properties: Immersion, interaction, and real-time sensory feedback. The Virtuality Continuum provides a theoretical foundation for levelling, categorizing and comparing immersive technologies. It clarifies: • The degree of virtuality (how much of the environment is artificial, computer-generated). • The direction of mediation (physical → virtual or virtual → physical). • The interaction balance between physical and digital components and entities. In research and development, the continuum helps different actors define design goals, interaction paradigms, and evaluation metrics for emerging XR (Extended Reality) systems and applications, encompassing AR, MR, AV, and VR under a unified conceptual umbrella. The Virtuality Continuum illustrates that reality and virtuality are not opposites, but rather two poles of a continuous spectrum where real and digital components can be combined in various proportions to create immersive, interactive, and intelligent configurable environments. 5. Technical Constraints for MMA (Multimedia and Mixed/Augmented Reality) Application Development Developing a Multimedia or Mixed Media Application (MMA) requires taking into account several technical dimensions and functional specifications. These constraints define how the information is represented, processed, transmitted, and experienced by the end-user. 5.1. 1. Specification by Dimension Specification Option Content Underlying Technology / Process Textual Symbols (Characters) Written language, textual data Coding, character encoding (e.g., UTF-8, Unicode) Visual Images Models, graphic representation 2D/3D modeling, rendering, computer graphics Auditory Sounds Speech, music, effects Signal processing, audio encoding Haptic Touch Force feedback, tactile perception Physics simulation, haptic modeling, mechanics Olfactory/Gustatory Smells/Tastes Chemical sensations Olfactory modeling, sensory chemistry Hypermedia Links/Tags Cross-referenced content Markup & metadata coding (HTML, XML, JSON-LD) 5.2. 2. Specification by Functionality Specification Option Tools / Methods Disciplines Involved Interactivity Data prediction, adaptive behavior Machine learning models, behavioral analytics Artificial Intelligence, Cognitive Psychology Communication scenarios (use cases) Case studies, scenario design Human-Computer Interaction Orphan cases (unforeseen user choices) Decision trees, state graphs Graph theory, Optimization Level of detail Resolution management Abstraction, Refinement models Broadcasting Local DB access, client-server Networking, Data management Distributed Cloud services, synchronization Distributed computing Security & Authenticity Content authenticity Watermarking, cryptography Information security Access rights Authentication, permissions Cybersecurity, Data governance Copyrights Ethical and legal compliance Digital Ethics, Intellectual Property Goals Real-time execution Time optimization Real-time systems design On-demand access Efficient data retrieval Storage optimization Optimal resource use Space/time trade-offs Systems engineering Rendering Multi-dimensional output Formats, display peripherals Visualization & rendering pipelines Interfacing HMI Usability testing, accessibility Ergonomics, Behavioral science An MMA application must integrate multi-sensory input and output, maintain security and ethical integrity, and ensure interactivity and performance. Successful implementation requires a crossdisciplinary approach, combining expertise in: - Computer Science (for graphics, AI, signal processing), - Engineering (for mechanics, electronics, physics), - Cognitive Science (for perception, usability), - Ethics / Law (for security, copyrights).