Memorandum on Multimedia Applications & Virtual Reality

Available on zenodo – DOI : 10.5281/zenodo.17627209 Memorandum on Multimedia Applications & Virtual Reality Modeling VR Worlds Author: Abdelkrim Mebarki Master’s Program in Computer Science – Artificial Intelligence & Applications Department of Computer Science – Faculty of Mathematics and Computer Science University of Science and Technology of Oran November 16, 2025 To perceive a virtual world, the user needs entities to be rendered on devices. But before rendering can happen, that world must first exist in a digital form — as a coherent structured representation of all its objects, spaces, and interactions. What does this mean? ➔ The virtual world the user sees, hears, or feels does not exist physically (No external existence). ➔ It must be created, described, and encoded in a form the computer can understand and process. ➔ This internal digital description is what we call the VR World Representation or VR Modeling. To create a Virtual Reality (VR), we must first model the virtual world — that is, design and implement its digital representation so that it can later be rendered and experienced through immersive devices. 1. Purpose of Modeling It defines the structure, behavior, and meaning of the elements composing the virtual world. Without accurate models, rendering (visual, auditory, haptic) cannot produce coherent or immersive experience. 2. Types of Rendering Once the world is modeled, the VR system can render it through various sensory channels: • Visual Rendering: Transforms 3D models into 2D images displayed to the user. • Auditory Rendering: Spatializes sounds based on object position and user orientation. • Haptic Rendering: provides tactile or force feedback corresponding to object interaction. VR applications combine all three to achieve multisensory immersion. 3. Modeling Constraints A valid virtual world model has to satisfy 6 constraints to ensure realism and coherence. When these constraints are fulfilled, the user perceives the virtual world as credible and become immersed in it: 1. Geometric consistency – Objects have realistic shape, size, and proportions. 2. Spatial coherence – Positions, distances, and orientations respect physical logic. 3. Physical plausibility – Interactions follow basic physical rules (gravity, collision, resistance). 4. Temporal continuity – Smooth and consistent object movements and transformations over time. 5. Semantic consistency – Each object’s meaning and function are preserved in context. 6. Computational efficiency – the model must be optimized for real-time rendering performance. 4. The Visual Model: The Core The visual model is the foundation of any virtual world. The other modalities (such as sound and haptics) are added on top to enrich user perception and enhance immersion. The design of the visual model typically involves four key stages: 4.1. The Geometric Modeling This model defines the shapes and the structure of objects and spaces using mathematical representations (meshes, surfaces, volumes) to describe the world geometry. 4.2. The Kinematic Modeling It defines the movements and the transformations (translation, rotation, scaling) that affect virtual entities over time. This step introduces animation and user-object motion. 4.3. The Topological Modeling This model defines spatial relationships between the virtual objects — adjacency, connectivity, inclusion. It ensures spatial continuity (e.g., doors connected to rooms, walls enclosing spaces). 4.4. The Semantic Modeling Defines the behavioral and interaction rules: Collisions, interferences, logical relations, user interactions giving meaning to actions. e.g. what happens when the user touches, moves, or manipulates an object. Memorandum on Multimedia Applications & Virtual Reality (A. Mebarki, 2025) Page 2 of 3 Modeling Type Purpose Impact on Immersion Geometric Defines shapes and spaces Supports visual realism Kinematic Controls motion and animation Increases interactivity Topological Structures spatial relationships Enhances spatial presence Semantic Defines interaction logic Builds meaning and coherence Auditory/Haptic Extend sensory realism Deepen immersion 5. Integration of Multisensory Models Once the visual core is ready, it’s time to integrate: • Sound models (spatialized, reactive audio cues) • Haptic models (touch, texture, or force feedback) These additional layers enhance the sense of presence and interactivity, the two pillars of immersion. 5.1. 🎧✋ Integration of Sound and Haptic Modalities within the Modeling Stages The visual model is the structural foundation, sound and haptic modalities are integrated progressively in the final design — mostly during the semantic stage, but they rely on the information built in the previous stages: Geometric Modeling Base for spatial references • Sound sources and haptic zones need positions and shapes to exist in the virtual space. • Example: defining where a loudspeaker or a vibrating object is located. Sound and haptics are not yet active here, but geometry provides the necessary coordinates. Kinematic Modeling Movement and transformation of sensory sources • When objects move, their associated sounds and haptic feedback have to follow. • Example: a drone’s buzzing sound moves with the drone’s trajectory. Sound/haptic parameters begin to depend on motion data. Topological Modeling Spatial and adjacency logic for sensory propagation • Defining how sound travels (through doors, between rooms) and where haptic effects apply. • Example: footsteps’ sound changes between rooms; vibration passes through connected surfaces. Topology defines the logic of sensory continuity and propagation. Semantic Modeling Integration and activation of multisensory interactions • This is where sound and haptic models are truly implemented and controlled, Defining rules, triggers, and responses linking user actions to multisensory outputs • Example: When the user touches an object → vibration + impact sound Full multisensory integration happens here. Modeling Stage Role for Sound & Haptics Integration Level Geometric Defines spatial location of sensory sources Basic Kinematic Links sensory cues to object movement Moderate Topological Controls sensory propagation and adjacency Moderate Semantic Defines interaction rules and sensory feedback Full integration Sound and haptics are anchored in the geometry, animated in the kinematics, propagated in the topology, and activated in the semantics. 6. 🧩 Classification of Objects in Virtual Worlds  Rigid and Non-Deformable Solids 🧱 Objects that do not deform — Only undergo rigid transformations (translation, rotation, scaling). ➔ Represent static or mechanical elements such as buildings, furniture, or vehicles. Fundamental for geometry, physics, and collisions. Approach – Polygonal / Mesh-Based Modeling + Rigid-Body Physic → No vertex deformation.  Articulated Solids ⚙️ Systems of rigid parts connected by joints (axes of rotation, sliding, or hierarchy). ➔ Used for skeletal or mechanical structures: avatars, robots, doors, or arms. Memorandum on Multimedia Applications & Virtual Reality (A. Mebarki, 2025) Page 3 of 3 Essential for animation and kinematics. Approach – Skeletal Modeling with Forward/Inverse kinematics (FK/IK) + joint constraints.  Deformable Solids 🧬 Objects changing shape or volume according to physical or procedural deformation laws. ➔ Usage: Common for organic materials (skin, muscles, clothes, rubber, vegetation). Crucial for realism in soft materials and living beings. Approach – Soft-Body Physics, Finite Element Modeling (FEM), Blendshape Animation → Vertex deformation.  Non-Solids (Continuous Media) 💧 Matter without a fixed shape or volume: fluids, gases, fire, smoke, or particle systems. ➔ Used to simulate natural environments, weather, or dynamic visual effects. Enhanced through GPU-based simulation and shaders. Approach – 🌫️ Particle Systems, Fluid Dynamics (SPH, Navier–Stokes), Volumetric rendering.  Abstract or Procedural Entities 🌀 (Modern Extension) Not physical, logical, interactive, data-driven (e.g., holograms, force fields, UI elements, or AI agents). ➔ Increasingly used in XR, data visualization, and interactive interfaces. Emerging category — bridges modeling and interaction logic. Modeling Approach – Procedural modeling, event-driven logic, or parametric generation. Category Behavior Examples Modeling Approach Rigid Solids No deformation Buildings, tools Mesh modeling + rigid-body physics Articulated Solids Rigid parts with joints Robots, avatars Skeletal modeling + kinematics Deformable Solids Elastic deformation Skin, cloth Soft-body / FEM / blendshapes Non-Solids No fixed volume Water, fire, smoke Particle / fluid dynamics Abstract Entities Logical / procedural UI, AI, holograms Procedural / event-driven 7. 🧮 Formats of Geometric Representation in Virtual Worlds Back to the geometric model, the core of the virtual world representation: Geometric structures can be represented using two main approaches: Primitive-based models or Analytical models.  Primitive-Based Representation 🧱 Built from basic geometric elements combined to form complex shapes. This is the most common format used in real-time 3D and VR applications including : Vertex-based: Defined by 3D points (vertices) in space. Edge-based: Defined by connections between vertices (edges). Face-based: Defined by polygonal surfaces (triangles, quads, meshes). These entities (polygonal and meshes) are suitable for interactive rendering and collision detection.  Analytical Representation 📈 Described using mathematical equations instead of discrete primitives focusing on precision and continuous surfaces. This includes: Parametric models – Surfaces defined by mathematical functions (e.g., Bézier, B-splines, NURBS). Used for CAD, organic shapes, and smooth curves. Non-parametric models – Defined implicitly or explicitly by equations without parameters. 🧮 Example: Implicit: F(x,y,z)=0 | Explicit: z=f(x,y) Used for volumetric forms, terrains, or implicit surfaces (metaballs). Equation-based and procedural models suit scientific visualization, simulation, or precise design. Representation Basis Typical Use Strengths Limitations Primitivebased Vertices, edges, faces Games, VR, real-time scenes Fast, intuitive, GPUfriendly Less precise, discrete Analytical Mathematical equations CAD, simulation Smooth, exact Harder to render in real time