Speaker: Jan Zdenek.
Abstract: Semantic speech retrieval systems typically focus on semantic relevance, while speaker identity can provide an additional retrieval constraint in scenarios where users seek semantically relevant speech from a specific speaker. This paper investigates speaker-conditioned semantic speech retrieval, where retrieved documents must satisfy both semantic relevance and speaker consistency constraints. The proposed retrieval framework integrates semantic and speaker information within a shared embedding space. The approach is evaluated against semantic-only and cascaded retrieval baselines on the SLUE Phase-2 SQA5 dataset and an evaluation set derived from the Fisher corpus. Experimental results show that incorporating speaker information improves retrieval performance, while the proposed fused semantic-speaker retrieval model outperforms independently optimized cascaded approaches. These findings indicate that joint semantic-speaker modeling is effective for speaker-conditioned retrieval from speech databases.