PodHost.ai: An Experiment

Because we all need someone to talk to.

I work with AI APIs quite a lot while working with clients to build their AI-assisted applications, and our own internal software at Riff. But I hadn’t had a need to interact with the OpenAI Realtime API, which can enable a speech-to-speech conversation with AI. As an avid podcast listener, the idea of having a podcast hosted by AI in which I was the guest seemed like a natural use-case to test out the APIs.

And so PodHost.ai was born. It’s a mobile-friendly web app that allows you to record a podcast (aka an audio recording) while an AI host guides the conversation. It features semantic turn-taking (AI will start speaking when it detects that your “turn” is done, based on context), built-in recording, voice selection, input balance control, and “episode” configuration - you set the topic, provide the docs, and your host will guide the conversation.

I had some interesting learnings about the realtime API along the way:

• The AI voice sounds quite exuberant unless you provide it instructions to chill out, e.g. “be boring”, which is what we’re doing with PodHost.ai

• There’s a turn_detection property on the session request which let’s you configure how the AI responds:

  • server_vad - AI will respond N seconds after you stop speaking

  • semantic_vad - AI will respond after it detects you’re done speaking, based on semantics

• Some of the WebRTC realtime session configuration can be updated after starting the session

  • For example, in the PodHost.ai control panel we have an “enable/disable reply” button which updates the session to prevent AI from responding while still gathering the context of what is being said

• In my testing setup, AI speech was feeding back into itself and being continually interrupted with its own speech

  • It may be my recording setup, but my fix for this was to automatically mute/unmute the mic with the AI starts/stops speaking

  • Conveniently we get events from the WebRTC session for when the audio buffer starts and stops, so this is easy to handle

We are considering whether to incorporate ElevenLabs for more realistic and customizable AI speech, but that’s a task for another day! In the meantime, join the waitlist at PodHost.ai to get updates on new features.

What will your podcast be about? Check it out and let us know what you think!