Latency Is a Budget. Humanlike Is the Goal. — Jesse Hall, LiveKit

Well, let's begin. Most of the agents Most of the agents we build we build we build today run in the today run in the today run in the background, background, background, right? They write right? They write right? They write code for us. They are code for us. They are code for us. They are conducting conducting conducting research. You give research. You give research. You give them a task, them a task, them a task, you leave, you you leave, you you leave, you come back, and no come back, and no come back, and no one cares if the one cares if the one cares if the background background background agent takes 10 seconds or agent takes 10 seconds or agent takes 10 seconds or even 10 minutes even 10 minutes even 10 minutes longer, right? We are longer, right? We are longer, right? We are used to this. But at used to this. But at used to this. But at some point some point some point last year, you last year, you last year, you probably probably probably interacted with a different interacted with a different interacted with a different type of agent—a type of agent—a real- real- real- time agent. Maybe it was time agent. Maybe it was time agent. Maybe it was refilling a prescription at refilling a prescription at refilling a prescription at the pharmacy or talking to the pharmacy or talking to the pharmacy or talking to the airline. I the airline. I the airline. I recently spoke to someone like this recently spoke to someone like this recently spoke to someone like this when I called a when I called a when I called a car dealership. And during car dealership. And during car dealership. And during that call, something that call, something that call, something happened. You probably happened. You probably happened. You probably did the most did the most did the most human thing human thing human thing possible. You killed possible. You killed possible. You killed this AI. And that this AI. And that this AI. And that voice agent " voice agent " broke", it broke", it broke", it felt felt felt unnatural, right? unnatural, right? If you ask anyone in the industry, you'll anyone in the industry, you'll anyone in the industry, you'll probably get a probably get a probably get a similar answer on how to similar answer on how to similar answer on how to fix it. This is fix it. This is fix it. This is speed: faster speed: faster speed: faster models, lower models, lower models, lower latency, reduce latency, reduce latency, reduce time by 100 milliseconds.

time by 100 milliseconds. But this is the wrong But this is the wrong answer. Delay answer. Delay is a budget, and I'll is a budget, and I'll is a budget, and I'll tell you exactly what that tell you exactly what that tell you exactly what that budget is. But the goal is budget is. But the goal is budget is. But the goal is completely different. The goal is to make completely different. The goal is to make completely different. The goal is to make voice voice voice AI as AI as AI as human-like as possible. human-like as possible. An agent that listens An agent that listens while you speak, that knows while you speak, that knows while you speak, that knows when you've when you've when you've finished a thought and finished a thought and finished a thought and when you're still thinking, and when you're still thinking, and when you're still thinking, and that adapts that adapts if you change your if you change your if you change your mind. mind. mind. Speed ​​alone won't Speed ​​alone won't Speed ​​alone won't do it. What is do it. What is do it. What is needed can be needed can be needed can be achieved by choosing achieved by choosing achieved by choosing models and models and models and configuring the configuring the configuring the processing pipeline. And this is the processing pipeline. And this is the processing pipeline. And this is the first problem first problem first problem we will face. No we will face. No we will face. No one tells you how to one tells you how to one tells you how to choose the right choose the right choose the right models. models. models. Leaderboards won't Leaderboards won't Leaderboards won't help you. Vendors help you. Vendors help you. Vendors definitely won't definitely won't definitely won't help you. So help you. So help you. So let's discuss let's discuss let's discuss how to solve this how to solve this how to solve this problem. Quick problem. Quick problem. Quick show of hands: who among show of hands: who among show of hands: who among you has created a you has created a you has created a voice agent and voice agent and voice agent and provided it to provided it to provided it to real real real users? Good. users? Good. users? Good. Of course. Some of Of course. Some of Of course. Some of you have experience and know the you have experience and know the you have experience and know the challenges you face challenges you face when deploying a when deploying a real- real- real- time agent. Another time agent. Another time agent. Another poll: who here poll: who here poll: who here used used ChatGPT's advanced voice mode to ChatGPT's advanced voice mode to communicate with him?

communicate with him? communicate with him? Yes, many of us Yes, many of us Yes, many of us have used it, right? have used it, right? have used it, right? OpenAI chose LiveKit for OpenAI chose LiveKit for OpenAI chose LiveKit for its work. LiveKit is an its work. LiveKit is an its work. LiveKit is an open-source framework and open-source framework and real-time infrastructure for real-time infrastructure for creating the creating the creating the most human-like most human-like most human-like voice agents. voice agents. Agents who can Agents who can hear, see, and hear, see, and hear, see, and talk to the talk to the talk to the interlocutor. interlocutor. So, not just voice, but So, not just voice, but video, data, and video, data, and video, data, and even physical AI. even physical AI. But since voice is complicated, is complicated, is complicated, we will we will we will focus on focus on focus on voice in this conversation. So, we voice in this conversation. So, we voice in this conversation. So, we will look at voice will look at voice will look at voice agents. They essentially agents. They essentially have three models have three models have three models working together as working together as working together as one system. These are one system. These are speech-to-text conversion, speech-to-text conversion, large language model ( large language model ( LLM), and text-to-speech synthesis LLM), and text-to-speech synthesis LLM), and text-to-speech synthesis . So, three . So, three . So, three components. I must components. I must components. I must also admit that there also admit that there also admit that there are are are real-time models. real-time models. real-time models. Real- Real- Real- time models are a newer time models are a newer time models are a newer concept. They are concept. They are concept. They are getting getting getting better and better, but they are better and better, but they are better and better, but they are not suitable for not suitable for not suitable for every situation. And every situation. And every situation. And the fact that the fact that the fact that real- real- real- time models exist only adds time models exist only adds time models exist only adds complexity to our complexity to our complexity to our task of determining task of determining task of determining which model is best which model is best which model is best for our for our for our voice pipeline.

voice pipeline. So, here's what you So, here's what you can choose from. Hundreds of can choose from. Hundreds of can choose from. Hundreds of models, dozens of models, dozens of models, dozens of providers, providers, providers, raise your hands, who raise your hands, who raise your hands, who chose a model just chose a model just chose a model just because they saw it at the because they saw it at the because they saw it at the top of the top of the top of the leaderboard? Yes, leaderboard? Yes, leaderboard? Yes, most of us. But most of us. But most of us. But here's why it doesn't work here's why it doesn't work here's why it doesn't work for for for real-time agents. real-time agents. Leaderboards Leaderboards evaluate components, evaluate components, evaluate components, and you release a and you release a and you release a cascade, a pipeline. cascade, a pipeline. Let's say your Let's say your client wants to client wants to client wants to make an appointment make an appointment make an appointment on Tuesday, the 3rd. on Tuesday, the 3rd. on Tuesday, the 3rd. Your Your speech recognition system hears " Tuesday, the 30th." From Tuesday, the 30th." From Tuesday, the 30th." From this point on, it this point on, it this point on, it doesn't matter how doesn't matter how doesn't matter how smart your language smart your language smart your language model is: it confidently model is: it confidently model is: it confidently books the wrong books the wrong books the wrong day, each error day, each error day, each error accumulates further, and accumulates further, and accumulates further, and the lower levels of the system the lower levels of the system the lower levels of the system don't even know about it don't even know about it don't even know about it . Those . Those leaderboard scores are obtained on leaderboard scores are obtained on clean audio in a quiet clean audio in a quiet clean audio in a quiet room. But your room. But your room. But your customers are calling customers are calling customers are calling on speakerphone on speakerphone on speakerphone in traffic jams, with kids in the in traffic jams, with kids in the in traffic jams, with kids in the back seat.

back seat. back seat. So those So those So those leaderboards were never leaderboards were never leaderboards were never seen by your seen by your seen by your users. users. users. There's even a name for this: the There's even a name for this: the leaderboard trap. The assessment leaderboard trap. The assessment measures the model, but measures the model, but measures the model, but again, you are again, you are again, you are starting a conversation. starting a conversation. Everyone who builds Everyone who builds voice agents voice agents voice agents asks: which asks: which asks: which recognition model, LLM or recognition model, LLM or recognition model, LLM or speech synthesis, is speech synthesis, is speech synthesis, is best? But the best? But the best? But the real question is: real question is: real question is: which stack to choose for which stack to choose for which stack to choose for implementation? These are two implementation? These are two implementation? These are two completely different completely different completely different issues, and until issues, and until issues, and until today, the today, the today, the leaderboard was only for leaderboard was only for leaderboard was only for one of them. We'll one of them. We'll one of them. We'll talk about that a talk about that a talk about that a little later. little later. little later. Let's say you Let's say you Let's say you fall into this trap fall into this trap . You choose models . You choose models . You choose models by rating and by rating and by rating and start measuring start measuring start measuring performance. This is the performance. This is the performance. This is the next problem. You are next problem. You are probably probably probably not measuring what you need. not measuring what you need. Your Your speech synthesis provider speech synthesis provider speech synthesis provider reports the time to reports the time to reports the time to first first first byte—it byte—it byte—it looks great on the dashboard, looks great on the dashboard, looks great on the dashboard, but bytes aren't but bytes aren't but bytes aren't sound. So, we sound. So, we sound. So, we measured the performance of measured the performance of measured the performance of popular popular popular providers that providers that providers that actually emit actually emit actually emit hundreds of milliseconds of hundreds of milliseconds of hundreds of milliseconds of silence at the beginning of their silence at the beginning of their silence at the beginning of their audio. And in the worst audio. And in the worst audio. And in the worst case, up to 3/4 case, up to 3/4 case, up to 3/4 second of complete second of complete second of complete silence before the silence before the silence before the first sound appears first sound appears . So time to first . So time to first . So time to first audio is exactly what audio is exactly what audio is exactly what your subscriber hears. And that is the your subscriber hears. And that is the your subscriber hears. And that is the distinction on distinction on distinction on which this whole conversation is based which this whole conversation is based which this whole conversation is based . Measured . Measured . Measured latency vs.

latency vs. latency vs. perceived latency. perceived latency. perceived latency. So what your So what your So what your dashboard shows vs what dashboard shows vs what dashboard shows vs what your user hears your user hears . The dashboard says everything is going . The dashboard says everything is going . The dashboard says everything is going very fast, but your very fast, but your very fast, but your ears say it's going ears say it's going slowly. It's the same slowly. It's the same slowly. It's the same story with language story with language story with language models. Time to models. Time to models. Time to first token is a first token is a first token is a textual metric. But textual metric. But textual metric. But your text-to- your text-to- your text-to- speech (TTS) can't speech (TTS) can't speech (TTS) can't start speaking start speaking start speaking with just one token, with just one token, with just one token, right? He needs a right? He needs a right? He needs a whole sentence. Therefore, this whole sentence. Therefore, this whole sentence. Therefore, this number is important number is important number is important for the voice, because the for the voice, because the for the voice, because the entire first entire first entire first sentence is needed, but the model sentence is needed, but the model sentence is needed, but the model ranks differently on ranks differently on ranks differently on this indicator. So, this indicator. So, this indicator. So, the model that streams tokens the fastest the model that streams tokens the fastest is not necessarily the model that that that completes sentences the fastest. completes sentences the fastest. Now, let's say you measure all of this perfectly, but there's measure all of this perfectly, but there's another part of the conversation another part of the conversation another part of the conversation that benchmarking that benchmarking that benchmarking completely ignores. completely ignores. completely ignores. Namely, a real Namely, a real Namely, a real agent doesn't just agent doesn't just agent doesn't just communicate. He has a communicate. He has a communicate. He has a task to complete, task to complete, task to complete, right? He has to search for right? He has to search for right? He has to search for orders, orders, orders, check availability check availability , and book rooms. Is , and book rooms. Is , and book rooms. Is n't that right? So, n't that right? So, n't that right? So, the tool the tool the tool will take time. 2 will take time. 2 will take time. 2 seconds, 10, 40 or seconds, 10, 40 or seconds, 10, 40 or more seconds. And more seconds. And more seconds. And no model choice no model choice no model choice can hide a can hide a can hide a delay of 40 delay of 40 delay of 40 seconds or more. So seconds or more. So seconds or more. So think about what a human administrator would do think about what a human administrator would do think about what a human administrator would do in this situation in this situation . It doesn't freeze . It doesn't freeze while the while the while the booking system is loading, booking system is loading, booking system is loading, right? He says, " right? He says, " Give me a second. I'm Give me a second. I'm Give me a second. I'm checking your checking your checking your account." He account." He account." He gains time. And he gains time. And he gains time. And he listens while continuing to

listens while continuing to listens while continuing to work. And this is exactly the work. And this is exactly the work. And this is exactly the human behavior human behavior human behavior we are looking for. So we are looking for. So we are looking for. So why can't the agent why can't the agent why can't the agent just keep just keep just keep talking and talking and talking and performing actions performing actions performing actions asynchronously? asynchronously? asynchronously? Actually, he can. Actually, he can. Actually, he can. This is what it looks like, and This is what it looks like, and This is what it looks like, and it's the only code you'll it's the only code you'll it's the only code you'll see. This is a see. This is a see. This is a common tool common tool . There is only one . There is only one . There is only one difference, and that is difference, and that is difference, and that is this line. The first this line. The first this line. The first update returns update returns update returns the microphone to the agent while the microphone to the agent while the microphone to the agent while it continues it continues it continues to run in the background. And that's the to run in the background. And that's the to run in the background. And that's the whole secret. whole secret. whole secret. The conversation and The conversation and The conversation and the tool no longer the tool no longer the tool no longer share the same share the same share the same thread. So don't take my thread. So don't take my thread. So don't take my word for it. word for it. word for it. Listen for yourself. Listen for yourself. Listen for yourself. Thank you for Thank you for Thank you for calling the calling the calling the Live Good Hotel. How Live Good Hotel. How Live Good Hotel. How can I help you? can I help you? can I help you? Can I book a Can I book a Can I book a room? room? room? Of course. What dates are you Of course. What dates are you Of course. What dates are you planning on? planning on? planning on? From October 10 to 12. From October 10 to 12. From October 10 to 12. From October 10 to 12? And for From October 10 to 12? And for From October 10 to 12? And for how many guests? how many guests? how many guests? Two. Two. I have a room with two I have a room with two queen beds. King, double, queen beds. King, double, queen beds. King, double, queen or suite? Or queen or suite? Or queen or suite? Or perhaps you would like a perhaps you would like a perhaps you would like a room with a room with a room with a city or ocean view?

city or ocean view? city or ocean view? To the ocean. To the ocean. In whose name should In whose name should I book the room? I book the room? I book the room? Jesse Hall. Jesse Hall. All set: All set: October 10-12, king room with October 10-12, king room with October 10-12, king room with ocean view, amount ocean view, amount ocean view, amount $582.40 from a card $582.40 from a card $582.40 from a card ending in 1111. ending in 1111. ending in 1111. Can I book? Can I book? Can I book? You know, actually... You know, actually... I need to I need to I need to change the dates to December change the dates to December change the dates to December 20th through December 24th 20th through December 24th . . . From December 20 to 24? Then the From December 20 to 24? Then the From December 20 to 24? Then the total amount will be total amount will be total amount will be $1,164.80 for the period from December 20 to $1,164.80 for the period from December 20 to $1,164.80 for the period from December 20 to 24. Can I 24. Can I 24. Can I make a reservation? make a reservation? make a reservation? Yes. Yes. Everything is ready. Okay, we had to Okay, we had to speed up some speed up some speed up some points due to points due to points due to time constraints, but the time constraints, but the time constraints, but the agent kept the agent kept the agent kept the conversation going. There were no conversation going. There were no conversation going. There were no awkward awkward awkward pauses. The agent pauses. The agent pauses. The agent continued to continued to continued to work even work even work even when the caller when the caller when the caller interrupted him. interrupted him. interrupted him. The client also The client also The client also changed his mind mid- changed his mind mid- changed his mind mid- conversation, and the agent conversation, and the agent conversation, and the agent just adjusted just adjusted , right? This is exactly the , right? This is exactly the , right? This is exactly the human behavior human behavior human behavior we are striving we are striving we are striving to achieve. So again to achieve. So again to achieve. So again , it's not one , it's not one , it's not one model that's good, it's the whole model that's good, it's the whole model that's good, it's the whole system that's working well.

system that's working well. It is an infrastructure, a It is an infrastructure, a framework, and models framework, and models framework, and models that work together. And this is that work together. And this is that work together. And this is exactly what no exactly what no exactly what no individual individual individual component or component or component or model benchmark sees. model benchmark sees. Now, there's another Now, there's another aspect that aspect that aspect that leaderboards completely ignore leaderboards completely ignore leaderboards completely ignore : knowing : knowing when to speak and when to speak and when to speak and when not to. People when not to. People when not to. People coordinate coordinate coordinate the order of their remarks within the order of their remarks within the order of their remarks within a few hundred a few hundred a few hundred milliseconds, milliseconds, milliseconds, largely due to largely due to largely due to tone. And the pipeline tone. And the pipeline tone. And the pipeline waiting for the full waiting for the full waiting for the full transcription transcription transcription is lagging behind, isn't is lagging behind, isn't is lagging behind, isn't it? So we created a it? So we created a it? So we created a queue detector that is queue detector that is queue detector that is based on the based on the based on the audio itself. It listens to audio itself. It listens to audio itself. It listens to the rhythm, pitch, and the rhythm, pitch, and the rhythm, pitch, and intonation of speech, and intonation of speech, and intonation of speech, and we have published data we have published data we have published data on this. So, given the on this. So, given the on this. So, given the same same same delay budget, the best delay budget, the best delay budget, the best alternative alternative alternative interrupts interrupts interrupts users 9.9% of the users 9.9% of the time. Our time. Our time. Our rate is 4.5%, and our rate is 4.5%, and our assessment tool is assessment tool is open source. You open source. You open source. You don't have to take my don't have to take my don't have to take my word for it, you word for it, you word for it, you can run can run can run your own data and your own data and your own data and see the result.

see the result. see the result. Let's go Let's go Let's go one level deeper. one level deeper. one level deeper. The pause between The pause between The pause between the customer's last syllable and the the customer's last syllable and the the customer's last syllable and the first sound from the first sound from the first sound from the agent. agent. agent. End-of-line detection End-of-line detection End-of-line detection is triggered on the audio is triggered on the audio is triggered on the audio before the transcript appears, before the transcript appears, before the transcript appears, the language model is already the language model is already the language model is already processing the previous processing the previous processing the previous text, and if the last text, and if the last text, and if the last words match, you words match, you words match, you save half a save half a save half a second. And then the second. And then the second. And then the speech synthesis (TTS) speech synthesis (TTS) speech synthesis (TTS) starts with the starts with the starts with the first complete first complete first complete sentence, not the sentence, not the sentence, not the last token. last token. So the first So the first sound heard is the moment when sound heard is the moment when sound heard is the moment when a person stops a person stops a person stops waiting. So, every waiting. So, every waiting. So, every model you model you model you choose appears choose appears choose appears here in this window, and here in this window, and here in this window, and this window has a this window has a this window has a fixed size. fixed size. fixed size. If you can If you can If you can fit in a second and a half fit in a second and a half fit in a second and a half from start from start from start to finish, the conversation is still to finish, the conversation is still to finish, the conversation is still going, but you're going, but you're going, but you're already on the edge. If you already on the edge. If you already on the edge. If you can get close can get close can get close to 600 milliseconds, then to 600 milliseconds, then to 600 milliseconds, then it starts to it starts to it starts to feel feel truly human. truly human. So the question is never So the question is never how how how fast can I fast can I fast can I work, but what is the work, but what is the work, but what is the most most most human agent that human agent that human agent that will fit into that will fit into that will fit into that window? So, window? So, window? So, back to the back to the back to the delay as a budget, delay as a budget, delay as a budget, but with humanity as the goal.

but with humanity as the goal. So to sum it So to sum it all up, the question all up, the question all up, the question we asked at the beginning is we asked at the beginning is : which models should I : which models should I : which models should I choose? There was choose? There was choose? There was never really never really never really an answer to it. So an answer to it. So an answer to it. So we created something that we created something that we created something that we wish already we wish already we wish already existed. Here's how we existed. Here's how we existed. Here's how we created this created this created this benchmark. We take a benchmark. We take a benchmark. We take a fully functional fully functional fully functional agent—the first one is the agent—the first one is the agent—the first one is the hotel administrator hotel administrator you just heard about you just heard about you just heard about —it takes —it takes —it takes real reservations, it real reservations, it real reservations, it has a real backend, has a real backend, has a real backend, and it's also open and it's also open and it's also open source, you can source, you can source, you can look at look at look at the repository, and we the repository, and we the repository, and we run hundreds of run hundreds of run hundreds of simulated simulated simulated calls to it. calls to it. The simulator interrupts, The simulator interrupts, mumbles, and changes its mumbles, and changes its mumbles, and changes its mind, just like a mind, just like a mind, just like a real subscriber. And real subscriber. And real subscriber. And then we evaluate not then we evaluate not then we evaluate not by impressions, but by by impressions, but by by impressions, but by results, results, results, defined as defined as defined as we would define them for a we would define them for a we would define them for a person. So, did person. So, did person. So, did the correct the correct the correct booking make it into the booking make it into the booking make it into the database? Did the database? Did the database? Did the call end? Are call end? Are call end? Are the names and the names and the names and dates correct? Were the dates correct? Were the tool challenges or tool challenges or personal personal personal data leaks fabricated, right? So data leaks fabricated, right? So data leaks fabricated, right? So we measure all these we measure all these we measure all these things against things against things against human behavior.

human behavior. human behavior. This is the bar. And This is the bar. And This is the bar. And just just just completing a task isn't completing a task isn't completing a task isn't enough, because enough, because enough, because completion completion completion can actually be manipulated, can actually be manipulated, can actually be manipulated, right? So the model right? So the model right? So the model can somehow guess the can somehow guess the can somehow guess the path to completion, path to completion, path to completion, right? So we right? So we right? So we punish for what the completion indicator doesn't punish for what the completion indicator doesn't punish for what the completion indicator doesn't see see see . . . For example, if he For example, if he For example, if he asks questions that should asks questions that should asks questions that should n't have been n't have been n't have been asked, or if asked, or if asked, or if he takes 10 he takes 10 he takes 10 steps to do steps to do steps to do something that should something that should something that should only take two. And then we only take two. And then we only take two. And then we replace the stack. replace the stack. replace the stack. Same agent, same Same agent, same Same agent, same scenarios, but different scenarios, but different scenarios, but different models. Each model models. Each model models. Each model sees exactly sees exactly sees exactly identical input identical input identical input data, byte by byte. But data, byte by byte. But data, byte by byte. But this creates benchmarks this creates benchmarks this creates benchmarks based on our based on our based on our agent example. agent example. There are as many There are as many benchmarks as there are benchmarks as there are benchmarks as there are agents. So this agents. So this agents. So this benchmark is actually benchmark is actually benchmark is actually yours. You can yours. You can yours. You can run this very thing, run this very thing, run this very thing, run your run your run your agent through this agent through this agent through this very benchmark, and very benchmark, and very benchmark, and get your get your get your results. Here's how you results. Here's how you results. Here's how you can spot can spot can spot surprises. All of this is surprises. All of this is surprises. All of this is open source: open source: open source: the methodology, the the methodology, the the methodology, the evaluation kits, the agent itself.

evaluation kits, the agent itself. evaluation kits, the agent itself. You can direct You can direct You can direct the same to your the same to your the same to your agent and let us agent and let us agent and let us know how it works for know how it works for know how it works for you. An important fact you. An important fact you. An important fact about LiveKit is about LiveKit is about LiveKit is that we do not that we do not that we do not build STT, build STT, build STT, LLM, or TTS models. Every LLM, or TTS models. Every LLM, or TTS models. Every provider provider provider connects to connects to connects to our stack. So our stack. So our stack. So we have no vested we have no vested we have no vested interests in this race interests in this race . We just constantly . We just constantly . We just constantly monitor the monitor the monitor the work of all of them. And that's why work of all of them. And that's why work of all of them. And that's why we can we can we can confidently choose confidently choose confidently choose winners. So what winners. So what winners. So what you see now you see now is a preview of the is a preview of the is a preview of the Gemini 4 31B model on LiveKit Gemini 4 31B model on LiveKit Gemini 4 31B model on LiveKit inference. But this isn't inference. But this isn't inference. But this isn't just your average just your average just your average Gemma 4 31B off Gemma 4 31B off Gemma 4 31B off the shelf model. It is the shelf model. It is the shelf model. It is specifically specifically specifically tuned for tuned for tuned for voice agents with voice agents with voice agents with long prompts, long prompts, long prompts, many many many tools, and tools, and tools, and tight tight tight latency requirements. She latency requirements. She latency requirements. She passes voice passes voice passes voice tests at 88% and begins tests at 88% and begins tests at 88% and begins speaking after about speaking after about speaking after about 380 milliseconds. 380 milliseconds. 380 milliseconds. That's more than twice as That's more than twice as That's more than twice as fast as the next fast as the next fast as the next model. Which makes it the model. Which makes it the model. Which makes it the fastest fastest fastest high-quality high-quality high-quality voice LLM voice LLM voice LLM available today. And available today. And available today. And it will actually be it will actually be it will actually be launched tomorrow on launched tomorrow on launched tomorrow on LiveKit inference. Again LiveKit inference. Again LiveKit inference. Again , our goal is , our goal is to create agents that are as to create agents that are as human-like as possible human-like as possible while adhering to a while adhering to a latency budget. So, if latency budget. So, if you remember you remember you remember one thing from this talk, one thing from this talk, one thing from this talk, let it be this. A let it be this. A let it be this. A voice agent is voice agent is voice agent is not a chatbot with a not a chatbot with a speaker attached to it. This is speaker attached to it. This is infrastructure that infrastructure that infrastructure that imitates a person. A imitates a person. A imitates a person. A real- real- real- time system that senses and time system that senses and time system that senses and responds just like a responds just like a responds just like a human would.

human would. human would. Once you start Once you start Once you start thinking about it that way thinking about it that way thinking about it that way , every , every , every decision will become decision will become decision will become clearer. Budget, clearer. Budget, clearer. Budget, metrics, models, and the metrics, models, and the metrics, models, and the bar you bar you bar you are testing against are testing against . So, here's the . So, here's the . So, here's the checklist. checklist. checklist. You can You can You can take a picture of him. take a picture of him. take a picture of him. First, set a First, set a First, set a delay budget. delay budget. delay budget. Measure the perceived Measure the perceived Measure the perceived delay. Spend delay. Spend delay. Spend the rest of your budget on the rest of your budget on the rest of your budget on model quality, not model quality, not model quality, not speed, which doesn't speed, which doesn't speed, which doesn't matter. And anything longer matter. And anything longer matter. And anything longer than a second should than a second should than a second should happen happen happen asynchronously so that asynchronously so that asynchronously so that the conversation never the conversation never the conversation never stops. And just stops. And just stops. And just test the conversation, test the conversation, test the conversation, not individual components. not individual components. not individual components. And continue And continue And continue testing in testing in testing in production. So, here's production. So, here's production. So, here's the benchmark. the benchmark. the benchmark. Scan the QR code. You Scan the QR code. You Scan the QR code. You can check out can check out can check out the benchmark. You the benchmark. You the benchmark. You can test can test can test it on your own it on your own it on your own agents. And remember, in agents. And remember, in agents. And remember, in the near future the near future the near future you will call the you will call the you will call the voice agent, voice agent, voice agent, interrupt him, and interrupt him, and interrupt him, and the system will not the system will not the system will not break. She break. She break. She will behave as a will behave as a person would in such a person would in such a situation. So go ahead and situation. So go ahead and situation. So go ahead and create it. Thank you.