The Autonomous Computer: Infrastructure for Computer Use Agents — Ang Li, Simular

A little about my past. A little about my past. Um, I'm the co-founder and Um, I'm the co-founder and Um, I'm the co-founder and CEO of Similar. And this CEO of Similar. And this CEO of Similar. And this year is about the third year is about the third year is about the third since I started since I started since I started interacting with interacting with interacting with computers. I'm currently computers. I'm currently computers. I'm currently trying to create a trying to create a trying to create a computer that is computer that is computer that is completely autonomous completely autonomous completely autonomous so that I don't have to so that I don't have to interact with it. If I interact with it. If I look back at my look back at my look back at my history, the first 10 history, the first 10 history, the first 10 years...I learned about years...I learned about years...I learned about computers when I was computers when I was computers when I was very young. For the first 10 very young. For the first 10 very young. For the first 10 years, I years, I years, I focused on focused on focused on sports sports sports programming, meaning I programming, meaning I programming, meaning I wrote code almost every day wrote code almost every day . And . And . And I spent the second decade training I spent the second decade training I spent the second decade training neural networks, neural networks, neural networks, mostly publishing mostly publishing mostly publishing a lot of research a lot of research a lot of research papers on AI during my papers on AI during my papers on AI during my PhD and PhD and PhD and later at DeepMind. And later at DeepMind. And later at DeepMind. And the third...the third the third...the third the third...the third decade. So, I decade. So, I decade. So, I joined DeepMind in joined DeepMind in joined DeepMind in 2017. This is the beginning of the 2017. This is the beginning of the 2017. This is the beginning of the third decade third decade . So this year is the 26th, . So this year is the 26th, . So this year is the 26th, almost 10 years. When I almost 10 years. When I almost 10 years. When I first came to DeepMind first came to DeepMind , we were really , we were really , we were really trying to solve a trying to solve a trying to solve a problem called problem called problem called AGI—artificial general AGI—artificial general AGI—artificial general intelligence. Everyone intelligence. Everyone intelligence. Everyone was still confused then. And was still confused then. And was still confused then. And even now, everyone is still even now, everyone is still even now, everyone is still confused about what confused about what confused about what AGI is. But AGI is. But AGI is. But overall, it's a overall, it's a overall, it's a general- general- general- purpose system that's purpose system that's purpose system that's similar to a human, similar to a human, similar to a human, meaning the machine can meaning the machine can meaning the machine can do different things. And do different things. And do different things. And I hope that I hope that I hope that next year we will next year we will next year we will be able to solve this be able to solve this be able to solve this problem. Then I problem. Then I problem. Then I will be able to think about will be able to think about what to do and what what to do and what what to do and what tasks to solve tasks to solve tasks to solve during the fourth during the fourth during the fourth decade. Okay, decade. Okay, decade. Okay, I want to start my I want to start my I want to start my conversation now. Actually conversation now. Actually , the reason I want to , the reason I want to , the reason I want to present this is because present this is because present this is because I I I spent 10 years on

spent 10 years on spent 10 years on programming, 10 programming, 10 programming, 10 years on neural networks, years on neural networks, years on neural networks, and 10 years on and 10 years on and 10 years on solving the AGI problem solving the AGI problem . This gives me a certain . This gives me a certain . This gives me a certain bias that we bias that we bias that we are trying are trying are trying to implement into our to implement into our to implement into our current company with current company with current company with this principle in mind this principle in mind this principle in mind . A . A . A general AI system must general AI system must general AI system must be neurosymbolic be neurosymbolic . So, code combined . So, code combined . So, code combined with neural networks? I'll with neural networks? I'll with neural networks? I'll quickly show you quickly show you quickly show you why we have this why we have this why we have this intuition. So the purpose of intuition. So the purpose of intuition. So the purpose of this workshop is: I this workshop is: I this workshop is: I know we're short on know we're short on know we're short on time, but I want to give time, but I want to give time, but I want to give you three things. First you three things. First , it is basic knowledge about , it is basic knowledge about , it is basic knowledge about using using using computers. We computers. We computers. We understand that 50% of you are understand that 50% of you are new to this industry. Secondly, these are the new to this industry. Secondly, these are the tools tools tools we worked on as we worked on as we worked on as part of the part of the part of the infrastructure. infrastructure. infrastructure. There are four different There are four different There are four different types of components that types of components that we believe we believe we believe will be useful for will be useful for will be useful for your future your future computer-based application development. And I'll also computer-based application development. And I'll also show you a few show you a few show you a few programs that programs that programs that other people, other people, other people, other developers, worked on. And other developers, worked on. And other developers, worked on. And the third thing, which is important, the third thing, which is important, the third thing, which is important, because we have a because we have a because we have a product called psi, product called psi, product called psi, and this product is still and this product is still and this product is still in closed in closed in closed beta right now. So I beta right now. So I beta right now. So I want to give everyone want to give everyone want to give everyone an invitation at the end of the an invitation at the end of the an invitation at the end of the seminar so you can have the seminar so you can have the seminar so you can have the opportunity opportunity opportunity to try this to try this to try this product. Also product. Also product. Also let me know. Feel let me know. Feel let me know. Feel free to free to free to follow follow follow me on Twitter and me on Twitter and me on Twitter and join Discord join Discord . Let me know if . Let me know if . Let me know if you need more you need more you need more credits. So I'll credits. So I'll credits. So I'll happily give more happily give more happily give more credit. So, the first credit. So, the first credit. So, the first question is: what are question is: what are computer usage agents? Right ? Usually during

? Usually during my performance I can my performance I can take a selfie here. With take a selfie here. With take a selfie here. With all of you. I'll do all of you. I'll do all of you. I'll do part of it. Yes. So, this is what part of it. Yes. So, this is what part of it. Yes. So, this is what I do every day I do every day I do every day now. I'm actually a now. I'm actually a now. I'm actually a CEO CEO , so I don't sit at a , so I don't sit at a , so I don't sit at a computer all computer all computer all day. I have my day. I have my day. I have my agents on my phone, agents on my phone, agents on my phone, so I can text the so I can text the so I can text the agent, send a agent, send a agent, send a photo, and tell them photo, and tell them photo, and tell them to post the photo on to post the photo on to post the photo on LinkedIn. Yes. And then the LinkedIn. Yes. And then the LinkedIn. Yes. And then the agent will start working on it agent will start working on it agent will start working on it . So, . So, . So, let's see. Everyone let's see. Everyone let's see. Everyone can see if the agent is can see if the agent is can see if the agent is responding. I see I responding. I see I responding. I see I got some Okay, got some Okay, got some Okay, let's see. I have let's see. I have let's see. I have my my my message. Okay, the message. Okay, the message. Okay, the agent replies, he agent replies, he agent replies, he got my photo. And you got my photo. And you got my photo. And you see the agent see the agent see the agent started asking me started asking me started asking me questions. So I won't questions. So I won't questions. So I won't interact with interact with interact with the agent right now, but I the agent right now, but I the agent right now, but I want to show you the want to show you the want to show you the basic interface.

basic interface. basic interface. So, agents for So, agents for So, agents for using using using a computer means an a computer means an a computer means an agent who has agent who has agent who has a computer. In this a computer. In this a computer. In this scenario, it's a scenario, it's a scenario, it's a full full full Windows desktop. And if I Windows desktop. And if I Windows desktop. And if I continue to interact continue to interact continue to interact with the computer, the mouse and with the computer, the mouse and with the computer, the mouse and keyboard will keyboard will keyboard will work on that work on that work on that desktop and desktop and desktop and later open later open later open LinkedIn. And I'll get back to LinkedIn. And I'll get back to LinkedIn. And I'll get back to the presentation. I just the presentation. I just the presentation. I just showed you an example with showed you an example with showed you an example with one agent. So, I one agent. So, I one agent. So, I can talk to the can talk to the can talk to the agent. The agent works agent. The agent works agent. The agent works with programs. But with programs. But with programs. But in reality, you may in reality, you may in reality, you may have several. This is what have several. This is what have several. This is what we we we are working on now. So, that's are working on now. So, that's are working on now. So, that's four agents. You four agents. You four agents. You see they have different see they have different see they have different names, right? So a QA names, right? So a QA agent essentially agent essentially agent essentially performs performs performs product testing. product testing. product testing. Product testing Product testing is one of the big is one of the big use cases. They use cases. They also do also do also do research, research, research, customer support, customer support, customer support, go to the go to the go to the Discord channel and review Discord channel and review Discord channel and review all the information there. all the information there. all the information there. So, real agents So, real agents So, real agents work slower. work slower. work slower. So this is actually a So this is actually a So this is actually a recording that's a recording that's a recording that's a few hours long, so I few hours long, so I few hours long, so I compressed it down to 30 compressed it down to 30 compressed it down to 30 seconds so you can seconds so you can seconds so you can quickly understand how quickly understand how quickly understand how agents work for agents work for computer usage. The general computer usage. The general idea is very idea is very idea is very different from different from different from most of the agents most of the agents most of the agents we are talking about now we are talking about now we are talking about now . Most . Most . Most agents are agents are agents are API-based. And you don't see API-based. And you don't see API-based. And you don't see visually what the visually what the visually what the agent is doing.

agent is doing. Computer usage agent is not Computer usage agent is not just a category, we just a category, we just a category, we believe it is a broader believe it is a broader believe it is a broader category that can category that can category that can perform many perform many perform many tasks where there is no API. tasks where there is no API. tasks where there is no API. You can still You can still You can still let agents let agents let agents work with them. work with them. work with them. Now I want to do a Now I want to do a Now I want to do a little ritual, little ritual, little ritual, because we founded the because we founded the because we founded the company in 2023. Is company in 2023. Is company in 2023. Is n't that right? This was right n't that right? This was right n't that right? This was right after the release of GPT 3.5 and after the release of GPT 3.5 and after the release of GPT 3.5 and GPT 4V. So I want to GPT 4V. So I want to GPT 4V. So I want to show you an example of show you an example of show you an example of how two years ago how two years ago how two years ago we were almost we were almost we were almost hopeless on this hopeless on this hopeless on this issue. I issue. I issue. I wrote about it on Twitter at the time. wrote about it on Twitter at the time. wrote about it on Twitter at the time. So, this is my tweet. So, this is my tweet. So, this is my tweet. Let me Let me Let me pause for a moment. This is my pause for a moment. This is my pause for a moment. This is my tweet from two years ago tweet from two years ago tweet from two years ago . The problem . The problem . The problem was, was, was, can we can we can we use GPT-4V to use GPT-4V to use GPT-4V to recognize the screen? And recognize the screen? And recognize the screen? And then click on the then click on the then click on the iMessage icon on the screen iMessage icon on the screen . A very, very simple, . A very, very simple, . A very, very simple, elementary task elementary task . I actually . I actually . I actually used the model used the model used the model to build an to build an to build an iterative iterative iterative positioning refinement. You positioning refinement. You positioning refinement. You see that see that see that the mouse is pointing the mouse is pointing the mouse is pointing incorrectly the first time. The second time incorrectly the first time. The second time incorrectly the first time. The second time she goes to the she goes to the she goes to the mail icon, and the third time she mail icon, and the third time she mail icon, and the third time she taps on iMessage. This is the taps on iMessage. This is the taps on iMessage. This is the only successful only successful only successful demonstration I have demonstration I have demonstration I have done. I asked done. I asked done. I asked the agent to click on the agent to click on the agent to click on Zoom. It didn't Zoom. It didn't Zoom. It didn't work at all. Two years work at all. Two years work at all. Two years ago this problem was ago this problem was ago this problem was almost hopeless, almost hopeless, almost hopeless, but we felt that the but we felt that the but we felt that the models models models would get better over time.

would get better over time. would get better over time. Eventually, this problem Eventually, this problem Eventually, this problem will be solved. will be solved. But two years ago, this But two years ago, this was not possible, and this was not possible, and this was not possible, and this prompted us prompted us prompted us to think about a to think about a to think about a strategy for solving the strategy for solving the strategy for solving the problem of problem of computer use by agents. And we have a name for And we have a name for it. This is called a it. This is called a it. This is called a stand-alone computer, stand-alone computer, stand-alone computer, because if a computer because if a computer because if a computer can perform can perform can perform all sorts of tasks, it's all sorts of tasks, it's all sorts of tasks, it's stand-alone, right? stand-alone, right? stand-alone, right? The question arises, how do The question arises, how do The question arises, how do we create a we create a we create a standalone computer? standalone computer? When we talk about When we talk about this, there are only two words: this, there are only two words: this, there are only two words: autonomous and autonomous and autonomous and computer. So it's computer. So it's computer. So it's natural to think about natural to think about natural to think about two directions. The first two directions. The first is autonomy, is autonomy, is autonomy, the second is a computer, the second is a computer, the second is a computer, right? Autonomy right? Autonomy right? Autonomy means: do we have a means: do we have a means: do we have a system that can system that can system that can actually understand the actually understand the actually understand the context, the context, the context, the computer environment, computer environment, computer environment, perform all the perform all the perform all the reasoning, reasoning, reasoning, planning, and planning, and planning, and correct its own correct its own correct its own errors. That is, it does not errors. That is, it does not errors. That is, it does not require a person to require a person to require a person to supervise the agent. And the supervise the agent. And the supervise the agent. And the second part is the second part is the second part is the computer. What computer. What computer. What computer can the computer can the agent use? So, this is a agent use? So, this is a key question. But back then key question. But back then key question. But back then , 2 years ago, we , 2 years ago, we , 2 years ago, we thought that a computer thought that a computer was an easy problem was an easy problem was an easy problem because everyone already had because everyone already had because everyone already had computers, right?

computers, right? computers, right? So we focused So we focused So we focused on the on the on the autonomy part. When we autonomy part. When we autonomy part. When we talk about talk about talk about autonomy, we autonomy, we autonomy, we go back to the go back to the go back to the basic principle basic principle basic principle that we had at that we had at that we had at the beginning—the the beginning—the neurosymbolic neurosymbolic neurosymbolic approach. Neuro-means approach. Neuro-means approach. Neuro-means the same thing that the same thing that the same thing that happens with happens with happens with driverless driverless driverless cars. This is cars. This is cars. This is essentially the same essentially the same essentially the same concept for concept for concept for all types of all types of all types of autonomous systems. autonomous systems. autonomous systems. They all have the They all have the They all have the same concept. same concept. same concept. You have a component You have a component You have a component that is neural. Here it is that is neural. Here it is that is neural. Here it is VLM, visual language VLM, visual language VLM, visual language model, VLM agent. And the model, VLM agent. And the model, VLM agent. And the second part is second part is symbolic. And here we symbolic. And here we symbolic. And here we called it the called it the called it the encoding agent. We encoding agent. We encoding agent. We need to solve these need to solve these need to solve these two technological two technological two technological challenges to make challenges to make challenges to make autonomy work autonomy work . I just showed you a . I just showed you a . I just showed you a demo where 2 years ago VLM demo where 2 years ago VLM demo where 2 years ago VLM didn't work at all, didn't work at all, didn't work at all, right? So we right? So we right? So we spent a lot of spent a lot of spent a lot of time 2 years ago time 2 years ago time 2 years ago working on the working on the working on the coding level to coding level to coding level to make sure we make sure we make sure we could control could control could control the computer more the computer more the computer more reliably. But the second reliably. But the second reliably. But the second problem was problem was problem was that 2 years ago we that 2 years ago we that 2 years ago we created a created a created a research group research group research group to think about how to to think about how to to think about how to improve this weak improve this weak improve this weak VLM for working with VLM for working with VLM for working with computers. Before I computers. Before I computers. Before I start talking start talking start talking about the trajectory of about the trajectory of about the trajectory of VLM agents, I VLM agents, I VLM agents, I want to briefly want to briefly want to briefly talk about talk about talk about signals. Like signals. Like signals. Like self-driving self-driving self-driving cars, cars, cars, like Waymo on like Waymo on like Waymo on the road, they have the road, they have the road, they have signals: lidar, radar, signals: lidar, radar, signals: lidar, radar, and a camera, right? Three and a camera, right? Three and a camera, right? Three different types of signals.

different types of signals. different types of signals. Computers have the same thing Computers have the same thing Computers have the same thing . So I often . So I often . So I often tell people, "We're tell people, "We're tell people, "We're working on working on working on driverless driverless driverless cars for cars for cars for your computer, in the your computer, in the your computer, in the digital world." digital world." digital world." In effect, replacing the In effect, replacing the In effect, replacing the steering wheel with your mouse and steering wheel with your mouse and steering wheel with your mouse and keyboard. So we keyboard. So we keyboard. So we still have three still have three still have three different types of signals. different types of signals. different types of signals. One is a One is a One is a screenshot, the other is an screenshot, the other is an screenshot, the other is an availability tree. The availability tree. The availability tree. The accessibility tree is something accessibility tree is something accessibility tree is something very unique. This is a very unique. This is a very unique. This is a pretty rare thing in pretty rare thing in pretty rare thing in our daily our daily our daily lives. The DOM tree is lives. The DOM tree is lives. The DOM tree is quite popular because quite popular because quite popular because we use we use we use browsers. browsers. browsers. Accessibility is a kind of Accessibility is a kind of Accessibility is a kind of DOM tree, but DOM tree, but DOM tree, but it is tied to it is tied to it is tied to each individual each individual each individual operating system. operating system. operating system. Mac has its own Mac has its own accessibility tree format. And Windows accessibility tree format. And Windows has a separate protocol has a separate protocol has a separate protocol for the for the accessibility tree. But this is the accessibility tree. But this is the base level that the base level that the base level that the operating system operating system operating system uses to uses to uses to display all display all display all windows. All the icons windows. All the icons windows. All the icons on your screen, on your screen, on your screen, you know. So, this you know. So, this you know. So, this part is very important part is very important part is very important because...Yes, these are the because...Yes, these are the because...Yes, these are the efforts that we have been efforts that we have been efforts that we have been working on. It's working on. It's working on. It's called Agent S. called Agent S. called Agent S. So the main idea So the main idea So the main idea was, was, was, can we build can we build can we build our framework? 2 years our framework? 2 years our framework? 2 years ago we didn't call ago we didn't call ago we didn't call it a framework.

it a framework. it a framework. Today everyone Today everyone Today everyone calls it a calls it a calls it a framework. So 2 framework. So 2 framework. So 2 years ago we years ago we years ago we tried to complete tried to complete tried to complete this project called this project called this project called Agent S, to build our Agent S, to build our Agent S, to build our framework based on framework based on framework based on all the base models, all the base models, all the base models, not just one. So not just one. So not just one. So you can easily you can easily you can easily switch between switch between switch between models. It is a model- models. It is a model- agnostic agnostic agnostic framework that framework that framework that allows you to allows you to allows you to get the best out of your get the best out of your computer. The first computer. The first generation was generation was generation was ready at the end of ready at the end of ready at the end of 2024. And it's 2024. And it's 2024. And it's called Agent S. We called Agent S. We called Agent S. We just call just call just call it Agent S because it's the it Agent S because it's the it Agent S because it's the beginning. And this agent beginning. And this agent beginning. And this agent achieved a 20% success rate in achieved a 20% success rate in achieved a 20% success rate in OS World. 20%—and that was the OS World. 20%—and that was the OS World. 20%—and that was the best best best result at the result at the result at the time. The best time. The best time. The best result then, in result then, in result then, in 2024, which is quite 2024, which is quite 2024, which is quite interesting. And this article interesting. And this article interesting. And this article also won also won also won the award for best the award for best the award for best paper at the ICLR paper at the ICLR paper at the ICLR 2025 workshop on agent-based AI. You 2025 workshop on agent-based AI. You 2025 workshop on agent-based AI. You know, the whole industry know, the whole industry know, the whole industry has been moving very quickly has been moving very quickly has been moving very quickly over the past over the past over the past year. And this is just year. And this is just year. And this is just madness. The choice in madness. The choice in madness. The choice in favor of this favor of this favor of this framework was framework was framework was due to the fact that due to the fact that due to the fact that we initially knew that we initially knew that we initially knew that visual-linguistic visual-linguistic visual-linguistic models were very bad.

models were very bad. models were very bad. This won't work. That's This won't work. That's This won't work. That's why we decided to why we decided to why we decided to use the use the accessibility tree in the accessibility tree in the first generation. We first generation. We first generation. We just look at just look at just look at all the underlying protocols of the all the underlying protocols of the all the underlying protocols of the tree structure to tree structure to tree structure to describe the current describe the current describe the current environment and environment and environment and pass that to a large pass that to a large pass that to a large language model, not language model, not language model, not just a language model. just a language model. And we use And we use ID-based binding, ID-based binding, ID-based binding, linearize all linearize all linearize all the buttons, assign them the buttons, assign them the buttons, assign them IDs, and ask LLM: IDs, and ask LLM: IDs, and ask LLM: which button, which which button, which which button, which button ID do I need to button ID do I need to button ID do I need to click now? This click now? This click now? This works to some extent. works to some extent. works to some extent. That's 20%, which is very That's 20%, which is very That's 20%, which is very exciting because exciting because exciting because at the beginning this at the beginning this at the beginning this benchmark only had a 5% benchmark only had a 5% success rate. In success rate. In success rate. In early 2024. And early 2024. And early 2024. And then, in early then, in early then, in early 2025, we started to 2025, we started to 2025, we started to see visual- see visual- linguistic models getting linguistic models getting linguistic models getting better. So in the better. So in the better. So in the second generation of Agent second generation of Agent second generation of Agent S, we actually did S, we actually did S, we actually did away away away with the with the with the special special special abilities tree entirely. So, this is an abilities tree entirely. So, this is an abilities tree entirely. So, this is an approach based approach based approach based solely on vision. And the solely on vision. And the solely on vision. And the most interesting thing about this most interesting thing about this most interesting thing about this architecture is that architecture is that architecture is that we have multiple we have multiple we have multiple levels, like levels, like levels, like managers and managers and managers and executors. It's something executors. It's something executors. It's something like a human like a human like a human organization. And we organization. And we organization. And we showed that it showed that it showed that it works. This work works. This work works. This work has generated a lot of has generated a lot of has generated a lot of interest among my friends interest among my friends interest among my friends in Silicon Valley.

in Silicon Valley. in Silicon Valley. They are very inspired by They are very inspired by They are very inspired by these results. these results. The reason is that it The reason is that it proves: two models are proves: two models are proves: two models are better than one. Having better than one. Having better than one. Having two base models in two base models in two base models in one system is better one system is better one system is better than one. And the second than one. And the second than one. And the second basic model is an basic model is an basic model is an open-source model with 7 open-source model with 7 open-source model with 7 billion parameters billion parameters . So yes. So my . So yes. So my . So yes. So my startup friends are startup friends are startup friends are excited because they see excited because they see excited because they see the opportunity to train the opportunity to train the opportunity to train small models and small models and small models and outperform Claude or outperform Claude or outperform Claude or OpenAI's computer vision OpenAI's computer vision OpenAI's computer vision by 20%. At the end of by 20%. At the end of by 20%. At the end of 2025, we introduced the 2025, we introduced the 2025, we introduced the third generation of third generation of third generation of agents, and the main agents, and the main agents, and the main idea was idea was idea was whether whether task execution times could be scaled. This task execution times could be scaled. This is called is called output time scaling. When you output time scaling. When you have an agent, you can have an agent, you can have an agent, you can engage multiple engage multiple engage multiple agents to agents to agents to solve the same solve the same solve the same task, which task, which ultimately increases ultimately increases ultimately increases productivity. And we productivity. And we productivity. And we proved it. This is a graph of the proved it. This is a graph of the proved it. This is a graph of the total total total resource budget: the total resource budget: the total resource budget: the total number of workers number of workers number of workers multiplied by the multiplied by the multiplied by the number of steps. This is number of steps. This is number of steps. This is necessary for a fair necessary for a fair necessary for a fair comparison. You can comparison. You can comparison. You can see that 10 agents see that 10 agents see that 10 agents provide better provide better provide better scaling scaling scaling success. Then, in success. Then, in success. Then, in December, we broke the December, we broke the human performance record in the OS human performance record in the OS World benchmark. Here is a comparison of World benchmark. Here is a comparison of World benchmark. Here is a comparison of the trajectories of the agents, the trajectories of the agents, the trajectories of the agents, Salvador and OpenAI. Both Salvador and OpenAI. Both Salvador and OpenAI. Both major labs major labs major labs reached the same

reached the same reached the same level of productivity level of productivity as humans in March of as humans in March of as humans in March of this year. That this year. That this year. That last December was an last December was an last December was an exciting moment exciting moment exciting moment when we realized that when we realized that when we realized that computer agents were computer agents were computer agents were finally becoming finally becoming finally becoming useful. But you know useful. But you know , this benchmark is only for , this benchmark is only for , this benchmark is only for research purposes. This does not research purposes. This does not research purposes. This does not mean that in mean that in mean that in real life we ​​have real life we ​​have real life we ​​have already reached 100%. Or that already reached 100%. Or that already reached 100%. Or that they are already better than they are already better than they are already better than humans. A few days humans. A few days humans. A few days ago, OS World released the ago, OS World released the ago, OS World released the second generation, second generation, second generation, called OS World 2. called OS World 2. I was particularly impressed by I was particularly impressed by this new benchmark. In the this new benchmark. In the this new benchmark. In the new test, the new test, the new test, the best model has a best model has a best model has a score of only 20%. So score of only 20%. So score of only 20%. So we're entering the second we're entering the second we're entering the second phase of phase of phase of computer agent development. computer agent development. I want to show you I want to show you one video. This was a one video. This was a one video. This was a very interesting very interesting very interesting demonstration that demonstration that demonstration that one of the one of the one of the developers used. This developers used. This developers used. This project won at project won at project won at CalHacks last year. CalHacks last year. This is one of the largest This is one of the largest hackathons hackathons hackathons taking place in Berkeley. taking place in Berkeley. They built their They built their application based on application based on application based on open-source agents.

open-source agents. You don't seem to hear the You don't seem to hear the sound. Is sound. Is that better? What if that better? What if you could control you could control you could control your computer from your computer from your computer from anywhere in the world anywhere in the world , like now? We , like now? We , like now? We have created a have created a have created a multimodal multimodal multimodal agent with full agent with full agent with full control over your control over your control over your computer. So computer. So computer. So instead of instead of instead of calling your mom calling your mom calling your mom to fix a to fix a to fix a CUDA memory error, you CUDA memory error, you CUDA memory error, you can just can just can just call call call your Mac via FaceTime. your Mac via FaceTime. your Mac via FaceTime. Let me show you Let me show you Let me show you how it works. how it works. how it works. Let's say you're Let's say you're Let's say you're training a model training a model training a model at home, but you've already left and just at home, but you've already left and just at home, but you've already left and just realized that a realized that a CUDA memory error has occurred. So, what's the CUDA memory error has occurred. So, what's the easiest way easiest way easiest way to fix this? Well, to fix this? Well, to fix this? Well, instead of instead of instead of driving back home, driving back home, driving back home, you can just you can just you can just call call call your computer and your computer and your computer and tell it to do it tell it to do it tell it to do it itself. You itself. You itself. You can can can call him call him call him and and and text him via text him via text him via FaceTime. And this will launch the FaceTime. And this will launch the FaceTime. And this will launch the computer agent computer agent computer agent right here to start a right here to start a right here to start a FaceTime call so I FaceTime call so I FaceTime call so I can say what I want to can say what I want to can say what I want to do. do. Yes, they did a Yes, they did a very interesting thing. very interesting thing. Let me Let me click on that. click on that. click on that. Now you see what the Now you see what the Now you see what the system says.

system says. system says. Can you fix Can you fix Can you fix my CUDA memory error my CUDA memory error my CUDA memory error in VS Code? in VS Code? in VS Code? Okay, the task is Okay, the task is to fix the to fix the to fix the CUDA memory error in VS Code. I'll CUDA memory error in VS Code. I'll CUDA memory error in VS Code. I'll press Command + Space to press Command + Space to press Command + Space to open VS Code. open VS Code. So, all this So, all this is done is done is done automatically automatically automatically by the computer. by the computer. by the computer. CUDA memory error. CUDA memory error. CUDA memory error. This file seems to be throwing a This file seems to be throwing a This file seems to be throwing a CUDA memory error. CUDA memory error. CUDA memory error. Judging by the comments, the Judging by the comments, the Judging by the comments, the tensor size is tensor size is tensor size is too large, so too large, so too large, so it should be reduced it should be reduced it should be reduced to acceptable to acceptable to acceptable values. values. Yes, I think you get Yes, I think you get the idea. The third the idea. The third the idea. The third idea is to create idea is to create idea is to create an application that allows you to an application that allows you to an application that allows you to control control control your computer via your computer via your computer via FaceTime. And FaceTime. And FaceTime. And use FaceTime use FaceTime use FaceTime to debug to debug to debug code in VS Code. This code in VS Code. This code in VS Code. This happened in October happened in October happened in October last year. This last year. This last year. This was before Open was before Open was before Open Claw started as a project. Claw started as a project. Then we wondered Then we wondered what the second phase of what the second phase of what the second phase of computer agents would be like. computer agents would be like. computer agents would be like. We have already surpassed We have already surpassed We have already surpassed human human human test scores. What next? We test scores. What next? We test scores. What next? We realized that there is an realized that there is an realized that there is an interesting concept interesting concept interesting concept called "pass to the power K".

called "pass to the power K". Previously, everyone Previously, everyone measured the measured the measured the "pass at K" indicator. This is quite "pass at K" indicator. This is quite "pass at K" indicator. This is quite popular when you popular when you popular when you task an task an task an agent to agent to agent to create code. Yes? If create code. Yes? If create code. Yes? If the code doesn't work, you the code doesn't work, you the code doesn't work, you add a few add a few add a few hints and it hints and it hints and it generates another code. generates another code. generates another code. If one of the If one of the If one of the code options code options code options worked, the worked, the worked, the task is considered task is considered task is considered completed. Because you are not completed. Because you are not completed. Because you are not interested in what happened interested in what happened interested in what happened before. You're before. You're before. You're only interested in whether you managed to only interested in whether you managed to only interested in whether you managed to get a working code, get a working code, get a working code, right? right? right? Using Using Using the computer is very the computer is very the computer is very different different different because we started because we started because we started getting clients 2 getting clients 2 getting clients 2 years ago. And all years ago. And all years ago. And all the clients asked me the clients asked me the clients asked me this question: Can I this question: Can I this question: Can I actually put actually put actually put your agents on a your agents on a your agents on a server and run server and run server and run this workflow this workflow this workflow every day? And I won't have to every day? And I won't have to every day? And I won't have to constantly constantly constantly monitor these monitor these monitor these workflows. workflows. workflows. I won't need to I won't need to I won't need to debug them. debug them. debug them. So, basically, everyone So, basically, everyone So, basically, everyone was telling me about the was telling me about the was telling me about the "pass@k" metric. This "pass@k" metric. This "pass@k" metric. This means that I means that I means that I run the agent K run the agent K run the agent K times, and it times, and it times, and it should succeed each time. And should succeed each time. And should succeed each time. And this metric is not this metric is not this metric is not taken into account by any of the taken into account by any of the taken into account by any of the existing existing existing benchmarks. So we benchmarks. So we benchmarks. So we started investigating started investigating started investigating this problem. We took this problem. We took this problem. We took the agents with the best the agents with the best the agents with the best performance in the performance in the performance in the OS world. Their OS world. Their OS world. Their performance is performance is performance is quite good in three quite good in three quite good in three different categories.

different categories. different categories. Then we ran Then we ran Then we ran these same agents these same agents these same agents three times. We measured three times. We measured three times. We measured the performance of "pass@3" the performance of "pass@3" , right? This , right? This , right? This graph shows that graph shows that graph shows that all agents' all agents' all agents' performance performance performance has decreased. They has decreased. They has decreased. They lost almost 10% of their lost almost 10% of their success rate. Even the success rate. Even the success rate. Even the best agent best agent best agent among them. The second among them. The second among them. The second part is about the part is about the part is about the perturbed environment perturbed environment : my query might be : my query might be : my query might be a little different, and a little different, and a little different, and the environment might be the environment might be the environment might be a little different each time a little different each time a little different each time . And . And . And the situation is getting even the situation is getting even the situation is getting even worse. This is still a worse. This is still a worse. This is still a problem for problem for problem for computer agents computer agents computer agents today, because you today, because you today, because you see Cloud X has see Cloud X has see Cloud X has agents. Cloud has agents. Cloud has agents. Cloud has agents. None of them agents. None of them agents. None of them really really really provide a reliable provide a reliable provide a reliable and repeatable and repeatable and repeatable process for you. This is a process for you. This is a process for you. This is a recent article that recent article that recent article that we published we published we published about 2 months about 2 months about 2 months ago. In it we ago. In it we ago. In it we discuss this discuss this discuss this problem. problem. problem. In this In this In this article, we offer a simple trick article, we offer a simple trick article, we offer a simple trick that anyone can that anyone can that anyone can adopt in the adopt in the adopt in the future if future if future if they want their agents they want their agents they want their agents to work more reliably.

to work more reliably. The trick is to The trick is to ask the agent to ask the agent to ask the agent to ask you ask you ask you clarifying clarifying clarifying questions when you questions when you questions when you give him give him give him instructions. The article instructions. The article instructions. The article says that says that says that if the agent always if the agent always if the agent always asks you about asks you about asks you about ambiguous points ambiguous points ambiguous points in your request. in your request. The reliability of the agent The reliability of the agent will be higher. There is a prompt in the article will be higher. There is a prompt in the article will be higher. There is a prompt in the article that everyone that everyone that everyone can see. It's can see. It's can see. It's pretty simple. It pretty simple. It pretty simple. It goes something like this goes something like this goes something like this : your goal is : your goal is to minimize to minimize to minimize clarification clarification clarification while maintaining the while maintaining the while maintaining the essence of the task. This essence of the task. This essence of the task. This line is very important. line is very important. line is very important. You add this prompt to the You add this prompt to the You add this prompt to the agent and it starts agent and it starts agent and it starts working better. working better. After adding After adding this clarification this clarification this clarification prompt, you see that the prompt, you see that the prompt, you see that the overall reliability overall reliability overall reliability of the agent improves. The of the agent improves. The dotted line is the dotted line is the initial initial initial performance without performance without performance without refinements. And the solid refinements. And the solid refinements. And the solid line is the one with line is the one with line is the one with the clarification, right? You the clarification, right? You the clarification, right? You see that Cloud is getting see that Cloud is getting see that Cloud is getting better. And Agent S from GPT- better. And Agent S from GPT- 5, interestingly, has the 5, interestingly, has the 5, interestingly, has the biggest improvements biggest improvements . So, the purple . So, the purple . So, the purple line is a model- line is a model- independent structure.

independent structure. independent structure. And the green line is, And the green line is, And the green line is, you know, computer you know, computer you know, computer agents from large agents from large agents from large laboratories. Okay, laboratories. Okay, laboratories. Okay, so basically this is our so basically this is our so basically this is our path to creating a path to creating a path to creating a VOM agent that VOM agent that VOM agent that seems to work seems to work seems to work pretty well. And pretty well. And pretty well. And people still use the people still use the people still use the Agent S framework Agent S framework Agent S framework because it is because it is because it is model-independent. You can model-independent. You can model-independent. You can connect any connect any connect any open open open source models. I'm source models. I'm source models. I'm also also also looking forward to the results with looking forward to the results with looking forward to the results with the latest large the latest large the latest large open model, like open model, like GLM 5.2, to see GLM 5.2, to see GLM 5.2, to see what what what the performance looks like there. And the the performance looks like there. And the the performance looks like there. And the second question: “Okay second question: “Okay , this doesn’t solve the , this doesn’t solve the , this doesn’t solve the reliability problem, reliability problem, reliability problem, right? It seems like these right? It seems like these right? It seems like these agents don't have...as long as agents don't have...as long as agents don't have...as long as you're using a you're using a you're using a neural network, as long as neural network, as long as neural network, as long as you're using VOM, you're using VOM, you're using VOM, it's ultimately a it's ultimately a it's ultimately a probabilistic model. probabilistic model. It gives a probability every time It gives a probability every time . So it . So it . So it won't give the same won't give the same won't give the same performance if performance if performance if you run it you run it you run it 24/7. 24/7. Then we had to Then we had to think about the think about the think about the next step, next step, next step, namely the symbolic namely the symbolic namely the symbolic level. We call level. We call level. We call this an encoding agent. And this an encoding agent. And this an encoding agent. And we opened the source we opened the source we opened the source code of this language. We code of this language. We code of this language. We call it SimuLand. This is a call it SimuLand. This is a call it SimuLand. This is a language for simulation.

language for simulation. language for simulation. We call it SimuLand. We call it SimuLand. We call it SimuLand. So this language, if you're So this language, if you're So this language, if you're familiar with the familiar with the familiar with the RPA industry, is similar to what the RPA industry, is similar to what the RPA industry, is similar to what the RPA industry has done RPA industry has done RPA industry has done before. They ask before. They ask before. They ask engineers to write code. engineers to write code. engineers to write code. But a common But a common But a common problem for RPA is that problem for RPA is that there is no single there is no single protocol common protocol common protocol common to the entire community to the entire community to the entire community that that that everyone can use. everyone can use. everyone can use. And we're trying to And we're trying to And we're trying to build this for build this for browser automation, we have browser automation, we have Playwright, it's a standard Playwright, it's a standard Playwright, it's a standard way. But there way. But there really is really is no industry no industry no industry standard in computer usage. That's why standard in computer usage. That's why standard in computer usage. That's why we opened this code. we opened this code. we opened this code. We hope We hope We hope this will be useful to all this will be useful to all this will be useful to all developers so that you developers so that you developers so that you can create your can create your can create your own own own scraping layer for different scraping layer for different scraping layer for different workflows workflows workflows that will be reliable and that will be reliable and that will be reliable and deterministic deterministic deterministic every time. So every time. So every time. So basically the goal is to basically the goal is to basically the goal is to allow allow allow you to build you to build you to build source code for Simulan source code for Simulan source code for Simulan and run it in and run it in and run it in your terminal, and your terminal, and use it use it in Cloud Code. So I want to in Cloud Code. So I want to in Cloud Code. So I want to do a quick do a quick do a quick demonstration. Here. demonstration. Here. demonstration. Here. Allow me. Okay.

Allow me. Okay. Okay, I'm in Cloud Code now. So, if you So, if you install...maybe install...maybe install...maybe I should show I should show I should show you...Simulan. Yes, that's the you...Simulan. Yes, that's the you...Simulan. Yes, that's the landing page, and landing page, and landing page, and if you go if you go if you go there, there's a simple there, there's a simple there, there's a simple one-line one-line one-line installer, npm install. installer, npm install. installer, npm install. Once installed, Once installed, Once installed, you will be able to you will be able to you will be able to use it use it use it in your own terminal. in your own terminal. I'm a Cloud Code user and I'm a Cloud Code user and I like to I like to I like to see this because see this because see this because no one writes code no one writes code no one writes code by hand these days, right? by hand these days, right? by hand these days, right? It is important It is important It is important to use " to use " live" programming live" programming live" programming for such for such for such automation. So, automation. So, automation. So, after installation, after installation, after installation, a skill appears a skill appears a skill appears that works in tandem with that works in tandem with that works in tandem with Simulan, so you can Simulan, so you can Simulan, so you can just just just use it. I use it. I use it. I can say: go can say: go can say: go to the AIE website and summarize the to the AIE website and summarize the to the AIE website and summarize the events. Okay, let's events. Okay, let's events. Okay, let's see how it see how it see how it works. He is currently works. He is currently works. He is currently checking the versions, these are checking the versions, these are checking the versions, these are purely technical purely technical purely technical points. But he points. But he points. But he understands that he is understands that he is understands that he is going to going to going to use Simulan to use Simulan to use Simulan to launch the AI launch the AI launch the AI Engineer World Fair website.

Engineer World Fair website. We continue. You We continue. You may notice that may notice that may notice that it is trying it is trying it is trying to use Simulan in JS. to use Simulan in JS. to use Simulan in JS. This is the JavaScript language. And now This is the JavaScript language. And now This is the JavaScript language. And now Clockwork will try to Clockwork will try to Clockwork will try to write code write code write code using the using the using the Simulan protocol. I guess Simulan protocol. I guess Simulan protocol. I guess it's because Clockwork is it's because Clockwork is it's because Clockwork is new. Therefore, he new. Therefore, he new. Therefore, he needs to make needs to make needs to make a lot of technical a lot of technical a lot of technical adjustments. I adjustments. I adjustments. I allow reading for allow reading for allow reading for everything. You might want everything. You might want everything. You might want to consider this to consider this to consider this approach. This is a different approach. This is a different approach. This is a different approach to how approach to how approach to how computers work. It computers work. It computers work. It effectively reduces effectively reduces effectively reduces computer problems computer problems computer problems to to to programming tasks. I have programming tasks. I have programming tasks. I have everything I need.

everything I need. everything I need. Let me write a Let me write a Let me write a script that script that script that opens a website and opens a website and opens a website and extracts extracts extracts content from it. And you see the content from it. And you see the content from it. And you see the agent writing this code. agent writing this code. After that, he After that, he opens the website. opens the website. opens the website. Automatically, so Simulan Automatically, so Simulan Automatically, so Simulan runs on your runs on your runs on your computer. If you computer. If you computer. If you have a virtual machine, have a virtual machine, have a virtual machine, you can deploy you can deploy you can deploy it there. So, here it is. And it there. So, here it is. And it there. So, here it is. And then, you know, after then, you know, after then, you know, after Clockwork Clockwork Clockwork ran the code, it ran the code, it ran the code, it recognized certain recognized certain recognized certain errors. After receiving errors. After receiving errors. After receiving errors, he errors, he errors, he tries to tries to tries to fix the code. It's fix the code. It's fix the code. It's almost, if you turn on almost, if you turn on almost, if you turn on auto-approval, auto-approval, auto-approval, actually a actually a actually a self-improving self-improving self-improving computer agent. computer agent. A certain level of A certain level of self-improvement is self-improvement is self-improvement is present. I want present. I want present. I want to see Him. He's to see Him. He's to see Him. He's not there. No, he's here. You not there. No, he's here. You not there. No, he's here. You can see that Clockwork can see that Clockwork can see that Clockwork decided to use an decided to use an decided to use an accessibility snapshot accessibility snapshot accessibility snapshot instead of a screenshot instead of a screenshot instead of a screenshot because this is a because this is a generalization task. Okay, it's almost here generalization task. Okay, it's almost here . I . I . I get some Yes, get some Yes, get some Yes, actually, the summary. Yes, actually, the summary. Yes, actually, the summary. Yes, he summarized all the he summarized all the he summarized all the events for AI Engineer.

events for AI Engineer. events for AI Engineer. Okay, I'll stop here. Okay, I'll stop here. Okay, I'll stop here. And I want to show you the And I want to show you the And I want to show you the full code, which would be full code, which would be full code, which would be interesting interesting interesting to look at. Yes, this is our to look at. Yes, this is our to look at. Yes, this is our full code. It full code. It full code. It looks like JavaScript, well, just looks like JavaScript, well, just looks like JavaScript, well, just like any other like any other like any other JavaScript. It opens JavaScript. It opens JavaScript. It opens a window, takes a picture, a window, takes a picture, a window, takes a picture, and writes it to a file. and writes it to a file. Okay, let me Okay, let me get back to my get back to my get back to my presentation. So, I presentation. So, I presentation. So, I was talking about two was talking about two was talking about two completely different completely different completely different approaches. They are approaches. They are approaches. They are orthogonal. This is not the orthogonal. This is not the orthogonal. This is not the same same same approach. But the question is approach. But the question is approach. But the question is , do we really , do we really , do we really need two need two need two frameworks? Yes. That's right frameworks? Yes. That's right frameworks? Yes. That's right . They are trying . They are trying . They are trying to compete with each to compete with each to compete with each other. This brings other. This brings other. This brings us back to a very interesting us back to a very interesting us back to a very interesting and important topic in and important topic in and important topic in deep learning, deep learning, deep learning, because when I worked because when I worked because when I worked at DeepMind, I specifically at DeepMind, I specifically at DeepMind, I specifically focused on a focused on a focused on a problem problem problem called called called continuous continuous continuous learning. That's why learning. That's why learning. That's why we've been doing all this we've been doing all this we've been doing all this research on research on research on artificial intelligence systems.

artificial intelligence systems. artificial intelligence systems. But in the end, we But in the end, we But in the end, we realized that there is realized that there is no universal solution. No matter what no universal solution. No matter what technology you technology you technology you use, you use, you use, you still have to still have to still have to address these two address these two address these two issues as a issues as a issues as a trade-off. Any trade-off. Any trade-off. Any training system training system training system must learn new things must learn new things must learn new things and at the same time and at the same time and at the same time guarantee the successful guarantee the successful guarantee the successful completion of tasks that completion of tasks that completion of tasks that it has already it has already it has already mastered before. mastered before. This is called the This is called the stability- stability- stability- plasticity dilemma. You plasticity dilemma. You plasticity dilemma. You want the want the want the education system to be education system to be education system to be stable. But stable. But stable. But at the same time, you want it to have at the same time, you want it to have the ability the ability to adapt to to adapt to to adapt to new environments. It is new environments. It is never the case that never the case that you are equally you are equally you are equally good at both good at both good at both aspects. So, this is the aspects. So, this is the aspects. So, this is the conclusion. Yes, it's conclusion. Yes, it's conclusion. Yes, it's always a compromise. always a compromise. always a compromise. Then we can Then we can Then we can think about how think about how think about how to balance these two to balance these two to balance these two things? And if you things? And if you things? And if you look at the two look at the two look at the two systems that I systems that I systems that I demonstrated, these demonstrated, these demonstrated, these two systems are two systems are coding agents and coding agents and real agents, right?

real agents, right? real agents, right? They actually They actually They actually represent the two represent the two represent the two extremes of this extremes of this extremes of this learning system. The learning system. The learning system. The coding agent coding agent coding agent creates code and always creates code and always creates code and always acts decisively, but it is acts decisively, but it is acts decisively, but it is quite stubborn. Yes? quite stubborn. Yes? quite stubborn. Yes? If you run If you run If you run code every day and the code every day and the code every day and the environment environment environment changes, the code changes, the code changes, the code breaks. This is the breaks. This is the breaks. This is the problem. And a problem. And a problem. And a real agent is real agent is real agent is adaptive, but the adaptive, but the adaptive, but the problem is that you problem is that you problem is that you can't can't can't rely on it rely on it rely on it to repeat the same to repeat the same to repeat the same process process process every time. So, the main every time. So, the main every time. So, the main question is: can we question is: can we question is: can we find a middle ground find a middle ground find a middle ground between them? between them? Because the main goal is the best is the best is the best adaptability and the adaptability and the adaptability and the best reliability, best reliability, best reliability, right? We also right? We also right? We also call this curve the " call this curve the " Pareto front". Can Pareto front". Can Pareto front". Can we actually we actually we actually find the perfect find the perfect find the perfect spot there? This is becoming a spot there? This is becoming a spot there? This is becoming a current area that current area that current area that our our our company is specifically company is specifically company is specifically focusing on. focusing on. Try to find a Try to find a single structure in single structure in single structure in which we could which we could which we could combine these two combine these two combine these two approaches into one approaches into one approaches into one principle, which we principle, which we principle, which we call neuro- call neuro- symbolic, right? symbolic, right? symbolic, right? Because there are neural Because there are neural Because there are neural and symbolic and symbolic and symbolic approaches. Is there a way approaches. Is there a way approaches. Is there a way to combine these two to combine these two to combine these two technologies into one technologies into one technologies into one system that is constantly system that is constantly system that is constantly self-improving, self-improving, self-improving, reliable, and reliable, and reliable, and adaptable to adaptable to adaptable to environmental changes? So I wouldn't environmental changes? So I wouldn't environmental changes? So I wouldn't say we've say we've say we've solved this problem.

solved this problem. solved this problem. I wouldn't say we've I wouldn't say we've I wouldn't say we've solved it, but we're solved it, but we're solved it, but we're already at the already at the already at the initial stage of initial stage of initial stage of moving in that direction moving in that direction . Because there is a clear . Because there is a clear . Because there is a clear paradigm that we paradigm that we paradigm that we can apply right can apply right can apply right now. We have a VILM now. We have a VILM agent that is better than a agent that is better than a agent that is better than a human in small human in small human in small tests. And we have a tests. And we have a tests. And we have a coding protocol that allows us coding protocol that allows us to write code to write code using an using an using an agent that creates agent that creates agent that creates deterministic deterministic deterministic workflows. This is an easy workflows. This is an easy workflows. This is an easy way to conduct way to conduct way to conduct such a new such a new such a new experiment. We experiment. We experiment. We let the VILM agent let the VILM agent let the VILM agent run in run in run in the environment, and it the environment, and it the environment, and it constantly writes code constantly writes code constantly writes code to a file. When this to a file. When this to a file. When this workflow workflow workflow completes successfully completes successfully , you get , you get , you get ready-to-use code. ready-to-use code. ready-to-use code. The next time The next time The next time we start the agent, we start the agent, we start the agent, we can we can we can execute this code first execute this code first execute this code first instead of starting the instead of starting the instead of starting the VILM agent itself. Yes? VILM agent itself. Yes? If it doesn't work, If it doesn't work, or the environment or the environment or the environment changes and everything changes and everything changes and everything breaks, as I just breaks, as I just breaks, as I just showed, there is a showed, there is a showed, there is a self-healing loop using the self-healing loop using the encoding agent. This is getting encoding agent. This is getting very interesting. And...yes, very interesting. And...yes, very interesting. And...yes, I added this slide.

I added this slide. I added this slide. Another interesting story: Another interesting story: Another interesting story: at this year's Cal Hacks at this year's Cal Hacks , at the same hackathon, , at the same hackathon, those who those who used used used our agent also took first place. But our agent also took first place. But our agent also took first place. But this time they this time they didn't use the agent to didn't use the agent to create a product. create a product. create a product. They used They used They used it to test the it to test the it to test the product. This is another product. This is another product. This is another interesting niche that interesting niche that all of us should pay attention to, because we are all of us should pay attention to, because we are developers, right? developers, right? developers, right? We write code, we know We write code, we know We write code, we know that there are that there are that there are more and more more and more more and more merge requests and code now, so merge requests and code now, so merge requests and code now, so who is going to test all this who is going to test all this who is going to test all this ? The solution ? The solution ? The solution to this is to this is to this is computer agents. computer agents. computer agents. In fact, it is an agent In fact, it is an agent In fact, it is an agent that can move that can move that can move the mouse, click on the the mouse, click on the the mouse, click on the control panel, and control panel, and control panel, and perform perform perform testing for you. Not testing for you. Not testing for you. Not sure if I have sure if I have sure if I have time. If I have time, I time. If I have time, I time. If I have time, I really want to show a really want to show a really want to show a demo about testing, demo about testing, demo about testing, but I can try. but I can try. Let me try . Let me show . Let me show . Let me show you the cycle that we have you the cycle that we have you the cycle that we have in the product. These are just the in the product. These are just the in the product. These are just the first steps. This first steps. This first steps. This works in many works in many works in many cases, but sometimes cases, but sometimes some problems still arise. So, some problems still arise. So, um. I look forward to um. I look forward to um. I look forward to everyone everyone everyone testing the product testing the product testing the product and giving us feedback and giving us feedback . Okay, let . Okay, let . Okay, let me. I'm not sure me. I'm not sure me. I'm not sure if this works. Oh, here it is if this works. Oh, here it is . OK. So, this is a . OK. So, this is a . OK. So, this is a valid virtual valid virtual valid virtual machine. This is an agent, it's machine. This is an agent, it's machine. This is an agent, it's not...As you can see, I have a

not...As you can see, I have a not...As you can see, I have a lot of lot of lot of virtual machines here. And virtual machines here. And virtual machines here. And I give them different names. I give them different names. Um, let me Um, let me try this one. So, try this one. So, try this one. So, to enable to enable to enable neurosymbol neurosymbol neurosymbol mode, you need to mode, you need to mode, you need to go to this go to this go to this control panel. Click control panel. Click control panel. Click on "agent". There are on "agent". There are on "agent". There are many special many special many special algorithms here, algorithms here, algorithms here, for example, for example, for example, humanized humanized humanized control, where the control, where the control, where the trajectory trajectory trajectory is different. is different. Classical integration. Classical integration. So you need to So you need to So you need to look at similar look at similar stream playback code. This is a specific stream playback code. This is a specific experiment that we are experiment that we are experiment that we are currently conducting. The whole currently conducting. The whole currently conducting. The whole idea is in idea is in idea is in the diagram I just the diagram I just the diagram I just showed. If you showed. If you showed. If you enable this, we can enable this, we can enable this, we can try something. try something. try something. Let me Let me Let me make sure the make sure the make sure the desktop is there. desktop is there. desktop is there. Good. So, we can Good. So, we can Good. So, we can try this...Yes, try this...Yes, try this...Yes, we can still we can still we can still try try try character mode. character mode. character mode. Just want to give you a Just want to give you a Just want to give you a general idea general idea general idea of ​​how it works. of ​​how it works. of ​​how it works. Go to...AIE. web Go to...AIE. web pages and...find the pages and...find the pages and...find the first speaker on first speaker on first speaker on Tuesday. Oh, there are...There are always Tuesday. Oh, there are...There are always Tuesday. Oh, there are...There are always problems during problems during problems during live demonstrations.

live demonstrations. live demonstrations. And then...Let me And then...Let me And then...Let me restart the agent restart the agent . I'll see if . I'll see if . I'll see if my VPN works. So, while my VPN works. So, while my VPN works. So, while we're waiting, let me we're waiting, let me we're waiting, let me show you another show you another show you another demonstration. Because demonstration. Because demonstration. Because I have a backup I have a backup . If the . If the . If the live demonstration doesn't work, live demonstration doesn't work, live demonstration doesn't work, we can show we can show we can show another one. This is a collaboration with another one. This is a collaboration with another one. This is a collaboration with Microsoft. Microsoft Microsoft. Microsoft Microsoft. Microsoft is also building this computing is also building this computing is also building this computing infrastructure, infrastructure, infrastructure, as they are the as they are the as they are the creators of Windows. So, creators of Windows. So, creators of Windows. So, these are virtual machines these are virtual machines these are virtual machines from Microsoft. It's from Microsoft. It's from Microsoft. It's called Microsoft called Microsoft called Microsoft Windows 365 for Agents. Windows 365 for Agents. Windows 365 for Agents. The idea is The idea is The idea is that... From this you can that... From this you can that... From this you can see, for example, the see, for example, the see, for example, the left side of left side of left side of the diagram. The idea is this: the diagram. The idea is this: the diagram. The idea is this: use use use our agents on our agents on Microsoft virtual machines to fill out Microsoft virtual machines to fill out forms in traditional ERP forms in traditional ERP systems. This is one of the systems. This is one of the systems. This is one of the biggest areas for biggest areas for computer computer use agents, because there use agents, because there use agents, because there are still many are still many are still many traditional industries.

traditional industries. traditional industries. They have nothing They have nothing They have nothing but APIs. They're still but APIs. They're still but APIs. They're still using old using old software on software on their Windows machines, their Windows machines, their Windows machines, right? So if you right? So if you right? So if you build apps build apps build apps that allow you that allow you that allow you to work in certain to work in certain to work in certain verticals, because verticals, because verticals, because we are not a vertical we are not a vertical we are not a vertical company, but a company, but a company, but a horizontal horizontal horizontal infrastructure. If infrastructure. If infrastructure. If you are interested in you are interested in you are interested in developing in different developing in different developing in different verticals, these are great verticals, these are great verticals, these are great opportunities. opportunities. opportunities. For example, some of For example, some of For example, some of our clients our clients our clients work with work with work with insurance websites. insurance websites. insurance websites. They specialize They specialize They specialize in insurance. in insurance. in insurance. Essentially, they Essentially, they Essentially, they use use use agents who agents who agents who go to go to go to insurance sites and insurance sites and insurance sites and pull pull pull data from there. And some of them data from there. And some of them data from there. And some of them work in work in work in healthcare. healthcare. healthcare. They collaborate with They collaborate with They collaborate with doctors. Doctors doctors. Doctors doctors. Doctors usually have old usually have old usually have old Windows computers with Windows computers with Windows computers with outdated outdated outdated ERP systems. You ERP systems. You ERP systems. You can can can use use use computer agents computer agents computer agents to automate to automate to automate data entry. So, there data entry. So, there data entry. So, there are a lot of ideas that are a lot of ideas that are a lot of ideas that people are exploring in people are exploring in people are exploring in this direction. So, this direction. So, this direction. So, hopefully this gives hopefully this gives hopefully this gives you some understanding of the you some understanding of the you some understanding of the situation. Oh, it situation. Oh, it situation. Oh, it works. Yes, works. Yes, works. Yes, let me let me since I already have a since I already have a few skills that are few skills that are few skills that are already there. Here is one of already there. Here is one of already there. Here is one of the skills. Let the skills. Let the skills. Let me delete it.

me delete it. Go to the AIE website and Go to the AIE website and see who see who see who will be speaking first on will be speaking first on will be speaking first on Tuesday. I already have a Tuesday. I already have a Tuesday. I already have a website. Let me website. Let me website. Let me create a channel. I will create a channel. I will create a channel. I will create a new one. And you create a new one. And you create a new one. And you can see all the can see all the can see all the logs here. What's interesting is logs here. What's interesting is logs here. What's interesting is that the agent writes the code that the agent writes the code that the agent writes the code almost line by line almost line by line . And then it . And then it . And then it combines all the code into combines all the code into combines all the code into one large file. one large file. Now we move on Now we move on to the schedule page. to the schedule page. Okay, so you Okay, so you get the information get the information . And if you noticed, the . And if you noticed, the . And if you noticed, the agent doesn't actually agent doesn't actually agent doesn't actually use use use a screenshot. When you a screenshot. When you a screenshot. When you try to find try to find try to find information, the agent decides for itself information, the agent decides for itself information, the agent decides for itself : take : take : take a screenshot or get an a screenshot or get an a screenshot or get an availability tree?

availability tree? availability tree? This time he This time he This time he uses an uses an uses an accessibility tree. So you accessibility tree. So you accessibility tree. So you can see all can see all can see all the information the information the information contained inside at a contained inside at a contained inside at a glance. And now I'll glance. And now I'll glance. And now I'll show you how to show you how to show you how to keep it as a keep it as a keep it as a habit. If I habit. If I habit. If I enable the new enable the new enable the new character mode and character mode and character mode and click "save as click "save as click "save as skill", that skill skill", that skill skill", that skill will get all the code will get all the code will get all the code that was just that was just that was just executed in the executed in the executed in the previous session. previous session. Now we can Now we can look at the look at the look at the AI ​​speakers for the day. AI ​​speakers for the day. Let me Let me find this skill. find this skill. Although I didn't find it. Okay, so this Okay, so this skill is saved, skill is saved, skill is saved, so I'll try another so I'll try another so I'll try another query for another day. query for another day.

Let's try Let's try Wednesday. You see, the Wednesday. You see, the replay mode has appeared. replay mode has appeared. The interface is a little The interface is a little The interface is a little different because it different because it different because it has ready-made code. has ready-made code. has ready-made code. He has a code, and he He has a code, and he He has a code, and he changes the parameters of changes the parameters of changes the parameters of that code. If that code. If that code. If you click on the you click on the you click on the activity log, you can activity log, you can activity log, you can see that the see that the see that the agent agent agent generates significantly generates significantly generates significantly more code in the first step. So it's the more code in the first step. So it's the more code in the first step. So it's the same code with same code with same code with different parameters different parameters different parameters inside. Uh, it's Tuesday by inside. Uh, it's Tuesday by inside. Uh, it's Tuesday by default default default , right? The default , right? The default , right? The default value is Tuesday. value is Tuesday. value is Tuesday. If you set If you set If you set Wednesday, it will, uh, Wednesday, it will, uh, Wednesday, it will, uh, change the setting. Here it is change the setting. Here it is . And there is a slight . And there is a slight . And there is a slight delay here. You see delay here. You see delay here. You see that there was a that there was a that there was a slight delay for the agent. slight delay for the agent. You need to You need to filter this out. filter this out. filter this out. He finds He finds He finds information and successfully information and successfully information and successfully repeats it. There was a repeats it. There was a repeats it. There was a slight delay. And slight delay. And slight delay. And after a delay, the agent, after a delay, the agent, after a delay, the agent, uh, switched to a uh, switched to a uh, switched to a neural model, neural model, neural model, that is, a VLM agent. So that is, a VLM agent. So that is, a VLM agent. So first he always first he always first he always tries to create tries to create tries to create all the code as much as all the code as much as all the code as much as possible. Because possible. Because possible. Because if you have a high enough if you have a high enough success rate, it success rate, it will ensure a very will ensure a very will ensure a very determined and determined and determined and successful successful successful workflow. In addition, workflow. In addition, workflow. In addition, the agent has a verification process in place the agent has a verification process in place the agent has a verification process in place . So, . So, . So, every time it every time it every time it executes code, executes code, verification occurs. If verification occurs. If something is wrong, it something is wrong, it something is wrong, it stops. It's like stops. It's like ...it's like when I ...it's like when I ...it's like when I was in school, we was in school, we was in school, we wrote down everything wrote down everything wrote down everything the teacher said in a the teacher said in a the teacher said in a notebook as notebook as notebook as instructions, right? And the

instructions, right? And the instructions, right? And the next time next time next time I want to do something, I I want to do something, I I want to do something, I look at the notebook and look at the notebook and look at the notebook and try to follow try to follow try to follow all the steps one by one. And all the steps one by one. And all the steps one by one. And if the environment is different, if the environment is different, if the environment is different, I notice: "Okay, I'm I notice: "Okay, I'm I notice: "Okay, I'm trying to do trying to do trying to do this action, but this action, but this action, but the environment doesn't the environment doesn't the environment doesn't allow me to allow me to allow me to do it." And then I have to do it." And then I have to do it." And then I have to think more, think more, think more, spend more spend more spend more time, you time, you time, you know, exploring the know, exploring the know, exploring the new environment and new environment and new environment and trying to figure out trying to figure out what strategy would be what strategy would be what strategy would be best. Okay, best. Okay, best. Okay, let's move on. Uh, I let's move on. Uh, I let's move on. Uh, I think this is a good think this is a good think this is a good direction for us and direction for us and direction for us and even for the entire even for the entire even for the entire industry to think industry to think industry to think about what better about what better about what better agents we can agents we can agents we can build with that build with that build with that balance of flexibility balance of flexibility balance of flexibility and reliability. And we and reliability. And we and reliability. And we see quite good see quite good see quite good success in these kinds of success in these kinds of success in these kinds of experiments. It's like, experiments. It's like, experiments. It's like, okay, I think the point is okay, I think the point is okay, I think the point is that you no longer that you no longer that you no longer have to spend have to spend have to spend the same amount of the same amount of the same amount of tokens every time with tokens every time with tokens every time with this kind of technology, this kind of technology, this kind of technology, because the second time because the second time because the second time you run the you run the you run the workflow, workflow, workflow, maybe 50% of the effort maybe 50% of the effort maybe 50% of the effort is done by code is done by code is done by code without any queries to the without any queries to the without any queries to the language model, right? And language model, right? And language model, right? And you can significantly you can significantly you can significantly reduce your reduce your reduce your costs. So we're costs. So we're costs. So we're also also also experimenting with experimenting with experimenting with this product, we're this product, we're this product, we're offering people a offering people a offering people a $100 plan with $100 plan with unlimited tasks.

unlimited tasks. So, the tariff plan So, the tariff plan So, the tariff plan has also changed. This won't has also changed. This won't has also changed. This won't always depend always depend always depend on usage, on usage, on usage, because as a human, when I because as a human, when I because as a human, when I do a task 100 do a task 100 do a task 100 times, I get so times, I get so times, I get so used to it that I used to it that I used to it that I can do it all can do it all can do it all almost with my almost with my almost with my eyes closed. Isn't that right? But eyes closed. Isn't that right? But eyes closed. Isn't that right? But today's agents today's agents today's agents don't have that don't have that don't have that option. Every time option. Every time option. Every time you perform the you perform the you perform the same task— same task— even a thousand or even a thousand or even a thousand or ten thousand times, ten thousand times, ten thousand times, even with the same even with the same even with the same query—you still query—you still query—you still pay the same amount to pay the same amount to pay the same amount to OpenAI, which doesn't seem OpenAI, which doesn't seem OpenAI, which doesn't seem very reasonable or very reasonable or very reasonable or very economical. So very economical. So very economical. So hopefully this gives hopefully this gives hopefully this gives you a good idea you a good idea you a good idea of ​​the direction of ​​the direction of ​​the direction we're heading in, and I we're heading in, and I we're heading in, and I encourage everyone to give encourage everyone to give encourage everyone to give it a try it a try it a try as we actively and as we actively and as we actively and rapidly refine rapidly refine rapidly refine neurosymbolic neurosymbolic neurosymbolic agents in the coming agents in the coming agents in the coming months, and months, and months, and hopefully hopefully hopefully we can make a we can make a we can make a breakthrough in this breakthrough in this breakthrough in this direction. The second direction. The second direction. The second problem we problem we problem we have to think about is: what should have to think about is: what should have to think about is: what should we do with we do with we do with the computer? Two years the computer? Two years the computer? Two years ago, we were working on ago, we were working on autonomous technology on autonomous technology on your computer. We your computer. We your computer. We had a product where the had a product where the had a product where the agent ran agent ran agent ran locally on your locally on your locally on your computer, but computer, but computer, but we got a lot of we got a lot of we got a lot of complaints because everyone complaints because everyone complaints because everyone only had one computer, and only had one computer, and only had one computer, and when the agent was running, when the agent was running, when the agent was running, it took up the entire screen it took up the entire screen and you couldn't and you couldn't and you couldn't do anything. This do anything. This do anything. This got us got us got us thinking: thinking: thinking: suppose we have a

suppose we have a suppose we have a human- human- human- level agent working. level agent working. level agent working. Let's say we have Let's say we have Let's say we have something that works. This is something that works. This is something that works. This is not quite true yet. We are very not quite true yet. We are very not quite true yet. We are very close to this. But close to this. But close to this. But when that day comes when that day comes when that day comes and we have a and we have a and we have a working agent, we will have to working agent, we will have to working agent, we will have to think about what think about what think about what to do with the computer. to do with the computer. to do with the computer. Do you still need Do you still need Do you still need one computer, or one computer, or one computer, or maybe you need maybe you need maybe you need ten? Our ten? Our ten? Our observation observation observation shows that once shows that once shows that once something starts something starts something starts working, working, working, users always users always users always ask: "Can you ask: "Can you ask: "Can you put this put this put this agent on a separate agent on a separate agent on a separate machine?" "Can you machine?" "Can you machine?" "Can you put it on a put it on a put it on a server?" This is always the same server?" This is always the same conclusion of most conclusion of most users. And then users. And then users. And then they start they start they start asking for more cars. asking for more cars. asking for more cars. So there is a certain So there is a certain So there is a certain trajectory. There is a certain trajectory. There is a certain trajectory. There is a certain trajectory. First, trajectory. First, trajectory. First, we are at the we are at the we are at the stage where stage where stage where we are trying to get the we are trying to get the we are trying to get the agents agents agents working. When working. When working. When they start working, you they start working, you they start working, you will need 10 machines, will need 10 machines, will need 10 machines, 100 machines. That's why it 100 machines. That's why it 100 machines. That's why it motivates us to motivates us to motivates us to work on work on work on cloud cloud cloud computing. I have already computing. I have already computing. I have already briefly talked about this briefly talked about this briefly talked about this . I think the . I think the . I think the main idea is main idea is main idea is that if we that if we that if we have a cloud- have a cloud- have a cloud- managed managed managed desktop, it becomes so desktop, it becomes so desktop, it becomes so scalable, and scalable, and scalable, and eventually, I hope eventually, I hope , everyone can have 100 , everyone can have 100 , everyone can have 100 computers, so you computers, so you computers, so you become the become the become the CEO of your own CEO of your own CEO of your own AI organization, AI organization, AI organization, right? So you have 100 AI right? So you have 100 AI workers, and each one workers, and each one workers, and each one becomes 100 times more becomes 100 times more becomes 100 times more productive. So, productive. So, productive. So, yes, essentially, yes, essentially, yes, essentially, we launched the we launched the we launched the product in March this year. This is product in March this year. This is product in March this year. This is actually a fairly actually a fairly actually a fairly new product, and new product, and new product, and along with it is Sidebox, which is a along with it is Sidebox, which is a along with it is Sidebox, which is a virtual virtual virtual computer in the cloud. And from

computer in the cloud. And from computer in the cloud. And from this demo, you can this demo, you can this demo, you can already see that already see that already see that I personally have I personally have 20 machines running in the background. And this is the 20 machines running in the background. And this is the general scheme. Side general scheme. Side general scheme. Side actually interacts, and actually interacts, and actually interacts, and Side is called a Side Side is called a Side agent, which does the agent, which does the agent, which does the autonomous part, autonomous part, autonomous part, does does does the planning and the planning and the planning and decision-making, and decision-making, and decision-making, and then executes then executes then executes operations on the Sidebox with the operations on the Sidebox with the operations on the Sidebox with the cloud computer. cloud computer. And okay, we have And okay, we have a lot of friends who are a lot of friends who are a lot of friends who are developers, so developers, so developers, so they keep they keep they keep asking us, " asking us, " asking us, " Can you open up the API?" Can you open up the API?" Can you open up the API?" So today we're So today we're So today we're actually releasing an actually releasing an actually releasing an API to access the API to access the API to access the cloud computer. I cloud computer. I cloud computer. I don't think I have time don't think I have time don't think I have time to demo it, but to demo it, but to demo it, but it's actually pretty it's actually pretty it's actually pretty simple. It's simple. It's simple. It's called Sap Pi, SAP I, called Sap Pi, SAP I, called Sap Pi, SAP I, Sap Pi. And basically, you Sap Pi. And basically, you Sap Pi. And basically, you can do everything that can do everything that can do everything that I did on the web portal I did on the web portal , through your terminal. , through your terminal. , through your terminal. I use the I use the I use the CLI tool. And you CLI tool. And you CLI tool. And you can also can also can also use the use the use the API tool there. API tool there. API tool there. And we just released And we just released And we just released this website, so this website, so this website, so feel free to check feel free to check feel free to check it out. There's a lot of APIs and it out. There's a lot of APIs and it out. There's a lot of APIs and CLIs there. And once you CLIs there. And once you CLIs there. And once you sign up, you'll sign up, you'll sign up, you'll have a machine in a have a machine in a have a machine in a remote remote remote setup, you setup, you setup, you can can can use this use this use this command line and API command line and API command line and API to control that to control that to control that computer and computer and computer and integrate it into integrate it into integrate it into your own product.

your own product. your own product. You know. Yeah, I'm going to skip the You know. Yeah, I'm going to skip the You know. Yeah, I'm going to skip the live demo. live demo. live demo. So the general So the general So the general vision for this vision for this vision for this direction is that direction is that direction is that we want we want we want everyone to have one everyone to have one everyone to have one single agent. And that single agent. And that single agent. And that agent can agent can agent can access access access multiple platforms, multiple platforms, multiple platforms, not just Windows. So not just Windows. So not just Windows. So we also have we also have we also have Mac OS support. Linux Mac OS support. Linux Mac OS support. Linux is coming soon. is coming soon. is coming soon. Linux is very cost- Linux is very cost- Linux is very cost- effective. It's pretty effective. It's pretty effective. It's pretty cheap in terms of cheap in terms of cheap in terms of our pricing our pricing . As per our . As per our . As per our internal evaluation. And internal evaluation. And internal evaluation. And the interesting part is the interesting part is that we also have Android. that we also have Android. that we also have Android. So Android is So Android is So Android is coming soon as well. coming soon as well. coming soon as well. I really hope I really hope I really hope that in the future you that in the future you that in the future you will have a couple of Android will have a couple of Android phones in the cloud, phones in the cloud, phones in the cloud, a couple of Windows machines, and a couple of Windows machines, and a couple of Windows machines, and the agent can the agent can the agent can coordinate coordinate coordinate all the devices to all the devices to all the devices to do do do various various various tasks. I want to tasks. I want to tasks. I want to show you some show you some show you some interesting demos interesting demos interesting demos for Android. So these are all for Android. So these are all for Android. So these are all our internal, uh, our internal, uh, our internal, uh, evaluations. This is, evaluations. This is, evaluations. This is, for example, the Angry for example, the Angry for example, the Angry Birds game. And the interesting part Birds game. And the interesting part is the actual is the actual is the actual dragging. Let me dragging. Let me dragging. Let me see. Yeah. Yeah.

see. Yeah. Yeah. see. Yeah. Yeah. You actually You actually You actually dragged, you know, dragged, you know, dragged, you know, it was a it was a it was a drag and drop action. He drag and drop action. He drag and drop action. He dragged the screen and dragged the screen and dragged the screen and did a task there. did a task there. did a task there. And another And another And another demo that demo that demo that I think is pretty I think is pretty I think is pretty cool is the cool is the cool is the authenticator. So authenticator. So authenticator. So he goes into the he goes into the he goes into the authenticator and, uh, authenticator and, uh, this is the same this is the same interface inside. interface inside. interface inside. Basically, we replaced Basically, we replaced Basically, we replaced our our our Windows virtual machine with Android. Windows virtual machine with Android. Let me do this. Let me do this. faster. Yes, he faster. Yes, he faster. Yes, he goes through this goes through this goes through this MFA MFA authentication process. authentication process. website and fills out website and fills out forms there. And then he forms there. And then he forms there. And then he gets a token. Yes, and gets a token. Yes, and gets a token. Yes, and then he succeeds. So that's then he succeeds. So that's then he succeeds. So that's one of the problems that one of the problems that one of the problems that we get asked all the time, we get asked all the time, we get asked all the time, "Okay, "Okay, "Okay, can we can we can we use use use two-factor two-factor two-factor authentication?" It's authentication?" It's authentication?" It's still possible, and I still possible, and I still possible, and I can see that there are some can see that there are some can see that there are some points here, but we points here, but we points here, but we have to have to have to use it with, use it with, use it with, uh, some uh, some uh, some caution. And this is an caution. And this is an caution. And this is an example, I think, of example, I think, of example, I think, of social media, uh, social media, uh, social media, uh, on the phone. You on the phone. You on the phone. You can do can do can do all kinds of all kinds of all kinds of things there. Yeah, it's things there. Yeah, it's things there. Yeah, it's pretty exciting. I pretty exciting. I pretty exciting. I feel like Android is an feel like Android is an feel like Android is an interesting area that the interesting area that the interesting area that the current industry has current industry has current industry has n't really explored yet. There n't really explored yet. There n't really explored yet. There should be a lot of should be a lot of should be a lot of applications that we applications that we applications that we can implement can implement can implement using using Android virtual machines. So, basically, it's all Android virtual machines. So, basically, it's all coming together now coming together now coming together now . I started with the . I started with the . I started with the autonomous autonomous autonomous computer and I broke it down computer and I broke it down computer and I broke it down into autonomy into autonomy into autonomy and computation. And

and computation. And and computation. And then autonomy then autonomy then autonomy split into two split into two split into two parts: neural and parts: neural and parts: neural and symbolic. symbolic. symbolic. Computing became Computing became Computing became local machines local machines local machines and cloud machines. and cloud machines. and cloud machines. So Cy actually So Cy actually So Cy actually serves for both serves for both serves for both purposes. You can purposes. You can purposes. You can integrate your own integrate your own integrate your own machines to machines to machines to become part of the fleet become part of the fleet become part of the fleet , and you , and you , and you can also buy our can also buy our can also buy our cloud computers. And cloud computers. And cloud computers. And all of that became Cy, which all of that became Cy, which all of that became Cy, which was released, uh, in was released, uh, in was released, uh, in March of this year. I'm March of this year. I'm March of this year. I'm really excited about this really excited about this really excited about this new product and I new product and I new product and I hope it hope it hope it brings a lot of new brings a lot of new brings a lot of new possibilities. possibilities. Because this is something that I couldn't have imagined two years ago. imagined two years ago. That we're going to get to That we're going to get to That we're going to get to this technology. We do this technology. We do this technology. We do n't know much n't know much n't know much about what kind of highly about what kind of highly about what kind of highly specialized specialized specialized solutions we can solutions we can solutions we can create. But we create. But we create. But we realized, for example, realized, for example, realized, for example, a startup from YC a startup from YC a startup from YC saw me at an saw me at an saw me at an event and said that event and said that event and said that they were already they were already they were already using Agena S using Agena S using Agena S for their own for their own for their own applications. I applications. I applications. I see a huge see a huge see a huge opportunity that opportunity that opportunity that over the next over the next over the next year, year, year, based on past based on past based on past assessments, the technology could become assessments, the technology could become assessments, the technology could become very interesting, and we'll very interesting, and we'll very interesting, and we'll see what see what see what happens. And I happens. And I happens. And I can imagine that there can imagine that there will be will be a lot of new a lot of new a lot of new applications in this area over the year. You So, applications in this area over the year. You So, applications in this area over the year. You So, to summarize, we have to summarize, we have to summarize, we have four proposals four proposals four proposals for our for our for our developers. Site is developers. Site is developers. Site is our work with the basic our work with the basic our work with the basic interface. You interface. You interface. You can explore.

can explore. can explore. You can You can You can use use use different algorithms in different algorithms in different algorithms in developer mode. developer mode. developer mode. Try it. Let us Try it. Let us Try it. Let us know how it works. And know how it works. And know how it works. And we are actively we are actively we are actively developing a new developing a new developing a new technology there. Agena S Simuland, technology there. Agena S Simuland, technology there. Agena S Simuland, which is open and which is open and which is open and available to everyone. available to everyone. available to everyone. It is almost It is almost It is almost free to free to free to use. And Site Box use. And Site Box is an API for is an API for is an API for virtual virtual virtual desktop. We desktop. We desktop. We just released it just released it just released it today. So today. So today. So expect some expect some expect some bugs. Be sure to bugs. Be sure to bugs. Be sure to let us know. And I let us know. And I let us know. And I will give you an will give you an will give you an invitation code for invitation code for invitation code for today: AIE WF 2026. today: AIE WF 2026. today: AIE WF 2026. Please try it Please try it . Let me know your . Let me know your . Let me know your feedback. We have a feedback. We have a feedback. We have a credit program, we are credit program, we are credit program, we are happy to collaborate in happy to collaborate in happy to collaborate in different formats and different formats and different formats and see how to see how to see how to make your make your make your development better in the development better in the development better in the future. Okay. future. Okay. future. Okay. Thank you very much. I want to Thank you very much. I want to Thank you very much. I want to leave some time leave some time leave some time for questions. So feel for questions. So feel for questions. So feel free to ask free to ask free to ask me questions. And me questions. And me questions. And thank you for your thank you for your thank you for your patience. I will patience. I will patience. I will definitely make sure that definitely make sure that definitely make sure that the photo is the photo is the photo is published. So published. So published. So let me know if you let me know if you let me know if you want to be want to be want to be mentioned there. Yes, mentioned there. Yes, mentioned there. Yes, feel free to ask feel free to ask , I will stay here for a , I will stay here for a , I will stay here for a few more minutes. But I few more minutes. But I few more minutes. But I understand, what's the understand, what's the understand, what's the other other other event going on there. Yeah, question? Are event going on there. Yeah, question? Are event going on there. Yeah, question? Are you you you using using using spatial memory in any way?

spatial memory in any way? spatial memory in any way? Spatial memory? Spatial memory? Spatial memory? Yeah, for the UI Yeah, for the UI Yeah, for the UI and and and how it how it how it looks visually and looks visually and looks visually and remembering remembering remembering that pattern. that pattern. that pattern. Right now, Right now, Right now, most of the memory most of the memory most of the memory is just text. is just text. is just text. You know, yeah. And as I You know, yeah. And as I You know, yeah. And as I mentioned with Agent mentioned with Agent mentioned with Agent Sku, we have a separate Sku, we have a separate Sku, we have a separate model. It's not just a model. It's not just a model. It's not just a wrapper for a big wrapper for a big wrapper for a big visual language visual language visual language model. We have other model. We have other model. We have other open open open source models with 7 billion source models with 7 billion source models with 7 billion parameters. Sometimes we parameters. Sometimes we parameters. Sometimes we also retrain that also retrain that also retrain that model. And this model is model. And this model is model. And this model is specifically specifically specifically responsible for responsible for responsible for screen recognition screen recognition . And what's interesting is that we . And what's interesting is that we . And what's interesting is that we know how quickly and know how quickly and know how quickly and well the well the well the fundamental fundamental fundamental models are progressing. But they models are progressing. But they models are progressing. But they still don't still don't still don't understand the screen very well. So understand the screen very well. So understand the screen very well. So that's an interesting, yeah, that's an interesting, yeah, that's an interesting, yeah, interesting finding. interesting finding. I guess it's because there's I guess it's because there's not a lot of screenshot data on the internet not a lot of screenshot data on the internet . So they . So they don't have access to don't have access to don't have access to that data to that data to that data to train their train their train their models. But I can models. But I can models. But I can imagine that the models imagine that the models imagine that the models will get will get will get better and better. Although there's always a better and better. Although there's always a gap. If you have a gap. If you have a model that's built model that's built model that's built specifically for specifically for specifically for your application, you your application, you your application, you can can can tweak it. It's tweak it. It's tweak it. It's always going to be better always going to be better always going to be better than a generic model than a generic model than a generic model from a big from a big from a big lab. So, any lab. So, any lab. So, any questions?

questions? questions? I have a couple of I have a couple of I have a couple of questions. questions. questions. So. So. So. First, what's the tradeoff First, what's the tradeoff First, what's the tradeoff when you when you when you use, use, use, for example, big for example, big for example, big language models for language models for language models for code generation? Is code generation? Is code generation? Is your approach more your approach more your approach more like continuous like continuous like continuous code generation, or have you code generation, or have you code generation, or have you used that in the used that in the used that in the process? Just the process? Just the process? Just the first one. first one. first one. So, let me So, let me So, let me answer answer answer your first question first. your first question first. your first question first. The demo that I The demo that I The demo that I showed is based on a showed is based on a showed is based on a language model, right? language model, right? language model, right? That is, code generation. That is, code generation. That is, code generation. And this approach I And this approach I And this approach I call "stateless call "stateless " programming. " programming. " programming. For the last 20 For the last 20 For the last 20 years, we've all years, we've all years, we've all been doing been doing been doing stateless stateless stateless programming. And the programming. And the programming. And the neuro-symbolic neuro-symbolic neuro-symbolic approach I call approach I call stateful programming. That means stateful programming. That means every time you every time you every time you execute a line of code, execute a line of code, execute a line of code, you look at the you look at the you look at the environment. You look at the environment. You look at the environment. You look at the state. And that state state. And that state state. And that state actually changes. actually changes. actually changes. So I think this is the So I think this is the So I think this is the new paradigm new paradigm new paradigm of programming in of programming in of programming in the future. So the future. So the future. So programming in the programming in the programming in the future will be future will be future will be more more more state-oriented. We also state-oriented. We also state-oriented. We also tested language tested language tested language models. But the problem is models. But the problem is models. But the problem is that this task that this task that this task is purely is purely is purely multimodal. Is multimodal. Is multimodal. Is n't it?

n't it? n't it? Accessibility features can Accessibility features can Accessibility features can be useful, but there be useful, but there is no universal solution. Some say that Some say that computer vision computer vision computer vision can be implemented can be implemented can be implemented through screenshots alone. through screenshots alone. through screenshots alone. Others say that Others say that Others say that accessibility is quite accessibility is quite accessibility is quite enough. Still others enough. Still others enough. Still others believe that DOM is believe that DOM is believe that DOM is all that is needed. But all that is needed. But all that is needed. But ultimately, after all ultimately, after all ultimately, after all the evaluations, it becomes the evaluations, it becomes the evaluations, it becomes clear that there clear that there is no universal solution. There is no universal solution. There will always be edge will always be edge will always be edge cases for one cases for one cases for one method that are not the method that are not the method that are not the same for another. same for another. same for another. So, in the end, we So, in the end, we So, in the end, we choose an approach choose an approach choose an approach similar to Waymo. It's like similar to Waymo. It's like similar to Waymo. It's like using using using lidar, radar, and a camera lidar, radar, and a camera lidar, radar, and a camera at the same time, right? Some at the same time, right? Some at the same time, right? Some companies choose the companies choose the companies choose the Tesla approach, where there is Tesla approach, where there is Tesla approach, where there is only a camera. But only a camera. But only a camera. But if you look at if you look at if you look at modern autopilots, modern autopilots, modern autopilots, we see a lot of we see a lot of we see a lot of Waymo cars on the road Waymo cars on the road Waymo cars on the road that are more that are more that are more reliable. It's like having reliable. It's like having reliable. It's like having multiple systems that multiple systems that multiple systems that are backing each are backing each are backing each other up to avoid other up to avoid other up to avoid mistakes. Especially mistakes. Especially mistakes. Especially fatal mistakes. You fatal mistakes. You fatal mistakes. You don't want an agent to don't want an agent to don't want an agent to go to your bank and go to your bank and go to your bank and add an extra zero add an extra zero add an extra zero when you send when you send when you send a check. other people. a check. other people. a check. other people. Second, yes.

Second, yes. Second, yes. Second question, you Second question, you Second question, you talked about talked about talked about accessibility goals. Yeah, accessibility goals. Yeah, accessibility goals. Yeah, the advantage of our the advantage of our the advantage of our accessibility tree is accessibility tree is accessibility tree is that this that this that this information information information is used to is used to is used to render the screen. render the screen. So, essentially, it So, essentially, it contains almost all contains almost all contains almost all the information that the information that the information that is needed to is needed to is needed to render the screen. And render the screen. And render the screen. And if you can if you can if you can use it, it's use it, it's use it, it's definitely more robust. definitely more robust. But there are some But there are some edge cases, edge cases, edge cases, like the developer like the developer like the developer of the application can of the application can of the application can intentionally specify that intentionally specify that intentionally specify that this part not this part not this part not be displayed in be displayed in be displayed in the tree. So you the tree. So you the tree. So you can still can still can still capture it in capture it in capture it in the screenshot, but not the screenshot, but not the screenshot, but not through the through the through the accessibility tree. But you accessibility tree. But you accessibility tree. But you can imagine some can imagine some can imagine some applications, like applications, like applications, like LinkedIn has a lot of LinkedIn has a lot of bot detection. They probably, I bot detection. They probably, I used to joke with used to joke with used to joke with friends that LinkedIn is the friends that LinkedIn is the friends that LinkedIn is the Turing test for AGI.

Turing test for AGI. Because yeah, if LinkedIn Because yeah, if LinkedIn doesn't recognize your doesn't recognize your doesn't recognize your agent as an agent, then agent as an agent, then agent as an agent, then you've almost achieved AGI, you've almost achieved AGI, you've almost achieved AGI, because you've passed the because you've passed the because you've passed the Turing test in an Turing test in an Turing test in an adversarial environment. And they have adversarial environment. And they have adversarial environment. And they have a lot of methods for a lot of methods for a lot of methods for detection. And so detection. And so detection. And so accessibility could accessibility could accessibility could be one of them. And be one of them. And be one of them. And there, yeah. So we there, yeah. So we there, yeah. So we still need a still need a still need a combination of different combination of different combination of different modalities. Yeah, there modalities. Yeah, there modalities. Yeah, there 's one person there. I 's one person there. I 's one person there. I let this person let this person let this person ask the question ask the question ask the question first. Yes, thank you. first. Yes, thank you. first. Yes, thank you. First of all, First of all, First of all, thank you very much. Um. We're from thank you very much. Um. We're from thank you very much. Um. We're from Germany. Germany. Germany. Yeah. Yeah. We have a use case in Germany for hospitals. Where you for hospitals. Where you take the information and take the information and take the information and then... do... Yeah. then... do... Yeah. Mm. Right. Right. So Mm. Right. Right. So the point is, the point is, the point is, we need very we need very we need very accurate information. We accurate information. We accurate information. We need everything to need everything to need everything to be 100%. Right? So be 100%. Right? So be 100%. Right? So even when you input

even when you input even when you input data, the result has to data, the result has to data, the result has to be 100%. be 100%. be 100%. That's the biggest That's the biggest That's the biggest problem with today's problem with today's problem with today's basic models, basic models, basic models, but we have a solution. but we have a solution. It's basically, as you It's basically, as you can see, everything is a cycle. can see, everything is a cycle. can see, everything is a cycle. Right? All the Right? All the Right? All the fundamental fundamental fundamental principles, everything is a cycle. principles, everything is a cycle. principles, everything is a cycle. When you do something, When you do something, When you do something, you always you always you always check and check and check and criticize yourself. It's the criticize yourself. It's the criticize yourself. It's the same with people. When same with people. When same with people. When I propose some ideas to I propose some ideas to I propose some ideas to my friends, they my friends, they my friends, they criticize them. So criticize them. So criticize them. So I have this cycle, and I I have this cycle, and I I have this cycle, and I can reason with can reason with can reason with less bias less bias . So the . So the . So the accuracy problem can also be accuracy problem can also be accuracy problem can also be solved with solved with solved with this this this approach. We did approach. We did approach. We did some experiments some experiments , and I can tell you , and I can tell you , and I can tell you one solution: one solution: one solution: when you fill out when you fill out when you fill out the form, you take it, you the form, you take it, you the form, you take it, you turn it over and turn it over and turn it over and you ask the model, "Does this you ask the model, "Does this you ask the model, "Does this make sense?" "Right? make sense?" "Right? make sense?" "Right? So, ask the So, ask the So, ask the model to check. model to check. model to check. Most of the time, the model Most of the time, the model Most of the time, the model will recognize if there's will recognize if there's will recognize if there's something wrong. something wrong. something wrong. Because checking is Because checking is Because checking is much easier much easier much easier than generating or than generating or than generating or general actions. And if general actions. And if general actions. And if something is wrong, you something is wrong, you return the results and return the results and ask the agent ask the agent ask the agent to do the same to do the same to do the same task again. That task again. That task again. That improves performance a lot improves performance a lot improves performance a lot . And . And . And another thing to another thing to another thing to think about is the think about is the think about is the availability tree.

availability tree. So that we can write So that we can write code and that it can code and that it can code and that it can be be be reproduced deterministically, that can reproduced deterministically, that can reproduced deterministically, that can only be done through an only be done through an only be done through an availability tree. You availability tree. You can't do that with a screenshot. can't do that with a screenshot. Right? Because a tree Right? Because a tree is text. Text is text. Text is text. Text allows you to achieve 100% allows you to achieve 100% match. So that 100% match. So that 100% textual match is also textual match is also textual match is also important for important for reliability. So reliability. So the conclusion is that you the conclusion is that you the conclusion is that you have to have a holistic have to have a holistic have to have a holistic system. It's not going to be system. It's not going to be system. It's not going to be one model that you one model that you one model that you just put in and just put in and just put in and hope that everything hope that everything hope that everything works 100%. works 100%. But for your But for your use case, there use case, there use case, there is definitely a solution. Yes. is definitely a solution. Yes. is definitely a solution. Yes. Sorry. Your turn. Sorry. Your turn. Sorry. Your turn. Yes. Yes. Yes. Yes. Thank you very much Yes. Thank you very much Yes. Thank you very much for opening up the code for for opening up the code for for opening up the code for that solution. that solution. that solution. Yes. We appreciate that. Yes. We appreciate that. Yes. We appreciate that. I think when I I think when I I think when I talk to people talk to people talk to people about about about computer vision agents, computer vision agents, computer vision agents, they're always concerned about they're always concerned about they're always concerned about restrictions and security. restrictions and security. restrictions and security. Yeah. Yeah. Yeah. How do you prevent it from How do you prevent it from accessing accessing things that you don't things that you don't things that you don't want it to be able to want it to be able to want it to be able to access? access? access? For example, if I don't For example, if I don't For example, if I don't want it to want it to want it to look in the mail look in the mail look in the mail app. I want it to app. I want it to only use Chrome and only use Chrome and only go to the Google site.

only go to the Google site. only go to the Google site. Yeah. Yeah. Yeah. How do you implement How do you implement How do you implement those restrictions in your those restrictions in your those restrictions in your product? product? product? Yeah. So on the Yeah. So on the Yeah. So on the product side, we have a product side, we have a product side, we have a restriction mechanism. restriction mechanism. restriction mechanism. When there's an action that's When there's an action that's When there's an action that's recognized as recognized as recognized as irreversible, the irreversible, the irreversible, the restriction system says " restriction system says " restriction system says " stop." So it's a stop." So it's a stop." So it's a separate system. Another separate system. Another separate system. Another system that's watching system that's watching system that's watching this agent. If the this agent. If the this agent. If the agent wants to do a agent wants to do a agent wants to do a certain action, the system certain action, the system certain action, the system interrupts it and interrupts it and interrupts it and asks for approval. asks for approval. asks for approval. But I think your But I think your But I think your problem is even more problem is even more problem is even more complex. It's complex. It's complex. It's more at the low more at the low more at the low level, at the level, at the level, at the operating system level. operating system level. operating system level. For example, the agent says For example, the agent says For example, the agent says to open certain to open certain to open certain applications, and you want applications, and you want applications, and you want that application to be that application to be that application to be blocked, right? blocked, right? So that requires an So that requires an additional layer at the additional layer at the additional layer at the OS level. So we OS level. So we OS level. So we think a cloud- think a cloud- think a cloud- managed Windows machine managed Windows machine is a better direction, is a better direction, is a better direction, because we have a because we have a because we have a virtual machine. virtual machine. virtual machine. So if you need to, So if you need to, So if you need to, we can also we can also we can also create an image, a create an image, a create an image, a virtual machine image virtual machine image virtual machine image for you. In this for you. In this for you. In this virtual virtual virtual machine image, you have a lot of machine image, you have a lot of machine image, you have a lot of possibilities to change the possibilities to change the possibilities to change the basic structure of the basic structure of the basic structure of the operating system. operating system. You can actually You can actually block some actions, block some actions, block some actions, use use use whitelists. So I think that's the whitelists. So I think that's the whitelists. So I think that's the ultimate ultimate ultimate direction. We're still in the direction. We're still in the direction. We're still in the early stages of working early stages of working early stages of working with virtual with virtual with virtual machines. We're machines. We're machines. We're trying trying trying to explore that to explore that to explore that direction, and if you're direction, and if you're direction, and if you're interested, we can interested, we can interested, we can think about think about think about partnering in partnering in partnering in the future on the future on the future on better better better protection mechanisms. Yes, that's protection mechanisms. Yes, that's protection mechanisms. Yes, that's definitely true, definitely true, definitely true, security is one of the security is one of the security is one of the important aspects.

important aspects. important aspects. Yes, thank you. Yes, thank you. Yes, thank you. Very, very quickly. Very, very quickly. Very, very quickly. Yes. Yes. Yes. Just detecting Just detecting Just detecting irreversible action. How do irreversible action. How do irreversible action. How do you do it for you do it for you do it for something that's so something that's so something that's so reliable? reliable? Yes. It comes down to the Yes. It comes down to the same principle. same principle. same principle. The first thing I mentioned is that The first thing I mentioned is that The first thing I mentioned is that everything is everything is everything is neurosymbolic. neurosymbolic. neurosymbolic. If you want If you want If you want everything to be reliable, everything to be reliable, everything to be reliable, everything has to be everything has to be everything has to be neurosymbolic. You neurosymbolic. You neurosymbolic. You always always always go to the code first, go to the code first, go to the code first, because the code can because the code can because the code can guarantee 100% of the guarantee 100% of the result, right? result, right? result, right? When the code When the code When the code is unclear, you is unclear, you is unclear, you go to go to go to flexible agent, and flexible agent, and flexible agent, and then you go back then you go back then you go back to the code. That's why code to the code. That's why code to the code. That's why code remains remains remains extremely important extremely important . That's the same principle . That's the same principle . That's the same principle for designing a for designing a for designing a security system. So the security system. So the security system. So the security system is security system is security system is also based on the also based on the also based on the principle of " principle of " principle of " code first." There are a lot of code first." There are a lot of code first." There are a lot of rules, and if rules, and if rules, and if a rule doesn't a rule doesn't a rule doesn't work, it work, it work, it goes to the agent, goes to the agent, goes to the agent, then to the human for then to the human for then to the human for review, review, and then it goes back. That's a design principle that we try to follow throughout the system, even in every component of the system. Yes, thank you. Any questions? Yes, one of the things you talked about is to try the code first, then the vision.

Yes. Yes. But as you But as you But as you loop loop loop through the same through the same through the same workflow, the workflow, the workflow, the same interface, does same interface, does same interface, does it iterate to it iterate to it iterate to get more and more get more and more get more and more code, to code, to code, to depend less on...? depend less on...? depend less on...? Right. Right. Right. Right. Right. Right. Right. So that's the Right. So that's the Right. So that's the trajectory. We want to get the trajectory. We want to get the token consumption to eventually token consumption to eventually go to zero. Yes, it go to zero. Yes, it go to zero. Yes, it eventually goes to zero. And eventually goes to zero. And eventually goes to zero. And you see, sometimes the code you see, sometimes the code you see, sometimes the code actually starts to actually starts to actually starts to have more branches. have more branches. As you get As you get different scenarios, you different scenarios, you different scenarios, you see more, it's like see more, it's like see more, it's like training a human, training a human, training a human, right? When you have right? When you have right? When you have more data samples, more data samples, more data samples, you're less biased in your you're less biased in your you're less biased in your decision-making. Yeah decision-making. Yeah , same with this , same with this , same with this approach. Oh, that's approach. Oh, that's approach. Oh, that's the question. Yeah. Right. the question. Yeah. Right.

Mm. Yeah, for this Mm. Yeah, for this version, you see there's a version, you see there's a version, you see there's a prompt and an agent prompt and an agent prompt and an agent that does the that does the that does the task. Well, maybe task. Well, maybe task. Well, maybe in a couple of months, we're in a couple of months, we're in a couple of months, we're thinking about that too, thinking about that too, thinking about that too, it's pretty common in the it's pretty common in the it's pretty common in the research research research community. So community. So community. So basically, if you basically, if you basically, if you can replicate what can replicate what can replicate what people are talking about people are talking about people are talking about in terms of models of the world, in terms of models of the world, in terms of models of the world, right? If you right? If you right? If you can replicate these can replicate these can replicate these programs with programs with programs with better control so that the better control so that the better control so that the agent doesn't do agent doesn't do agent doesn't do bad things, if bad things, if bad things, if you have safeguards. And you have safeguards. And you have safeguards. And then you can then you can then you can do do do research. We research. We research. We call this call this environmental research. So environmental research. So basically you basically you basically you let agents let agents let agents freely explore the freely explore the freely explore the environment, environment, environment, like 100 times. And like 100 times. And like 100 times. And then the agent becomes more then the agent becomes more then the agent becomes more familiar with the familiar with the familiar with the environment. And then environment. And then environment. And then your query is less your query is less your query is less important, you don't important, you don't important, you don't need very precise need very precise need very precise prompts, whether prompts, whether prompts, whether to press this button to press this button to press this button or that. You can also or that. You can also or that. You can also succeed in succeed in succeed in finding the first finding the first finding the first successful trajectory, successful trajectory, successful trajectory, that's basically it. that's basically it. that's basically it. goal. Once you goal. Once you goal. Once you find it, it can find it, it can find it, it can be successful every time. be successful every time. be successful every time. If you already know If you already know If you already know the process, you the process, you the process, you can already tell that it's an can already tell that it's an can already tell that it's an accurate process.

accurate process. accurate process. Yes. Yes, yes, yes. Yes, Yes. Yes, yes, yes. Yes, Yes. Yes, yes, yes. Yes, a lot of companies a lot of companies a lot of companies call it an SOP. So call it an SOP. So call it an SOP. So yes, they have internal yes, they have internal yes, they have internal SOPs. We actually SOPs. We actually SOPs. We actually did an interesting did an interesting did an interesting experiment. We experiment. We experiment. We submitted one SOP submitted one SOP submitted one SOP directly to the directly to the directly to the coding agent. coding agent. coding agent. Right? And all at Right? And all at Right? And all at once, all at once. And it once, all at once. And it once, all at once. And it was 100% successful was 100% successful was 100% successful all at once. So it's all at once. So it's all at once. So it's possible, it's entirely possible, it's entirely possible, it's entirely possible, but I'll possible, but I'll possible, but I'll leave it at that, that's leave it at that, that's still in the still in the demonstration stage. We demonstration stage. We demonstration stage. We want a technology want a technology want a technology that's that's that's generic enough that every generic enough that every generic enough that every customer comes in and customer comes in and customer comes in and we don't have to we don't have to we don't have to worry about something worry about something worry about something breaking. So there's breaking. So there's breaking. So there's still a lot of still a lot of still a lot of edge cases edge cases edge cases that we need to that we need to that we need to address, you know. I address, you know. I address, you know. I remember a remember a remember a gentleman raising his gentleman raising his gentleman raising his hand earlier. hand earlier. Yes, well. Yes, well. Oh, yes. Oh. Okay, Oh, yes. Oh. Okay, Oh, yes. Oh. Okay, thank you very much. Oh, there's one more. thank you very much. Oh, there's one more. What about that second What about that second question, thank you for question, thank you for question, thank you for your presentation, it was your presentation, it was your presentation, it was related to related to related to our other question about our other question about our other question about using using using legacy systems.

legacy systems. Uh. Uh. And I actually did And I actually did Uh. Uh. this is for use this is for use this is for use and I would like to ask and I would like to ask and I would like to ask your opinion on your opinion on your opinion on this. this. this. Uh. Uh. Uh. So I asked if it's So I asked if it's So I asked if it's allowed to allowed to allowed to use this use this use this to actually to actually to actually store the errors store the errors store the errors that we have, you can that we have, you can that we have, you can have a map of the have a map of the software itself. So software itself. So it's basically it's basically pixel based, so you pixel based, so you can see how uh can see how uh we actually we actually click on the pixels of click on the pixels of click on the pixels of the memory and the memory and the memory and map them to uh map them to uh things, so things, so things, so Uh. Uh. Uh. we actually eventually we actually eventually we actually eventually have an MCP guide at have an MCP guide at have an MCP guide at this company for this company for this company for use. So uh use. So uh use. So uh it means that the it means that the computer user doesn't even computer user doesn't even have to see have to see have to see what screen they're what screen they're what screen they're on. They just on. They just on. They just have to hold that have to hold that have to hold that position and store position and store position and store it to perform a it to perform a it to perform a certain action. Uh, I don't certain action. Uh, I don't certain action. Uh, I don't know if I know if I know if I explained it clearly, but is this the explained it clearly, but is this the explained it clearly, but is this the same thing that you're working on now same thing that you're working on now same thing that you're working on now ? Is that what ? Is that what ? Is that what it is? I understand that it is? I understand that it is? I understand that you want you want you want to generalize how the to generalize how the to generalize how the agent works, but is that still what agent works, but is that still what agent works, but is that still what you're you're you're going to do uh going going to do uh going going to do uh going forward?

forward? Yeah, I, I, my impression Yeah, I, I, my impression is that it's a map and is that it's a map and is that it's a map and then you're trying to then you're trying to then you're trying to let the agent let the agent let the agent walk this map. walk this map. walk this map. So he can just So he can just So he can just walk blindly. He doesn't walk blindly. He doesn't walk blindly. He doesn't need to, if you need to, if you need to, if you ask him, yeah, ask him, yeah, ask him, yeah, just add this just add this just add this user. user. user. Mm. Mm. Mm. He doesn't need to He doesn't need to He doesn't need to read uh take a read uh take a read uh take a screenshot, go screenshot, go screenshot, go there, figure it out. there, figure it out. there, figure it out. Just have a map of Just have a map of how he should move, how he should move, how he should move, like uh where he should like uh where he should like uh where he should click to click to click to get there, get there, get there, you know. So it doesn't you know. So it doesn't you know. So it doesn't waste tokens. It's waste tokens. It's waste tokens. It's faster and all that. faster and all that. faster and all that. Yeah, yeah, yeah. So that's the Yeah, yeah, yeah. So that's the Yeah, yeah, yeah. So that's the whole idea of ​​the whole idea of ​​the whole idea of ​​the coding layer. So the coding layer coding layer. So the coding layer basically encodes the whole basically encodes the whole trajectory into code, right? trajectory into code, right? trajectory into code, right? And I think, I think the And I think, I think the And I think, I think the problem that we're problem that we're problem that we're trying to solve trying to solve is uh let's say there's is uh let's say there's is uh let's say there's legacy legacy legacy software, then software, then software, then it's not going to it's not going to it's not going to change. Then this change. Then this change. Then this approach works. But approach works. But approach works. But if we're talking about if we're talking about if we're talking about sites, for example, right? sites, for example, right? sites, for example, right? For social media For social media For social media or different websites, or different websites, or different websites, they actually they actually they actually change. And when change. And when change. And when they change, your they change, your they change, your path there won't path there won't path there won't work. Is there a work. Is there a work. Is there a way for us way for us way for us to actually adapt the to actually adapt the to actually adapt the uh agent during uh agent during uh agent during this scenario?

this scenario? this scenario? Yeah, I I mean, uh, Yeah, I I mean, uh, my answer to that my answer to that my answer to that problem problem problem Mhm. Mhm. Mhm. I did, it was, I did, it was, I did, it was, if uh, if your if uh, if your if uh, if your main main main computer agent uh computer agent uh Mhm. continuous continuous improvement improvement Yeah. Yeah. Yeah. Yeah, I Yeah. Yeah. Yeah. Yeah, I get your point. get your point. get your point. So basically, his So basically, his So basically, his solution is, okay, solution is, okay, solution is, okay, every time there's every time there's every time there's a change, a change, a change, I have a swarm of agents I have a swarm of agents I have a swarm of agents doing doing doing research. And so research. And so research. And so I always know I always know I always know the best uh, yeah, the best uh, yeah, the best uh, yeah, best option best option best option out there. Yeah. Yeah. Yeah, that's out there. Yeah. Yeah. Yeah, that's out there. Yeah. Yeah. Yeah, that's exactly my answer exactly my answer exactly my answer to that gentleman. to that gentleman. to that gentleman. So the basic idea So the basic idea So the basic idea is, is, is, if you have a if you have a if you have a legacy system and legacy system and legacy system and zero zero zero research cost? research cost? research cost? Right? Because, like, Right? Because, like, Right? Because, like, if it's a bank, if it's a bank, if it's a bank, research costs research costs research costs money. I can't money. I can't money. I can't research every research every research every possible action. But if possible action. But if possible action. But if you don't have any you don't have any research cost, you research cost, you just do just do just do it. I put agents in there it. I put agents in there it. I put agents in there , let them , let them , let them try different things, try different things, try different things, and map the whole and map the whole and map the whole trajectory. It's trajectory. It's trajectory. It's actually a simpler actually a simpler actually a simpler problem to solve.

problem to solve. problem to solve. So basically, I So basically, I So basically, I think 80% of applications think 80% of applications think 80% of applications now run on now run on now run on legacy systems. legacy systems. And if we have... it's And if we have... it's actually easier actually easier actually easier than a generic than a generic than a generic system. So it system. So it system. So it can already bring a lot of can already bring a lot of can already bring a lot of economic value economic value economic value to the current industry. to the current industry. to the current industry. Yes. Yes. Thank you for your Yes. Yes. Thank you for your Yes. Yes. Thank you for your feedback. Yes. Also feedback. Yes. Also , Yes, the question is: , Yes, the question is: , Yes, the question is: let's say we have let's say we have let's say we have enough data, enough data, enough data, can we train can we train can we train agents to be very agents to be very agents to be very successful? There's successful? There's successful? There's been a lot of been a lot of been a lot of discussion around this. My discussion around this. My discussion around this. My conclusion is that you conclusion is that you conclusion is that you need need need exponentially exponentially exponentially more money than more money than more money than OpenAI and Strawbic to OpenAI and Strawbic to OpenAI and Strawbic to do this. Because do this. Because do this. Because visual data is visual data is visual data is two-dimensional. Text is two-dimensional. Text is two-dimensional. Text is one-dimensional. So one-dimensional. So one-dimensional. So we're talking about something we're talking about something we're talking about something much larger, much larger, much larger, right? So if you right? So if you right? So if you can give me can give me can give me a few trillion a few trillion a few trillion dollars, we dollars, we dollars, we can certainly can certainly can certainly look at it. Yes. But look at it. Yes. But look at it. Yes. But it's difficult.

it's difficult. it's difficult. It might be possible with a new It might be possible with a new It might be possible with a new paradigm, but with the paradigm, but with the paradigm, but with the current paradigm of current paradigm of current paradigm of deep learning, I deep learning, I deep learning, I don't think it's don't think it's don't think it's achievable anytime soon achievable anytime soon achievable anytime soon . Yes. Last . Yes. Last . Yes. Last question. question. question. So the code is a direct So the code is a direct So the code is a direct message. message. message. Yes. Yes. How does it How does it handle the granularity that you need handle the granularity that you need handle the granularity that you need ? Because I I ? Because I I ? Because I I imagine: if I have a imagine: if I have a imagine: if I have a very long very long very long workflow and the whole process is workflow and the whole process is workflow and the whole process is coded, there are a lot of coded, there are a lot of coded, there are a lot of places where things can places where things can places where things can go wrong. go wrong. go wrong. Yeah. Yeah. So what So what granularity of code granularity of code granularity of code and scripts do you and scripts do you and scripts do you recommend? recommend? Like, two or three Like, two or three steps, and then you steps, and then you steps, and then you call something call something call something useful. useful. Oh, eventually your code Oh, eventually your code will have a lot of will have a lot of will have a lot of if-else logic. You can if-else logic. You can if-else logic. You can imagine that code like that is imagine that code like that is imagine that code like that is definitely not definitely not definitely not suitable for suitable for suitable for human control. human control. But an agent But an agent can still write this can still write this can still write this code with lots of code with lots of code with lots of complex complex complex if-else logic, if-else logic, if-else logic, while loops, and it while loops, and it while loops, and it will work. And yes, will work. And yes, will work. And yes, that's the ultimate goal of this that's the ultimate goal of this that's the ultimate goal of this system. Yeah. But you system. Yeah. But you system. Yeah. But you can also break can also break can also break it down to it down to it down to get simpler get simpler get simpler tasks each time. But then there's the tasks each time. But then there's the boundary problem: well, boundary problem: well, how do you define a smaller how do you define a smaller how do you define a smaller task? So where is that task? So where is that task? So where is that boundary? What are the inputs and boundary? What are the inputs and boundary? What are the inputs and outputs, right? So outputs, right? So outputs, right? So sometimes you have a simple sometimes you have a simple sometimes you have a simple solution that's solution that's solution that's actually even actually even actually even better. Okay. Thank you, better. Okay. Thank you, better. Okay. Thank you, thank you all.

thank you all.