OpenAI's AI Wrote 700 Math Papers. Engineers Still Won't Trust AI With Their Code.

OpenAI OpenAI has just published over 700 has just published over 700 has just published over 700 scientific papers in scientific papers in scientific papers in mathematics. And what mathematics. And what impresses me most is how they impresses me most is how they did it. Almost every did it. Almost every did it. Almost every result was obtained result was obtained result was obtained with a single with a single with a single query to a model query to a model query to a model they haven't even they haven't even they haven't even released. This is the released. This is the released. This is the clearest example of clearest example of clearest example of superintelligence superintelligence superintelligence driven by large driven by large driven by large language models. You've language models. You've language models. You've been hearing about this for been hearing about this for been hearing about this for years, right? You years, right? You years, right? You heard about it on SNL, from heard about it on SNL, from heard about it on SNL, from Anderson Cooper. Anderson Cooper. Anderson Cooper. She will take your She will take your She will take your job. The world is coming to job. The world is coming to job. The world is coming to an end. And I understand that. an end. And I understand that. an end. And I understand that. Looks like proof, Looks like proof, Looks like proof, right? Well, I right? Well, I right? Well, I read this release and read this release and read this release and came to the opposite came to the opposite came to the opposite conclusion. I still conclusion. I still conclusion. I still know engineers who know engineers who know engineers who won't let a won't let a won't let a coding agent into their coding agent into their coding agent into their work environment. work environment. work environment. In fact, most In fact, most In fact, most of them of them of them don't. They don't. They don't. They install a bunch of install a bunch of install a bunch of checks and operate checks and operate checks and operate in a in a in a zero-trust environment. zero-trust environment. zero-trust environment. This difference is worth This difference is worth This difference is worth paying attention to, because is paying attention to, because is paying attention to, because is n't the code n't the code n't the code verifiable verifiable ? Every lab is ? Every lab is ? Every lab is pouring money into models pouring money into models that can write that can write that can write code. There is easy code. There is easy code. There is easy verification here. Is it verification here. Is it verification here. Is it working or not? working or not? working or not? If the model was If the model was If the model was superhuman at superhuman at superhuman at math, you would math, you would math, you would think it would be think it would be think it would be turning turning turning your codebase upside down by now. But your codebase upside down by now. But your codebase upside down by now. But that's not true. This gap is that's not true. This gap is the most useful thing you the most useful thing you the most useful thing you can understand about can understand about can understand about AI right now, because it AI right now, because it AI right now, because it explains what explains what explains what your your your work relies on and why. So work relies on and why. So work relies on and why. So today we're going to today we're going to today we're going to look at why you look at why you look at why you should actually should actually should actually care about care about care about math, even math, even math, even if you didn't study if you didn't study if you didn't study calculus, what we calculus, what we calculus, what we learned from all the learned from all the learned from all the math insights math insights math insights this week, and what it this week, and what it this week, and what it doesn't tell us about doesn't tell us about doesn't tell us about work. Because I think work. Because I think work. Because I think we look at the world we look at the world we look at the world with a narrow with a narrow with a narrow superintelligence, and superintelligence, and superintelligence, and some of the consequences of that some of the consequences of that some of the consequences of that narrow narrow narrow superintelligence are very superintelligence are very superintelligence are very surprising. We surprising. We surprising. We will trace this in the code.

will trace this in the code. will trace this in the code. We will trace this in We will trace this in We will trace this in business processes. And business processes. And business processes. And then let's talk about then let's talk about then let's talk about your skills. Why is it your skills. Why is it your skills. Why is it worth studying in this worth studying in this worth studying in this world? How do we world? How do we world? How do we use use use the abilities that the abilities that the abilities that these systems give us? And how do these systems give us? And how do these systems give us? And how do we respond and we respond and we respond and improve on the improve on the improve on the things that things that things that truly matter to us? So truly matter to us? So truly matter to us? So let's start with what let's start with what let's start with what these works actually contain these works actually contain these works actually contain . It will be a little . It will be a little . It will be a little too clever, but not too much. too clever, but not too much. too clever, but not too much. On a On a On a math exam, someone already math exam, someone already math exam, someone already knows the answer, knows the answer, knows the answer, right? In right? In right? In mathematical mathematical mathematical research, you're research, you're research, you're trying trying trying to establish something that to establish something that to establish something that people haven't really people haven't really people haven't really figured out yet. Therefore, a figured out yet. Therefore, a figured out yet. Therefore, a useful result useful result useful result becomes a foundation on which becomes a foundation on which becomes a foundation on which other researchers can build other researchers can build other researchers can build . Ultimately, . Ultimately, . Ultimately, this could change the this could change the this could change the methods available to methods available to methods available to engineers and the products engineers and the products we all we all we all use every day. use every day. use every day. Let's take something very Let's take something very Let's take something very complex, like complex, like complex, like matrix multiplication. matrix multiplication. matrix multiplication. Maybe you learned this in Maybe you learned this in Maybe you learned this in school, maybe you didn't. school, maybe you didn't. school, maybe you didn't. This is the arithmetic This is the arithmetic This is the arithmetic behind multiplying behind multiplying behind multiplying grids of numbers, and AI grids of numbers, and AI systems perform a systems perform a systems perform a huge amount of it. huge amount of it. huge amount of it. And one of these And one of these And one of these results improves the results improves the results improves the theoretical limit on theoretical limit on theoretical limit on how much math how much math how much math you need as these you need as these you need as these grids grow. grids grow. grids grow. Another difficult Another difficult Another difficult result concerns result concerns result concerns how to find out what's how to find out what's how to find out what's inside a material inside a material inside a material from measurements on the from measurements on the from measurements on the surface, for example, surface, for example, surface, for example, measuring the surface of a measuring the surface of a measuring the surface of a beach ball and beach ball and beach ball and finding out what's finding out what's finding out what's inside. For a inside. For a inside. For a certain class of certain class of certain class of elastic elastic elastic materials, the evidence materials, the evidence materials, the evidence actually confirms actually confirms that you can that you can that you can do this. You can do this. You can do this. You can measure the surface of a measure the surface of a measure the surface of a beach ball and beach ball and beach ball and find out what's find out what's find out what's inside. If you're inside. If you're inside. If you're trying to inspect trying to inspect trying to inspect an object without cutting it open an object without cutting it open an object without cutting it open , this is extremely , this is extremely , this is extremely useful knowledge. This useful knowledge. This useful knowledge. This tells you that tells you that tells you that the information you need is the information you need is the information you need is present in the present in the surface measurements. And now you surface measurements. And now you can work on can work on can work on restoring it in restoring it in restoring it in real-world conditions, where real-world conditions, where real-world conditions, where measurements are measurements are measurements are incomplete and noisy.

incomplete and noisy. incomplete and noisy. And there's a reason to care about And there's a reason to care about And there's a reason to care about it, even if it, even if it, even if math is your math is your math is your least favorite least favorite least favorite subject. These discoveries subject. These discoveries subject. These discoveries will eventually come will eventually come will eventually come to you through cheaper to you through cheaper to you through cheaper computing, better computing, better computing, better ways of testing ways of testing ways of testing things, and things, and things, and improved improved software. So, software. So, a model can a model can a model can improve the improve the improve the tools tools tools available in your available in your available in your industry through such industry through such industry through such mathematical mathematical mathematical discoveries, but that doesn't discoveries, but that doesn't discoveries, but that doesn't mean it's an mean it's an mean it's an expert in doing expert in doing expert in doing business in your business in your business in your industry, right? In industry, right? In industry, right? In other words, other words, other words, discoveries from discoveries from discoveries from mathematical works mathematical works mathematical works today may today may today may spread around the spread around the spread around the world, but that doesn't world, but that doesn't world, but that doesn't mean that mean that mean that magically a model that magically a model that magically a model that can solve these can solve these can solve these mathematical problems mathematical problems mathematical problems will transfer its will transfer its will transfer its judgmental ability judgmental ability judgmental ability to our everyday to our everyday to our everyday work. But I want to work. But I want to work. But I want to make it clear: while make it clear: while make it clear: while this seems far this seems far this seems far from everyday life, from everyday life, from everyday life, it could it could it could come surprisingly come surprisingly come surprisingly quickly precisely because quickly precisely because quickly precisely because we have AI. In other we have AI. In other we have AI. In other words, AI can words, AI can words, AI can make it possible to make it possible to make it possible to translate this theory translate this theory translate this theory into engineering and into engineering and into engineering and technology that we technology that we technology that we can use can use can use every day, much every day, much every day, much faster than 10 years faster than 10 years faster than 10 years ago. And that's part of why ago. And that's part of why ago. And that's part of why I'm excited I'm excited . And I think that gives . And I think that gives . And I think that gives us a chance to actually us a chance to actually us a chance to actually improve our world, improve our world, improve our world, probably in hundreds of probably in hundreds of probably in hundreds of ways at once. And ways at once. And ways at once. And the model that created all this has the model that created all this has the model that created all this has not yet been not yet been not yet been released. I released. I released. I don't have access to it, don't have access to it, don't have access to it, you don't have access you don't have access you don't have access to it. What to it. What to it. What researchers have in their researchers have in their researchers have in their hands is just the hands is just the hands is just the work created by this work created by this work created by this model. And OpenAI model. And OpenAI model. And OpenAI reports that this reports that this reports that this model is so model is so model is so powerful that it

powerful that it only took about 3 only took about 3 hours of thinking to produce any of the results I just described. So, hours of thinking to produce any of the results I just described. So, hours of thinking to produce any of the results I just described. So, the model the model the model solved some solved some solved some problems in 3 hours that problems in 3 hours that problems in 3 hours that mathematicians can't mathematicians can't mathematicians can't solve in a solve in a solve in a lifetime. This is lifetime. This is lifetime. This is superintelligence. There is no superintelligence. There is no superintelligence. There is no other way to other way to other way to describe it. So what describe it. So what describe it. So what can we learn here? can we learn here? can we learn here? Why should we Why should we Why should we care? Let's care? Let's care? Let's put aside the put aside the put aside the math part, math part, math part, let's assume for a let's assume for a let's assume for a second that we don't second that we don't second that we don't care. What can we care. What can we care. What can we learn about AI and learn about AI and learn about AI and ourselves? I ourselves? I ourselves? I believe that narrow believe that narrow believe that narrow superintelligence is the superintelligence is the superintelligence is the correct way to correct way to correct way to understand what we understand what we understand what we see. This is a system see. This is a system see. This is a system capable of surpassing capable of surpassing capable of surpassing human abilities in human abilities in human abilities in very specific very specific very specific types of types of types of intellectual intellectual intellectual work, while other work, while other work, while other abilities abilities abilities develop very develop very develop very unevenly. And that unevenly. And that unevenly. And that makes the mathematical makes the mathematical makes the mathematical aspect particularly aspect particularly aspect particularly interesting, because you interesting, because you interesting, because you can clearly can clearly can clearly articulate what articulate what articulate what success will mean and success will mean and success will mean and then spend your entire then spend your entire then spend your entire life life life achieving it. achieving it. achieving it. The question may The question may The question may remain perfectly remain perfectly remain perfectly clear clear clear throughout, while throughout, while throughout, while the reasoning the reasoning the reasoning required to required to required to achieve the goal is achieve the goal is achieve the goal is extremely complex, extremely complex, extremely complex, intricate, and intricate, and intricate, and requires mastering requires mastering requires mastering several branches of several branches of several branches of mathematics simultaneously mathematics simultaneously . When you train a . When you train a . When you train a system by system by system by rewarding rewarding rewarding correct answers, correct answers, correct answers, being able to being able to being able to recognize the correct recognize the correct recognize the correct answer helps a lot answer helps a lot answer helps a lot . And . And . And mathematics provides such an mathematics provides such an mathematics provides such an opportunity, along with opportunity, along with opportunity, along with tasks that tasks that tasks that become extremely become extremely become extremely difficult. I difficult. I difficult. I suspect that it is suspect that it is suspect that it is extremely well extremely well extremely well formulated for formulated for formulated for the advances in the advances in the advances in super-intellectual super-intellectual super-intellectual reasoning that we are reasoning that we are reasoning that we are witnessing now.

witnessing now. witnessing now. OpenAI claims that OpenAI claims that OpenAI claims that they moved to they moved to they moved to open open open research research research problems because problems because problems because existing mathematical existing mathematical existing mathematical assessments had exhausted assessments had exhausted assessments had exhausted themselves. The tests simply weren't themselves. The tests simply weren't difficult enough anymore. difficult enough anymore. The lab needed The lab needed The lab needed harder problems, harder problems, harder problems, and these were things that and these were things that and these were things that people people people had built their careers around had built their careers around , but never , but never , but never solved. And this solved. And this solved. And this helps explain the helps explain the helps explain the conflict with the conflict with the conflict with the mathematical mathematical mathematical community. I don't community. I don't community. I don't think it's worth think it's worth think it's worth imagining a world where imagining a world where imagining a world where OpenAI is trying OpenAI is trying OpenAI is trying to harm to harm to harm mathematicians. We have mathematicians. We have mathematicians. We have no evidence of no evidence of no evidence of this. The researchers in this. The researchers in this. The researchers in the lab needed the lab needed the lab needed to find out to find out to find out how capable how capable how capable their model had become. They their model had become. They their model had become. They had to take on had to take on had to take on difficult tasks, while for difficult tasks, while for difficult tasks, while for someone else, such a someone else, such a someone else, such a challenge is years of challenge is years of challenge is years of work on the contribution work on the contribution work on the contribution they they they hope to make hope to make . And for the lab, it's . And for the lab, it's . And for the lab, it's just finding something just finding something just finding something complex enough complex enough complex enough to test the to test the to test the model. But first of all model. But first of all model. But first of all , there is an important assumption here that I , there is an important assumption here that I think we think we need to test need to test need to test in relation to our in relation to our in relation to our own work. A own work. A own work. A breakthrough in breakthrough in breakthrough in mathematical mathematical mathematical reasoning reasoning reasoning tells us little about tells us little about tells us little about how those same abilities how those same abilities how those same abilities transfer to other transfer to other transfer to other areas of work, areas of work, areas of work, such as engineering such as engineering such as engineering judgment or business judgment or business intuition. I'll focus intuition. I'll focus intuition. I'll focus on the engineering on the engineering on the engineering aspect because I aspect because I aspect because I think think think programming is programming is programming is exactly where it exactly where it exactly where it gets really interesting, gets really interesting, gets really interesting, and we have and we have and we have n't really talked about it yet. A huge investment n't really talked about it yet. A huge investment has been made in the programming of the laboratory , that's obvious. There is a , that's obvious. There is a huge huge huge commercial demand for a commercial demand for a commercial demand for a machine that can machine that can machine that can program better.

program better. program better. You can run the You can run the You can run the code, write tests, and code, write tests, and code, write tests, and check if check if check if it produces the correct it produces the correct it produces the correct answer. So, if answer. So, if answer. So, if verifiable verifiable verifiable rewards and rewards and rewards and sufficient investment sufficient investment sufficient investment were enough to were enough to were enough to achieve achieve achieve superintelligence, superintelligence, superintelligence, programming is programming is programming is exactly where you would exactly where you would exactly where you would expect to see a expect to see a machine advantage first. But machine advantage first. But we don't see it. AI is we don't see it. AI is we don't see it. AI is already extremely already extremely already extremely adept at the adept at the adept at the programming parts, but programming parts, but programming parts, but all week I've been all week I've been all week I've been talking about the systems talking about the systems talking about the systems engineering required engineering required engineering required to make to make AI-generated software AI-generated software work. We work. We work. We talk about talk about zero-trust environments, zero-trust environments, limiting access or limiting access or limiting access or the ability to change the ability to change the ability to change AI, rather than AI, rather than AI, rather than assuming it assuming it assuming it will behave will behave will behave correctly. Ratings, correctly. Ratings, correctly. Ratings, ratings, ratings. We're ratings, ratings. We're ratings, ratings. We're talking about finding talking about finding talking about finding bugs and maintaining the bugs and maintaining the bugs and maintaining the viability of the code viability of the code viability of the code as the as the as the system grows. And the engineers system grows. And the engineers system grows. And the engineers who install who install who install the fuses do the fuses do the fuses do it because they are it because they are experienced professionals experienced professionals experienced professionals who know what's what who know what's what . It is the human . It is the human . It is the human contribution that creates the contribution that creates the contribution that creates the main value here. And main value here. And main value here. And we can't we can't we can't attribute all this attribute all this attribute all this value to a model just value to a model just value to a model just because it generates a because it generates a because it generates a lot of code. Here is a lot of code. Here is a lot of code. Here is a simple example of the simple example of the simple example of the problem I am problem I am problem I am talking about. Imagine that talking about. Imagine that talking about. Imagine that a company asked a company asked a company asked AI to create a AI to create a AI to create a document search function document search function document search function for employees. So for employees. So for employees. So he creates this he creates this he creates this function. The answers are function. The answers are function. The answers are useful. The tests are useful. The tests are useful. The tests are passing successfully. This passing successfully. This passing successfully. This can be checked. And can be checked. And can be checked. And then the engineer then the engineer then the engineer asks: what happens asks: what happens when someone's when someone's when someone's permission to permission to permission to view the document is revoked?

view the document is revoked? view the document is revoked? Could an old Could an old Could an old search index or search index or search index or saved response saved response saved response accidentally reveal accidentally reveal accidentally reveal content? If the answer is content? If the answer is content? If the answer is yes, then the company yes, then the company yes, then the company has a has a has a privacy problem, privacy problem, privacy problem, right? The feature right? The feature right? The feature seemed successful seemed successful seemed successful because no one checked because no one checked because no one checked a condition that a condition that a condition that was actually important for was actually important for was actually important for risk and risk and risk and safety assessment. The engineer safety assessment. The engineer safety assessment. The engineer eventually recognized this eventually recognized this eventually recognized this condition. The engineer condition. The engineer condition. The engineer traced its influence in the traced its influence in the traced its influence in the system. The engineer knew system. The engineer knew exactly what needed to exactly what needed to exactly what needed to be changed. You can be changed. You can be changed. You can write the test once you write the test once you write the test once you understand understand understand what question what question what question to ask. The ability to to ask. The ability to to ask. The ability to ask such a ask such a ask such a question in this question in this question in this example is part of example is part of example is part of the intelligence the intelligence the intelligence required to required to required to do the job. As well do the job. As well do the job. As well as understanding how as understanding how as understanding how that response may that response may that response may depend on other depend on other depend on other parts of the system, outside of the parts of the system, outside of the parts of the system, outside of the code that code that code that wrote the model. That's wrote the model. That's wrote the model. That's why I see why I see why I see programming as such an programming as such an programming as such an important challenge important challenge important challenge to the idea that we're to the idea that we're to the idea that we're all on the same path all on the same path all on the same path to superintelligence. to superintelligence. to superintelligence. Ask Ask Ask experienced experienced experienced engineers if they have engineers if they have engineers if they have run out of run out of run out of difficult tasks. No, it's difficult tasks. No, it's difficult tasks. No, it's not over. They not over. They not over. They deal with deal with deal with changes that changes that changes that interact with interact with interact with old code, old code, old code, assumptions that are assumptions that are assumptions that are no longer no longer no longer valid, and valid, and valid, and decisions that decisions that decisions that determine whether the system will be determine whether the system will be serviceable serviceable in six months. A in six months. A programming benchmark can programming benchmark can cease to be a cease to be a cease to be a useful test, you useful test, you useful test, you can exhaust can exhaust can exhaust its capabilities, and we have its capabilities, and we have its capabilities, and we have already done that. Although there are still already done that. Although there are still many very, very many very, very complex engineering complex engineering complex engineering tasks that tasks that only humans can currently handle.

And this distinction is And this distinction is important because important because important because generating generating generating features quickly is valuable, but features quickly is valuable, but features quickly is valuable, but creating more creating more creating more features features features that that that only a only a only a limited number of limited number of limited number of engineers can test and maintain just engineers can test and maintain just engineers can test and maintain just creates more creates more creates more work for them, right? This work for them, right? This work for them, right? This ultimately becomes the whole ultimately becomes the whole ultimately becomes the whole point of point of point of engineering, and this is what we engineering, and this is what we engineering, and this is what we are seeing across the are seeing across the are seeing across the industry right now. In industry right now. In other words, other words, engineering is a field engineering is a field engineering is a field where we would expect where we would expect where we would expect to see to see to see superintelligence, superintelligence, superintelligence, because it would have to because it would have to because it would have to manifest itself in a domain manifest itself in a domain manifest itself in a domain that is that is that is testable, where we have testable, where we have testable, where we have invested a lot of invested a lot of invested a lot of money, and where you would have to money, and where you would have to money, and where you would have to create something like a create something like a superhuman superhuman engineer. But we do engineer. But we do engineer. But we do n't see it. n't see it. n't see it. In fact, we see the In fact, we see the In fact, we see the opposite. We see opposite. We see engineers engineers engineers having to having to having to hedge against hedge against hedge against AI's weaknesses, AI's weaknesses, AI's weaknesses, even as AI becomes even as AI becomes even as AI becomes increasingly faster at increasingly faster at increasingly faster at writing code. This is not writing code. This is not writing code. This is not to say that AI is not to say that AI is not to say that AI is not extremely useful extremely useful extremely useful in engineering. That's right. It's in engineering. That's right. It's just that its just that its usefulness lies in usefulness lies in usefulness lies in generating code, not in generating code, not in generating code, not in designing code designing code designing code that can be that can be that can be maintained and maintained and maintained and scaled at the scaled at the scaled at the system system system architecture level. I know there architecture level. I know there architecture level. I know there are people in the labs are people in the labs are people in the labs who will say it's who will say it's who will say it's just a matter of time just a matter of time just a matter of time and we'll do it. But and we'll do it. But and we'll do it. But why didn't it why didn't it why didn't it happen then? Why did happen then? Why did happen then? Why did n't this happen? I think n't this happen? I think n't this happen? I think it's because it's hard to it's because it's hard to it's because it's hard to take the gains in take the gains in take the gains in intelligence and logic intelligence and logic intelligence and logic that you see in that you see in that you see in fields like fields like fields like mathematics and immediately mathematics and immediately mathematics and immediately apply them to apply them to apply them to systems engineering.

systems engineering. This doesn't seem to This doesn't seem to transfer from one to transfer from one to transfer from one to another. It doesn't another. It doesn't another. It doesn't transfer in the transfer in the transfer in the same way, and we see this same way, and we see this same way, and we see this even more clearly when we even more clearly when we even more clearly when we go beyond the go beyond the go beyond the technical sphere, technical sphere, technical sphere, beyond the purely logical beyond the purely logical beyond the purely logical field, into business. And field, into business. And field, into business. And if we follow the if we follow the if we follow the same line in business, same line in business, same line in business, it becomes even more it becomes even more it becomes even more relevant for us. relevant for us. relevant for us. Let's say the Let's say the Let's say the search function I just search function I just search function I just described was fixed by that described was fixed by that described was fixed by that engineer. Now it engineer. Now it engineer. Now it works great. You works great. You works great. You can sit down with can sit down with can sit down with the client. You can the client. You can the client. You can watch them watch them watch them use it. You use it. You use it. You see them see them see them find conflicting documents find conflicting documents , the client doesn't , the client doesn't know which one know which one know which one to trust, and can't to trust, and can't to trust, and can't make a decision—and make a decision—and make a decision—and now you have another now you have another now you have another problem, right? problem, right? problem, right? Now you've made it easier to Now you've made it easier to Now you've made it easier to find conflicting find conflicting find conflicting information that information that information that was always in your was always in your was always in your repository, and you've uncovered repository, and you've uncovered repository, and you've uncovered a bunch of discrepancies a bunch of discrepancies a bunch of discrepancies that the client needs to that the client needs to that the client needs to resolve. Now in resolve. Now in resolve. Now in our example, you our example, you our example, you could decide to could decide to could decide to spend another month spend another month spend another month improving search, improving search, improving search, leaving the problem leaving the problem leaving the problem as is, right? Someone as is, right? Someone as is, right? Someone has to determine who has to determine who has to determine who owns the information, owns the information, owns the information, which version is which version is which version is current, how it current, how it current, how it will remain will remain will remain accurate, and you can accurate, and you can accurate, and you can go through all the go through all the go through all the business rules, you business rules, you business rules, you can delegate that to the can delegate that to the can delegate that to the product manager. This product manager. This product manager. This could mean changing could mean changing could mean changing the way the way the way many teams work, and by the many teams work, and by the many teams work, and by the time you're time you're time you're done, it will be a done, it will be a done, it will be a completely different completely different completely different project. The person who project. The person who project. The person who discovers this problem discovers this problem discovers this problem makes an extremely makes an extremely makes an extremely valuable contribution to the valuable contribution to the valuable contribution to the business because they business because they business because they have found an obstacle have found an obstacle have found an obstacle to obtaining to obtaining to obtaining real value real value real value that the customer is willing to that the customer is willing to that the customer is willing to pay for. They pay for. They pay for. They found out what found out what found out what was happening in the was happening in the was happening in the client's work and client's work and client's work and changed the changed the changed the company's decision about what was company's decision about what was company's decision about what was needed to needed to needed to fix the situation fix the situation . You couldn't . You couldn't . You couldn't put this put this put this understanding into the understanding into the understanding into the initial instructions

initial instructions because you haven't because you haven't because you haven't discovered it yet. Finding discovered it yet. Finding discovered it yet. Finding missing missing missing information is information is information is part of the job, and I part of the job, and I part of the job, and I know you're saying, " know you're saying, " Oh, that was Oh, that was Oh, that was incredibly obvious." incredibly obvious." But this is a simplified But this is a simplified But this is a simplified example of a story that example of a story that example of a story that I have personally seen I have personally seen I have personally seen many times. In many times. In many times. In business intuition, business intuition, business intuition, finding the right finding the right finding the right question is often question is often question is often more important than anything more important than anything more important than anything else. And I find it else. And I find it else. And I find it interesting that we interesting that we interesting that we don't really have don't really have don't really have many examples of many examples of many examples of LLMs doing this. And LLMs doing this. And LLMs doing this. And this has implications for this has implications for this has implications for how businesses how businesses how businesses use AI. In use AI. In use AI. In our example, our example, our example, faster development is faster development is faster development is only beneficial because only beneficial because only beneficial because you get you get you get something to the customer something to the customer something to the customer to try sooner. If you to try sooner. If you to try sooner. If you use this use this use this opportunity to opportunity to opportunity to learn, you learn, you will definitely move faster will definitely move faster overall. You will be able to overall. You will be able to overall. You will be able to fix fix fix the project sooner. If you the project sooner. If you the project sooner. If you keep keep keep building features building features building features without knowing why without knowing why without knowing why the customer is stuck, you the customer is stuck, you the customer is stuck, you can move very, can move very, very quickly and very quickly and very quickly and lose a lot. So for lose a lot. So for lose a lot. So for all of us who are not all of us who are not all of us who are not mathematicians, who are mathematicians, who are mathematicians, who are stuck in this stuck in this stuck in this business world or the business world or the business world or the tech world, we're tech world, we're tech world, we're trying to trying to trying to figure out how figure out how figure out how to work with AI that is to work with AI that is to work with AI that is extremely good extremely good extremely good at generating. I described the at generating. I described the coding example because we are all coding example because we are all already familiar with it. But already familiar with it. But maybe it's not maybe it's not maybe it's not as good at as good at as good at some of the things some of the things some of the things we'd expect a we'd expect a we'd expect a super-smart super-smart super-smart AI to be good at. And I AI to be good at. And I AI to be good at. And I think one of the think one of the think one of the most important questions of most important questions of most important questions of our time is whether our time is whether our time is whether this will change in the this will change in the this will change in the near future.

near future. near future. The good news is that The good news is that The good news is that we will all live through the we will all live through the we will all live through the next few next few next few years and find out years and find out years and find out . But one . But one . But one conclusion that can be conclusion that can be conclusion that can be drawn from this drawn from this drawn from this point in the development of point in the development of point in the development of superintelligence superintelligence superintelligence is that it is that it is that it is not a guaranteed is not a guaranteed is not a guaranteed fact that fact that fact that superintelligence in superintelligence in superintelligence in one area one area one area will immediately transfer to others. will immediately transfer to others. And I think the reason is And I think the reason is very simple. Not every very simple. Not every very simple. Not every problem depends problem depends problem depends on intelligence. There are such on intelligence. There are such on intelligence. There are such things as business things as business intuition, as common intuition, as common intuition, as common sense in sense in sense in systems design. There are context-dependent things in business systems design. There are context-dependent things in business where it's where it's very, very difficult very, very difficult very, very difficult to get AI to be to get AI to be to get AI to be extremely extremely extremely effective. There are even effective. There are even effective. There are even things where you things where you things where you would expect progress but would expect progress but would expect progress but don't see it in AI yet don't see it in AI yet . And a good example— . And a good example— you would expect AI to be you would expect AI to be you would expect AI to be better at better at better at questioning your questioning your questioning your assumptions. You would assumptions. You would assumptions. You would expect that at this expect that at this expect that at this point AI would be better at point AI would be better at point AI would be better at reformulating reformulating reformulating questions and questions and questions and helping you helping you helping you ask them better. But ask them better. But ask them better. But because we've put because we've put because we've put so much effort into so much effort into so much effort into teaching AI to be teaching AI to be teaching AI to be useful, we've kind of useful, we've kind of useful, we've kind of pushed it in the pushed it in the pushed it in the opposite opposite opposite direction. He direction. He direction. He doesn't get better at doesn't get better at questioning your questioning your assumptions over time. So, this assumptions over time. So, this assumptions over time. So, this should encourage you.

should encourage you. should encourage you. Because if you look Because if you look Because if you look at stories like the one I at stories like the one I at stories like the one I described, described, described, these are real cases these are real cases these are real cases that I see over and over again that I see over and over again that I see over and over again where people where people where people create tremendous create tremendous create tremendous value. Is value. Is value. Is AI changing their role? Yes. Is AI changing their role? Yes. Is AI changing their role? Yes. Is AI leading to more and AI leading to more and more more more engineers not engineers not engineers not working with code working with code working with code directly? Yes. But does directly? Yes. But does directly? Yes. But does this mean they are this mean they are this mean they are left without a left without a left without a job? No. This means job? No. This means that they are changing the that they are changing the that they are changing the way they work because they way they work because they way they work because they have a new have a new have a new tool. And I think tool. And I think tool. And I think when we look at the when we look at the when we look at the example with example with example with mathematics, we mathematics, we mathematics, we see that this will change the see that this will change the see that this will change the way way way mathematicians mathematicians mathematicians do do do mathematics. This mathematics. This mathematics. This will change our approach to will change our approach to will change our approach to translating translating translating mathematical theories mathematical theories mathematical theories into applied into applied into applied engineering problems and engineering problems and engineering problems and products. But this does not products. But this does not products. But this does not necessarily mean necessarily mean necessarily mean that he will immediately know that he will immediately know that he will immediately know everything about business everything about business solutions and system solutions and system solutions and system design. It design. It design. It doesn't work that way. The level of doesn't work that way. The level of doesn't work that way. The level of intelligence is not intelligence is not intelligence is not wasted like that. And so wasted like that. And so wasted like that. And so I don't think the lesson is that I don't think the lesson is that you can stop you can stop learning in your learning in your learning in your field because of the fear that field because of the fear that field because of the fear that tomorrow AI will be tomorrow AI will be tomorrow AI will be so perfect so perfect . So, if you're . So, if you're . So, if you're trying to trying to trying to figure out where figure out where figure out where to put your to put your to put your skill development efforts, skill development efforts, skill development efforts, right? Why right? Why right? Why learn so you don't learn so you don't learn so you don't regret it in regret it in regret it in this world? I would this world? I would this world? I would advise you to think about advise you to think about advise you to think about how to leverage your how to leverage your how to leverage your experience, passion, and experience, passion, and experience, passion, and drive, drive, drive, focusing on an focusing on an focusing on an area of ​​influence that you area of ​​influence that you area of ​​influence that you can develop, can develop, can develop, viewing AI as a viewing AI as a viewing AI as a super-powerful super-powerful super-powerful tool to tool to tool to achieve your goals. You achieve your goals. You achieve your goals. You definitely definitely definitely need to learn how need to learn how need to learn how to use it.

to use it. to use it. You definitely You definitely You definitely need to learn need to learn need to learn how to approach this how to approach this . But it is your . But it is your . But it is your focus on focus on focus on business outcomes that business outcomes that business outcomes that determines when you determines when you determines when you use this use this use this tool. That's what tool. That's what tool. That's what defines when you defines when you defines when you put him put him put him aside and say, "I aside and say, "I aside and say, "I don't trust him." And it's don't trust him." And it's don't trust him." And it's a skill that I a skill that I a skill that I see in many see in many see in many areas of business now. This is a areas of business now. This is a areas of business now. This is a skill I see skill I see skill I see when I talk to when I talk to when I talk to engineers. This is engineers. This is engineers. This is a skill that I a skill that I a skill that I see experienced see experienced see experienced engineers wanting to engineers wanting to engineers wanting to see in beginners. see in beginners. see in beginners. This is a skill that This is a skill that salespeople are trying to master when salespeople are trying to master when thinking about how to thinking about how to thinking about how to set up an set up an set up an automated automated automated sales funnel to sales funnel to sales funnel to ensure ensure ensure the right conversations are happening at the the right conversations are happening at the the right conversations are happening at the right time to right time to right time to close deals. And I close deals. And I close deals. And I think one of the think one of the think one of the conclusions I conclusions I conclusions I want you to draw want you to draw want you to draw for yourself as you for yourself as you for yourself as you reflect on reflect on reflect on this moment is that this moment is that this moment is that we have an unexpectedly we have an unexpectedly we have an unexpectedly large amount of leeway in large amount of leeway in large amount of leeway in choosing how to choosing how to choosing how to use AI in the use AI in the use AI in the business world. We business world. We business world. We have unexpectedly have unexpectedly have unexpectedly great freedom of action in great freedom of action in great freedom of action in choosing how choosing how choosing how to use it to use it to use it in our personal in our personal in our personal lives. We lives. We lives. We choose where we want to choose where we want to choose where we want to use this use this use this tool and where we don't, and tool and where we don't, and tool and where we don't, and we determine we determine we determine where it is truly where it is truly where it is truly successful and where it is not. And successful and where it is not. And successful and where it is not. And one of the things I one of the things I one of the things I like about this like about this like about this is that we're making is that we're making is that we're making AI adapt AI adapt AI adapt to human conditions. We to human conditions. We to human conditions. We say: if AI say: if AI say: if AI is going to exist in our is going to exist in our is going to exist in our world, it has to world, it has to world, it has to exist amidst exist amidst exist amidst all the chaos that it all the chaos that it all the chaos that it means to be human, means to be human, means to be human, amidst the chaos in the amidst the chaos in the amidst the chaos in the workplace, in workplace, in workplace, in databases, and in our databases, and in our databases, and in our messy CRMs. It has to messy CRMs. It has to messy CRMs. It has to really work.

really work. really work. So I understand that it's So I understand that it's So I understand that it's cool to get cool to get cool to get superhuman superhuman superhuman results in results in results in math. This is a milestone in math. This is a milestone in math. This is a milestone in the development of artificial the development of artificial the development of artificial intelligence and, intelligence and, intelligence and, of course, an important of course, an important of course, an important stage in the history of stage in the history of stage in the history of mathematics. I'm mathematics. I'm mathematics. I'm thrilled, but I also have thrilled, but I also have thrilled, but I also have a lot of sympathy for the people a lot of sympathy for the people a lot of sympathy for the people who have worked on who have worked on who have worked on these problems their whole these problems their whole these problems their whole lives, and now that lives, and now that lives, and now that they've been solved, it's a they've been solved, it's a they've been solved, it's a difficult situation. difficult situation. But in the midst of all But in the midst of all this, I keep this, I keep this, I keep remembering that I want to have a remembering that I want to have a remembering that I want to have a clear idea of ​​what it clear idea of ​​what it clear idea of ​​what it means to be means to be means to be narrowly narrowly narrowly superintelligent. I don't superintelligent. I don't superintelligent. I don't want to suggest that want to suggest that want to suggest that if we have an AI that if we have an AI that if we have an AI that has such a sharp has such a sharp has such a sharp edge in edge in edge in mathematics and mathematics and mathematics and formal logic, formal logic, formal logic, it will automatically be it will automatically be it will automatically be able to design able to design able to design engineering systems engineering systems engineering systems so that we no longer so that we no longer so that we no longer need fuses need fuses . No, no. It doesn't . No, no. It doesn't . No, no. It doesn't work that way. This does not mean work that way. This does not mean work that way. This does not mean that he will immediately be able that he will immediately be able that he will immediately be able to produce truly to produce truly to produce truly wonderful prose that wonderful prose that wonderful prose that will seem will seem will seem incredibly incredibly incredibly convincing to us. I know convincing to us. I know convincing to us. I know the internet is the internet is the internet is full of full of AI "hallucinations," and AI "hallucinations," and AI "hallucinations," and we've been working on this for a we've been working on this for a we've been working on this for a long time. That doesn't long time. That doesn't long time. That doesn't mean he's mean he's mean he's able to instantly able to instantly able to instantly understand business understand business decisions, because those things decisions, because those things decisions, because those things aren't necessarily a aren't necessarily a aren't necessarily a function of function of function of IQ scores, right? And I think IQ scores, right? And I think IQ scores, right? And I think sometimes labs are sometimes labs are sometimes labs are just chasing just chasing just chasing IQ numbers. But we IQ numbers. But we IQ numbers. But we see that for see that for see that for many practical many practical many practical applications, it doesn't applications, it doesn't applications, it doesn't matter whether matter whether matter whether you have an IQ of 150 or 140.

you have an IQ of 150 or 140. you have an IQ of 150 or 140. It's about whether It's about whether It's about whether you can actually you can actually you can actually make the right make the right make the right decision and implement decision and implement decision and implement it. One of it. One of it. One of the things I the things I the things I thought about a lot was a thought about a lot was a thought about a lot was a comment made by comment made by comment made by Alexander Wang Alexander Wang Alexander Wang when he was talking about when he was talking about when he was talking about Muse and what Muse and what Muse and what they focused on. they focused on. they focused on. He spoke about He spoke about He spoke about the importance of execution the importance of execution . He spoke about . He spoke about . He spoke about the importance of striving the importance of striving the importance of striving for something, true for something, true for something, true focus, and focus, and focus, and seeing things through to seeing things through to seeing things through to the end. And I thought about the end. And I thought about the end. And I thought about it, because anyone who's been in it, because anyone who's been in it, because anyone who's been in business will tell you that there's business will tell you that there's no substitute for that kind of focus. You no substitute for that kind of focus. You must have a fire must have a fire must have a fire inside. You have to inside. You have to inside. You have to strive for something. strive for something. strive for something. That's not a function of IQ, is That's not a function of IQ, is That's not a function of IQ, is it? This is not a function of it? This is not a function of it? This is not a function of "superintelligence", in "superintelligence", in "superintelligence", in quotes. It is a function of quotes. It is a function of quotes. It is a function of the human desire to the human desire to the human desire to go out and change the world. go out and change the world. go out and change the world. And this, this And this, this And this, this cannot be taken away from us. The cannot be taken away from us. The cannot be taken away from us. The mathematical mathematical mathematical result will not result will not result will not change this. So I don't know change this. So I don't know what tomorrow will bring what tomorrow will bring what tomorrow will bring , but it , but it , but it looks like we live in a looks like we live in a looks like we live in a world with world with world with fragmented, fragmented, fragmented, narrow narrow narrow superintelligence, and superintelligence, and superintelligence, and we should plan we should plan we should plan accordingly. And accordingly. And accordingly. And I think in a way, I think in a way, I think in a way, that should give that should give that should give us more peace of mind us more peace of mind us more peace of mind than you might think. than you might think. than you might think. Because the area where this Because the area where this Because the area where this emerged says a lot emerged says a lot emerged says a lot about what labs are about what labs are about what labs are really really really focusing on focusing on focusing on and what they are and what they are and what they are able to get from able to get from able to get from these models. If these models. If these models. If they are forced to choose a they are forced to choose a they are forced to choose a problem like the problem like the problem like the Navier-Stokes equations Navier-Stokes equations , it is because they , it is because they , it is because they run out of problems run out of problems run out of problems in all other areas.

in all other areas. in all other areas. So, what's next? Not So, what's next? Not So, what's next? Not sure they have a sure they have a sure they have a perfect answer perfect answer perfect answer to that. We'll see. I to that. We'll see. I to that. We'll see. I will be ready for this will be ready for this will be ready for this journey with you journey with you . Let us know in . Let us know in . Let us know in the comments what you the comments what you the comments what you think about these think about these think about these mathematical mathematical mathematical results.