Judge a Transaction by the Company It Keeps — Somya Rai & MD Sharique, EXL

Hi, I'm Swami Iyer, Hi, I'm Swami Iyer, VP of VP of VP of AI Development. What we AI Development. What we AI Development. What we see here are two see here are two see here are two evaluations of the evaluations of the evaluations of the same transaction. On same transaction. On the left is one online purchase, where we purchase, where we purchase, where we take into account the amount, take into account the amount, take into account the amount, seller, MCC code, seller, MCC code, seller, MCC code, device type, time of day, and the device type, time of day, and the device type, time of day, and the 198 features we 198 features we 198 features we created. The estimate created. The estimate created. The estimate is 0.04—well is 0.04—well is 0.04—well below below any threshold at which we any threshold at which we any threshold at which we would sound the alarm. From would sound the alarm. From would sound the alarm. From our perspective, this our perspective, this our perspective, this transaction looks transaction looks transaction looks clean. The case is the clean. The case is the clean. The case is the same transaction, the same transaction, the same transaction, the same feature vector. same feature vector. same feature vector. No byte in this No byte in this No byte in this string is string is string is different. The only thing different. The only thing different. The only thing we change is what we change is what we change is what the model sees, the model sees, the model sees, that is, the environment in that is, the environment in that is, the environment in which it is located. which it is located. which it is located. Three out of four Three out of four Three out of four transactions, transactions, transactions, user, user, user, merchant, and two other merchant, and two other merchant, and two other transactions were transactions were transactions were flagged as flagged as flagged as fraudulent in the fraudulent in the fraudulent in the last hour. last hour. last hour. The score is now 0.9. The score is now 0.9. The score is now 0.9. So the signal was not in So the signal was not in So the signal was not in the string itself, but in the the string itself, but in the the string itself, but in the connections. Therefore, connections. Therefore, connections. Therefore, fraud is fraud is fraud is not a property not a property not a property of the transaction, but of of the transaction, but of of the transaction, but of the environment in which the environment in which the environment in which it occurs. it occurs. Let me describe Let me describe the structure of this the structure of this the structure of this problem, because these are two problem, because these are two problem, because these are two problems under the same problems under the same problems under the same name. A physical name. A physical name. A physical point-of- point-of- point-of- sale transaction has an sale transaction has an sale transaction has an approximate 0.05% approximate 0.05% natural natural natural fraud rate. This is one fraud rate. This is one fraud rate. This is one case in 2000. This is a case in 2000. This is a case in 2000. This is a clear signal with clear signal with clear signal with low noise, low noise, low noise, and fraud is and fraud is and fraud is mostly dependent mostly dependent mostly dependent on the speed of on the speed of card transactions. Whereas card transactions. Whereas online transactions online transactions online transactions have a 0.6% have a 0.6% have a 0.6% fraud rate, which is 12 fraud rate, which is 12 fraud rate, which is 12 times higher. There is higher times higher. There is higher times higher. There is higher volume, more noise, and volume, more noise, and volume, more noise, and much more often much more often much more often the signal is hidden in the signal is hidden in the signal is hidden in the context of the seller the context of the seller the context of the seller and the environment. These two and the environment. These two and the environment. These two portfolios have

portfolios have portfolios have different different different baselines and different baselines and different baselines and different signal geometries. signal geometries. signal geometries. Therefore, in practice, Therefore, in practice, Therefore, in practice, two models are needed, two models are needed, two models are needed, not one. And here is not one. And here is not one. And here is the limitation that makes the limitation that makes the limitation that makes this an engineering this an engineering this an engineering problem, not just a problem, not just a problem, not just a modeling issue modeling issue . The notification budget . The notification budget . The notification budget is limited. Analysts is limited. Analysts is limited. Analysts check from 0.5 to check from 0.5 to check from 0.5 to 1.5% of the evaluated 1.5% of the evaluated 1.5% of the evaluated transactions. And that's it. transactions. And that's it. So, the area under So, the area under the curve is not the only the curve is not the only the curve is not the only goal; accuracy is more important goal; accuracy is more important goal; accuracy is more important at at at fixed K. For fixed K. For fixed K. For the model to be the model to be the model to be worth anything, it must worth anything, it must worth anything, it must detect real detect real detect real fraud in this fraud in this fraud in this segment with the same segment with the same segment with the same delay. The obvious delay. The obvious delay. The obvious question is: why question is: why question is: why not just build a not just build a not just build a better XGBoost model? We better XGBoost model? We better XGBoost model? We did it. For did it. For did it. For physical points physical points physical points of sale, we increased of sale, we increased of sale, we increased the number of features from 40 the number of features from 40 the number of features from 40 raw raw raw attributes to 87 attributes to 87 attributes to 87 developed ones. After developed ones. After developed ones. After screening and screening and screening and data leak checks, the data leak checks, the data leak checks, the true true true positive positive positive rate increased from 35.5 rate increased from 35.5 rate increased from 35.5 to 36.6. to 36.6. The situation is similar with online transactions. The situation is similar with online transactions. Almost 200 Almost 200 Almost 200 features developed, and we get features developed, and we get features developed, and we get about one about one about one percentage point of percentage point of percentage point of growth. So, we have growth. So, we have growth. So, we have 2–5 times more 2–5 times more 2–5 times more surface area of ​​features, but surface area of ​​features, but surface area of ​​features, but we get only one we get only one we get only one TPR point. So, this is not a TPR point. So, this is not a TPR point. So, this is not a configuration error configuration error , right? This is , right? This is , right? This is ridiculous. Because ridiculous. Because ridiculous. Because every feature we have every feature we have every feature we have developed is just a developed is just a developed is just a function of the same function of the same function of the same line and the story of the line and the story of the line and the story of the same card. We are only same card. We are only same card. We are only projecting projecting projecting information we already information we already information we already had. Tabular signs had. Tabular signs had. Tabular signs answer one answer one answer one question. What does question. What does question. What does this this this transaction look like? They can't transaction look like? They can't transaction look like? They can't answer the answer the answer the second one. Who is she second one. Who is she second one. Who is she connected to and what are they connected to and what are they connected to and what are they doing right now?

doing right now? Device sharing, card speed chains, seller card speed chains, seller who became problematic 40 who became problematic 40 who became problematic 40 minutes ago. This exists minutes ago. This exists minutes ago. This exists in the relationships between in the relationships between in the relationships between lines. And a model that lines. And a model that lines. And a model that only sees one only sees one only sees one row at a time is row at a time is row at a time is inherently blind to inherently blind to inherently blind to that, no matter how many that, no matter how many that, no matter how many columns we columns we columns we add. Good. So, add. Good. So, add. Good. So, here is the mental model here is the mental model here is the mental model for the rest of the report. A for the rest of the report. A for the rest of the report. A graph neural graph neural graph neural network does not learn network does not learn network does not learn a node based on its a node based on its a node based on its own features. She own features. She own features. She learns from learns from learns from her surroundings. One her surroundings. One her surroundings. One layer, three operations. layer, three operations. layer, three operations. The first is the gathering. We The first is the gathering. We The first is the gathering. We pull up the pull up the pull up the hidden state of hidden state of hidden state of each neighbor through each neighbor through each neighbor through connections such as connections such as connections such as transaction to transaction to transaction to user, user, user, transaction to transaction to transaction to merchant. Aggregation. merchant. Aggregation. merchant. Aggregation. We collapse this We collapse this We collapse this set of neighbors, which, set of neighbors, which, set of neighbors, which, you know, is of different you know, is of different you know, is of different sizes for each sizes for each sizes for each node, into a single vector of node, into a single vector of node, into a single vector of fixed width. And fixed width. And fixed width. And we do this using a we do this using a permutation- invariant function invariant function . The choice of feature . The choice of feature mattered much more than we mattered much more than we expected, and Shalik expected, and Shalik expected, and Shalik will return to this later. will return to this later. will return to this later. Third—update. We Third—update. We Third—update. We concatenate this concatenate this concatenate this aggregate with the aggregate with the aggregate with the node's own state, node's own state, node's own state, pass it through the pass it through the pass it through the trained weights, and then trained weights, and then trained weights, and then stack it twice.

stack it twice. stack it twice. So each transaction So each transaction So each transaction knows about its knows about its knows about its neighbors and neighbors and neighbors and its neighbors' neighbors even before its neighbors' neighbors even before the classifier sees it. So, are these results confirmed? Yes, that's these results confirmed? Yes, that's right. And I want to be right. And I want to be right. And I want to be precise about what precise about what precise about what exactly we're comparing. exactly we're comparing. exactly we're comparing. We didn't replace XGBoost, We didn't replace XGBoost, We didn't replace XGBoost, we fueled it. we fueled it. we fueled it. The chain generates The chain generates The chain generates embeddings. These embeddings. These embeddings. These embeddings go embeddings go embeddings go into the same into the same into the same gradient gradient gradient boosting model, and it is the one that boosting model, and it is the one that boosting model, and it is the one that gets the score. So, gets the score. So, gets the score. So, XGBoost alone versus XGBoost XGBoost alone versus XGBoost XGBoost alone versus XGBoost plus graph—that's what plus graph—that's what plus graph—that's what we're comparing. So, we're comparing. So, we're comparing. So, three numbers we three numbers we three numbers we would like to highlight. would like to highlight. would like to highlight. The first is a relative The first is a relative The first is a relative increase of 27-33% in increase of 27-33% in the proportion of true the proportion of true the proportion of true positives positives compared to the compared to the tuned baseline tuned baseline tuned baseline XGBoost at the same XGBoost at the same XGBoost at the same notification rate, the notification rate, the notification rate, the same evaluation window, same evaluation window, same evaluation window, and the same features for and the same features for and the same features for both models. The second is both models. The second is approximately approximately approximately $65 million in $65 million in $65 million in projected projected projected annual additional annual additional annual additional fraud detection fraud detection fraud detection across two types of across two types of across two types of card card card fraud. And the third—3 fraud. And the third—3 fraud. And the third—3 milliseconds of P99 milliseconds of P99 milliseconds of P99 delay. So this delay. So this delay. So this report is not just about report is not just about report is not just about the numbers, it's about how the numbers, it's about how the numbers, it's about how we got there. we got there.

All decisions and everything that All decisions and everything that matters. First of matters. First of matters. First of all, the sample. all, the sample. all, the sample. Fraud happens Fraud happens Fraud happens in one out of 2000 in one out of 2000 in one out of 2000 cases, so we can't cases, so we can't cases, so we can't train train train on a large on a large on a large distribution. We need to distribution. We need to distribution. We need to improve improve improve fraud detection fraud detection , but this is not a tabular , but this is not a tabular , but this is not a tabular dataset where the rows are dataset where the rows are dataset where the rows are independent. Every independent. Every independent. Every rejected rejected rejected negative example negative example negative example takes with it its takes with it its takes with it its connections. If we connections. If we connections. If we simplify the sample, we simplify the sample, we simplify the sample, we keep the labels but keep the labels but keep the labels but lose the context lose the context lose the context that makes them that makes them that makes them important. So, important. So, important. So, the key was the key was the key was to maintain the to maintain the to maintain the fraud target fraud target fraud target without breaking without breaking without breaking the connection. Secondly, the connection. Secondly, the connection. Secondly, the graph and the signs. the graph and the signs. the graph and the signs. Creating a secure Creating a secure Creating a secure system is more difficult system is more difficult system is more difficult than it seems when the than it seems when the than it seems when the edges have edges have edges have timestamps and 32 derived timestamps and 32 derived timestamps and 32 derived features for features for features for fraud detection. The third is fraud detection. The third is fraud detection. The third is explainability. explainability. explainability. An analyst will not act An analyst will not act An analyst will not act solely on the basis of a number. solely on the basis of a number. solely on the basis of a number. This is a regulated area This is a regulated area This is a regulated area where the assessment must be where the assessment must be where the assessment must be accompanied by accompanied by accompanied by evidence. So, evidence. So, evidence. So, otherwise the product will not be otherwise the product will not be otherwise the product will not be released. released. released. Regarding the 3- Regarding the 3- Regarding the 3- millisecond speed: the millisecond speed: the aggregation method and the transition aggregation method and the transition from from from node-level prediction to node-level prediction to node-level prediction to edge-level led us to this. Finally, we edge-level led us to this. Finally, we edge-level led us to this. Finally, we implemented a implemented a implemented a post-launch iteration: post-launch iteration: post-launch iteration: training the model on a training the model on a training the model on a moving window with a moving window with a moving window with a time time time decay approach. And this decay approach. And this decay approach. And this improved the performance again.

improved the performance again. Now I'll turn the floor over to Now I'll turn the floor over to Shari to go Shari to go Shari to go into the details of into the details of into the details of graph creation, graph graph creation, graph graph creation, graph optimization, and optimization, and optimization, and engineering solutions engineering solutions engineering solutions to improve to improve to improve this model. this model. Hi, I'm Muhammad Hi, I'm Muhammad Sariq, Chief Sariq, Chief Sariq, Chief Architect of AI. These are Architect of AI. These are Architect of AI. These are Jaja transactions, and I'll Jaja transactions, and I'll Jaja transactions, and I'll show how our show how our show how our graph neural graph neural graph neural network outperformed network outperformed network outperformed XGBoost by 30% in XGBoost by 30% in XGBoost by 30% in fraud detection and runs fraud detection and runs fraud detection and runs in 3 milliseconds. I in 3 milliseconds. I in 3 milliseconds. I will talk about the will talk about the will talk about the technical implementation, technical implementation, technical implementation, sampling, sampling, sampling, graph construction, graph construction, graph construction, maintenance, and maintenance, and maintenance, and explainability. So, explainability. So, explainability. So, let's begin. The first let's begin. The first let's begin. The first decision that decision that decision that most teams most teams most teams make wrong is make wrong is make wrong is data sampling. We data sampling. We data sampling. We maintain a maintain a maintain a fraud target of 0.5% fraud target of 0.5% for card-based for card-based for card-based merchant transactions. merchant transactions. merchant transactions. This is about 10 This is about 10 This is about 10 times the times the times the natural natural natural fraud rate of 0.05%. And 6% fraud rate of 0.05%. And 6% for online for online transactions, again transactions, again transactions, again 10 times the 10 times the 10 times the natural 0.6%.

natural 0.6%. natural 0.6%. Both options are Both options are Both options are much softer than the much softer than the much softer than the aggressive sampling aggressive sampling aggressive sampling used in used in used in tabular pipelines tabular pipelines . Why? Because the complete . Why? Because the complete . Why? Because the complete transaction history of transaction history of transaction history of each selected each selected each selected user user user remains in the graph. remains in the graph. remains in the graph. If you shorten If you shorten If you shorten the story of a typical the story of a typical the story of a typical user for the sake of user for the sake of class correlation, you destroy class correlation, you destroy the connections that the connections that the connections that his neighbors depend on. his neighbors depend on. his neighbors depend on. And the fraudster's own real And the fraudster's own real And the fraudster's own real transactions transactions transactions remain as remain as remain as complex negative complex negative complex negative examples— examples— the most useful for the most useful for the most useful for defining the boundaries of defining the boundaries of defining the boundaries of solutions. If solutions. If solutions. If the limit is triggered the limit is triggered , only the " , only the " word budget" of word budget" of word budget" of regular regular regular users is reduced. Scammers users is reduced. Scammers users is reduced. Scammers never never never limit themselves. limit themselves. limit themselves. Class imbalance Class imbalance Class imbalance is addressed later is addressed later is addressed later using using using balanced balanced balanced within- within- within- trainer sampling. Here are trainer sampling. Here are trainer sampling. Here are the results. Average the results. Average the results. Average number of number of number of user connections during user connections during user connections during training: from 11.5 to training: from 11.5 to training: from 11.5 to almost 14 for card-based almost 14 for card-based almost 14 for card-based shopping transactions. shopping transactions. shopping transactions. Matches what Matches what Matches what the model sees during the model sees during the model sees during inference. If inference. If inference. If the environment during the environment during the environment during training does not training does not training does not match the match the match the environment during environment during environment during inference, the model inference, the model inference, the model learns the wrong learns the wrong learns the wrong things. This has been consistent things. This has been consistent things. This has been consistent over the 12 over the 12 over the 12 months we months we months we tested, with a tested, with a tested, with a natural frequency of natural frequency of natural frequency of 0.05% to 0.6%. There is no 0.05% to 0.6%. There is no 0.05% to 0.6%. There is no drift that needs to be drift that needs to be drift that needs to be corrected.

corrected. Second solution. Leakage control Leakage control is built into the graph itself is built into the graph itself , rather than checked , rather than checked , rather than checked after the fact. Each after the fact. Each after the fact. Each calendar month calendar month is a separate is a separate is a separate disconnected graph. disconnected graph. disconnected graph. No edge No edge No edge crosses the boundary. crosses the boundary. A transaction is a A transaction is a central node with central node with central node with edge types to edge types to edge types to the user and the user and the user and merchant, plus a merchant, plus a merchant, plus a chain of five chain of five chain of five transitions to its transitions to its transitions to its own recent own recent own recent history. Since history. Since history. Since the months never the months never the months never connect, there is no connect, there is no connect, there is no way for way for way for information to leak from one information to leak from one information to leak from one segment to another. segment to another. Four properties Four properties follow from this follow from this follow from this construction. construction. construction. No No No future leakage: A transaction future leakage: A transaction future leakage: A transaction always aggregates only always aggregates only always aggregates only neighbors with an earlier neighbors with an earlier neighbors with an earlier timestamp. timestamp. timestamp. No leakage No leakage No leakage between training, between training, between training, validation, and validation, and validation, and test samples test samples test samples due to the separation of due to the separation of due to the separation of months. No months. No months. No label leakage: The label leakage: The label leakage: The fraud label fraud label fraud label is a target, not a feature is a target, not a feature is a target, not a feature or edge. And or edge. And or edge. And no no no distribution leakage. Each distribution leakage. Each distribution leakage. Each scalar and dictionary scalar and dictionary scalar and dictionary are set only are set only are set only on the training on the training on the training month and then month and then month and then fixed. Four out of fixed. Four out of fixed. Four out of four four four are confirmed are confirmed are confirmed by construction, not by by construction, not by by construction, not by post-facto audit.

post-facto audit. Another decision before Another decision before architecture. 202 architecture. 202 architecture. 202 model features for model features for model features for three node types. 73 three node types. 73 three node types. 73 for the user, 109 for the user, 109 for the user, 109 for the transaction, 20 for the for the transaction, 20 for the for the transaction, 20 for the merchant. 32 merchant. 32 merchant. 32 transactional features transactional features transactional features are derivative. are derivative. are derivative. Specifically designed Specifically designed Specifically designed to detect to detect to detect fraud, added in fraud, added in fraud, added in three waves. 10 three waves. 10 three waves. 10 signs of the history of the signs of the history of the signs of the history of the entity. Number of entity. Number of entity. Number of past transactions, past transactions, past transactions, days since last days since last days since last activity, average, activity, average, activity, average, standard standard standard deviation, and deviation, and deviation, and maximum of the last maximum of the last maximum of the last five amounts. five amounts. five amounts. Behavioral basis Behavioral basis Behavioral basis for every for every for every user and user and user and seller. Eight seller. Eight seller. Eight risk aggregates. risk aggregates. risk aggregates. Estimate the amount regarding Estimate the amount regarding Estimate the amount regarding this story. this story. this story. The ratio of The ratio of The ratio of recent recent recent activity to activity to activity to past activity. Number of past activity. Number of past activity. Number of transactions per day. transactions per day. transactions per day. They mark They mark They mark deviations from one deviations from one deviations from one 's own basis of 's own basis of 's own basis of being, not from being, not from being, not from some global some global some global threshold. And 14 real threshold. And 14 real threshold. And 14 real signs of speed signs of speed signs of speed and time and time and time in the in the in the sliding window.

sliding window. sliding window. Number and amounts of Number and amounts of Number and amounts of transactions in 5 transactions in 5 transactions in 5 minutes, 1 hour and 24 minutes, 1 hour and 24 minutes, 1 hour and 24 hours. Number of hours. Number of hours. Number of counterparties in the counterparties in the counterparties in the last day, last day, last day, seconds since the last seconds since the last seconds since the last transaction, even a transaction, even a country change indicator to detect country change indicator to detect impossible trips impossible trips . Why is this important? . Why is this important? . Why is this important? The graph tells The graph tells The graph tells the model who the model who the model who the transaction is related to. the transaction is related to. These derived traits These derived traits indicate how this indicate how this indicate how this entity behaves entity behaves entity behaves differently than usual. differently than usual. These are additional signals, These are additional signals, not redundant ones. Under not redundant ones. Under not redundant ones. Under the hood is the the hood is the the hood is the GraphSAGE architecture. GraphSAGE architecture. Sampling, aggregation, Sampling, aggregation, combining, combining, combining, embedding. embedding. embedding. Sampling. A Sampling. A Sampling. A fixed set of fixed set of fixed set of neighbors for each neighbors for each neighbors for each step, taking into account step, taking into account step, taking into account the age. So one the age. So one the age. So one huge huge huge vendor node can't vendor node can't vendor node can't dominate every dominate every dominate every message. message. message. Aggregation. And here is a Aggregation. And here is a Aggregation. And here is a detail worth detail worth detail worth noting. We don't just noting. We don't just noting. We don't just take take take the mean, we the mean, we the mean, we concatenate three concatenate three concatenate three invariants to invariants to invariants to the order of the statistic: the order of the statistic: the order of the statistic: the mean, the maximum, and the the mean, the maximum, and the the mean, the maximum, and the standard standard standard deviation.

deviation. deviation. We combine this with the We combine this with the node's eigenvector through the learned node's eigenvector through the learned weights and embed a weights and embed a weights and embed a fixed- fixed- fixed- length vector in two steps. length vector in two steps. length vector in two steps. This is an inductive This is an inductive This is an inductive approach. We learn approach. We learn that the aggregator weights that the aggregator weights that the aggregator weights are not a fixed are not a fixed are not a fixed vector for a node, vector for a node, vector for a node, so a new so a new so a new user or user or user or seller seller seller is incorporated “on is incorporated “on is incorporated “on the fly” without the fly” without the fly” without retraining. Why retraining. Why retraining. Why three statistics and not three statistics and not three statistics and not one? Let me one? Let me one? Let me show you. Imagine show you. Imagine show you. Imagine four neighbors of four neighbors of four neighbors of a transaction. Three a transaction. Three a transaction. Three look quite look quite look quite normal, scores normal, scores normal, scores around 0.1. The fourth around 0.1. The fourth around 0.1. The fourth is associated with a group of is associated with a group of is associated with a group of scammers, with a score of 0.92. scammers, with a score of 0.92. scammers, with a score of 0.92. Average these four Average these four Average these four values ​​and you values ​​and you values ​​and you get 0.32. This get 0.32. This get 0.32. This suspicious neighbor suspicious neighbor suspicious neighbor simply disappeared simply disappeared simply disappeared among the rest. Take among the rest. Take among the rest. Take the maximum, and it the maximum, and it the maximum, and it will appear unchanged: will appear unchanged: will appear unchanged: 0.92. That's why we don't 0.92. That's why we don't 0.92. That's why we don't just rely on the just rely on the just rely on the average. average. One dangerous One dangerous neighbor among many neighbor among many neighbor among many safe ones is exactly safe ones is exactly safe ones is exactly the pattern that the pattern that the pattern that scammers create, and scammers create, and scammers create, and averaging just averaging just averaging just hides it.

hides it. Standard Standard deviation adds a deviation adds a deviation adds a third way: third way: third way: how much the neighbors are not the how much the neighbors are not the how much the neighbors are not the same as each other. A same as each other. A homogeneous environment is homogeneous environment is not at all like one not at all like one not at all like one where there is a bright emission. And where there is a bright emission. And where there is a bright emission. And it does not require it does not require it does not require additional costs additional costs additional costs during operation. during operation. No new No new parameters, parameters, parameters, fixed fixed fixed output size regardless of output size regardless of output size regardless of the number of neighbors. A the number of neighbors. A quick note quick note on hyperparameters on hyperparameters . Dimension 128 is the " . Dimension 128 is the " golden mean". golden mean". golden mean". Dimension 64 Dimension 64 Dimension 64 is close, but has a is close, but has a is close, but has a slight understudy. slight understudy. slight understudy. 256 requires more 256 requires more 256 requires more GPU resources, but does not GPU resources, but does not GPU resources, but does not provide any advantages. The provide any advantages. The validation efficiency levels validation efficiency levels off after 128 off after 128 , and then , and then , and then retraining begins. This is not a retraining begins. This is not a retraining begins. This is not a philosophical debate, but philosophical debate, but philosophical debate, but simply a selection of simply a selection of simply a selection of parameters. Now let parameters. Now let parameters. Now let 's compare the real 's compare the real 's compare the real numbers. For card numbers. For card numbers. For card transactions, the best transactions, the best transactions, the best XGBoost gives 36% true XGBoost gives 36% true positives at a positives at a notification level of 0.5%. GNN notification level of 0.5%. GNN notification level of 0.5%. GNN together with XGBoost gives 48%.

together with XGBoost gives 48%. together with XGBoost gives 48%. This is a 33% relative This is a 33% relative This is a 33% relative increase, $5.6 million increase, $5.6 million increase, $5.6 million was discovered at a was discovered at a was discovered at a disclosure level of 83%. disclosure level of 83%. disclosure level of 83%. For online For online transactions, the figure transactions, the figure transactions, the figure increased from 48% to 61%. This is a 27% increased from 48% to 61%. This is a 27% relative increase, relative increase, relative increase, $35.4 million $35.4 million $35.4 million was found at a level of 71%. was found at a level of 71%. Online transactions Online transactions provide the provide the provide the majority of these $65 million in majority of these $65 million in majority of these $65 million in savings, savings, savings, as as as fraud rates are 12 times fraud rates are 12 times fraud rates are 12 times higher there. Third solution: higher there. Third solution: higher there. Third solution: explainability. We explainability. We explainability. We performed performed performed subgraph attribution analysis subgraph attribution analysis subgraph attribution analysis for 200 labeled for 200 labeled for 200 labeled cases in each cases in each cases in each channel. channel. channel. Transaction speed relationships Transaction speed relationships Transaction speed relationships have an average have an average have an average importance of 0.94 in importance of 0.94 in importance of 0.94 in card transactions card transactions card transactions and 0.62 in online. They are the and 0.62 in online. They are the and 0.62 in online. They are the main evidence in main evidence in main evidence in all 200 cases with all 200 cases with all 200 cases with physical presence of physical presence of physical presence of the card. Relationships with the card. Relationships with the card. Relationships with sellers weighed sellers weighed sellers weighed more in online more in online transactions. 0.19 versus transactions. 0.19 versus transactions. 0.19 versus 0.04 in cases with a 0.04 in cases with a 0.04 in cases with a card. Where card. Where card. Where fraud evaluates fraud evaluates fraud evaluates the speed of the speed of the speed of the card itself, not the point of the card itself, not the point of the card itself, not the point of acceptance. Here's acceptance. Here's acceptance. Here's the difference: the model says the difference: the model says the difference: the model says "0.91" or the model says "0.91" or the model says "0.91" or the model says "0.91 because of these three "0.91 because of these three "0.91 because of these three related transactions

related transactions related transactions in the last 40 minutes." in the last 40 minutes." Why is this so important, Why is this so important, especially in the area of especially in the area of especially in the area of fraud? Four fraud? Four fraud? Four reasons. First: reasons. First: reasons. First: regulatory requirements regulatory requirements regulatory requirements and and and model risk management. model risk management. model risk management. Financial institutions Financial institutions Financial institutions operate under operate under operate under rules that rules that rules that require require require documented documented documented justification for justification for justification for automated automated automated decisions. This is not an option, decisions. This is not an option, decisions. This is not an option, it is an obligation. Second: it is an obligation. Second: it is an obligation. Second: the trust of analysts and the trust of analysts and the trust of analysts and the speed of the speed of the speed of investigations. An investigations. An investigations. An inexplicably high inexplicably high inexplicably high score is simply score is simply score is simply ignored or ignored or automatically approved. automatically approved. Edge-level proofs Edge-level proofs Edge-level proofs transform a transform a zero-sum case into a verification of zero-sum case into a verification of three related three related three related transactions. Third, transactions. Third, transactions. Third, model tuning. model tuning. model tuning. Attribution patterns Attribution patterns Attribution patterns show when show when show when a model relies on a a model relies on a a model relies on a false or false or false or outdated signal before outdated signal before performance performance noticeably drops in the noticeably drops in the noticeably drops in the work environment. work environment. work environment. And fourth, the quality of the And fourth, the quality of the And fourth, the quality of the feedback loop feedback loop feedback loop . When . When . When the explanations match the explanations match the analyst's suspicions, he the analyst's suspicions, he trusts the marginal trusts the marginal trusts the marginal solutions enough solutions enough solutions enough to label them correctly to label them correctly to label them correctly , , , improving the next improving the next improving the next stage of training stage of training stage of training data. The exact "black data. The exact "black data. The exact "black box" is simply not box" is simply not box" is simply not used. And used. And used. And now the real now the real now the real struggle: delivering struggle: delivering struggle: delivering results in 3 results in 3 results in 3 milliseconds. Offline, milliseconds. Offline, milliseconds. Offline, we run a we run a memory-limited layer-by-layer memory-limited layer-by-layer GraphSAGE forward pass on the GPU GraphSAGE forward pass on the GPU GraphSAGE forward pass on the GPU for the entire graph with for the entire graph with for the entire graph with packetization at the packetization at the packetization at the target node. So

target node. So , this is numerically , this is numerically , this is numerically identical to a full identical to a full identical to a full graph traversal, only graph traversal, only graph traversal, only limited to a few limited to a few limited to a few gigabytes across a gigabytes across a gigabytes across a few GPUs. Each few GPUs. Each few GPUs. Each user, user, user, merchant, and merchant, and merchant, and recent transaction recent transaction recent transaction receives receives receives a representation of a representation of a representation of each layer in the each layer in the low-latency embedding store. low-latency embedding store. The last five The last five The last five transactions of each transactions of each transactions of each user user user are in a are in a are in a ring buffer. ring buffer. Online at the time Online at the time of request, a lightweight of request, a lightweight of request, a lightweight build step restores build step restores build step restores neighbor states neighbor states neighbor states directly from directly from directly from this cache. No this cache. No this cache. No graph construction, graph construction, graph construction, no sampling. The no sampling. The exponentially exponentially decaying freshness weight accurately decaying freshness weight accurately decaying freshness weight accurately reproduces the reproduces the reproduces the learning process. The learning process. The learning process. The real- real- real- time service then only runs a time service then only runs a time service then only runs a new two-layer forward new two-layer forward pass for pass for pass for transactions from these transactions from these transactions from these cached states. cached states. cached states. Plus, they Plus, they Plus, they use " use " head". Nothing that head". Nothing that head". Nothing that scales with scales with scales with the size of the graph is the size of the graph is the size of the graph is executed during the executed during the executed during the query. Three little query. Three little query. Three little tricks make this tricks make this tricks make this quick in practice.

quick in practice. quick in practice. We limit the thread pools of We limit the thread pools of We limit the thread pools of math math math libraries to one libraries to one libraries to one even before loading even before loading even before loading heavy libraries. This is what heavy libraries. This is what heavy libraries. This is what makes the difference makes the difference makes the difference between 30 milliseconds between 30 milliseconds between 30 milliseconds and one. We and one. We and one. We warm up the model warm up the model warm up the model during startup so that the during startup so that the during startup so that the first real first real first real query is not query is not query is not slow. And today slow. And today slow. And today this is the only process in this is the only process in this is the only process in memory. The same memory. The same memory. The same interface is directly interface is directly interface is directly mapped to the mapped to the mapped to the distributed cache distributed cache distributed cache when we need to when we need to when we need to scale. But there is a scale. But there is a scale. But there is a nuance. This trick nuance. This trick nuance. This trick requires caching the requires caching the requires caching the representation for representation for representation for each node, each node, each node, including each including each including each transaction. On an transaction. On an transaction. On an industrial industrial industrial scale, this is 60 GB, which scale, this is 60 GB, which scale, this is 60 GB, which requires expensive requires expensive requires expensive server server server hardware with a large hardware with a large hardware with a large amount of memory. When amount of memory. When amount of memory. When embeddings are delayed embeddings are delayed embeddings are delayed by one day in by one day in by one day in real time, the real time, the real time, the true positive rate true positive rate drops from 48 % to 44%. Solution: make % to 44%. Solution: make % to 44%. Solution: make predictions on edges, predictions on edges, predictions on edges, not nodes. not nodes. not nodes. Pre- Pre- Pre- calculate calculate calculate embeds only for embeds only for embeds only for users and users and users and merchants, never merchants, never for each for each for each transaction. This transaction. This transaction. This reduces the size to 2-3 reduces the size to 2-3 reduces the size to 2-3 gigabytes, which is 20 gigabytes, which is 20 gigabytes, which is 20 times smaller, and times smaller, and times smaller, and cheap enough to cheap enough to cheap enough to run on a CPU of the same run on a CPU of the same run on a CPU of the same class as XGBoost next to it.

class as XGBoost next to it. Nvidia has independently Nvidia has independently recommended the recommended the recommended the same approach and expects same approach and expects it to perform it to perform it to perform on par with the on par with the on par with the node model. Here is the path of the node model. Here is the path of the node model. Here is the path of the real query. real query. Preprocessing, Preprocessing, cache lookups, and cache lookups, and cache lookups, and XGBoost head take XGBoost head take XGBoost head take fractions of a millisecond. The fractions of a millisecond. The GNN's own two-layer forward pass is the GNN's own two-layer forward pass is the largest part — largest part — 1.61 milliseconds. 1.61 milliseconds. 1.61 milliseconds. End-to-end performance: End-to-end performance: End-to-end performance: 2.3 milliseconds in 2.3 milliseconds in 2.3 milliseconds in median, 3.1 at P99 median, 3.1 at P99 , which , which , which fits comfortably within our SLA fits comfortably within our SLA fits comfortably within our SLA of 3 to 4 milliseconds. of 3 to 4 milliseconds. of 3 to 4 milliseconds. If we need If we need If we need more headroom, the direct more headroom, the direct more headroom, the direct propagation GNN is the propagation GNN is the propagation GNN is the first thing we first thing we first thing we recompile or recompile or recompile or quantize. Thank you for quantize. Thank you for quantize. Thank you for watching. We are both watching. We are both watching. We are both willing to delve into the willing to delve into the willing to delve into the details: sampling, details: sampling, details: sampling, graph construction, graph construction, graph construction, interpretability, interpretability, interpretability, or maintenance.

or maintenance. or maintenance. Contact us Contact us anytime. Thank you.