Your GPU Neighbor Can Hear Your Model — Branislav Brzak, Stealthium

Hello everyone. This Hello everyone. This conversation is about conversation is about conversation is about the dangers of the dangers of the dangers of information leakage from your LLM information leakage from your LLM information leakage from your LLM through shared through shared through shared GPU hardware. If you GPU hardware. If you GPU hardware. If you run a model in the run a model in the run a model in the cloud on a GPU shared cloud on a GPU shared cloud on a GPU shared with other tenants, with other tenants, with other tenants, someone nearby can someone nearby can someone nearby can learn learn learn certain things about it. They certain things about it. They certain things about it. They can determine what can determine what can determine what model you are running, model you are running, model you are running, how quickly it how quickly it how quickly it responds, and even responds, and even responds, and even what type of work it what type of work it what type of work it is currently doing, without is currently doing, without is currently doing, without touching the data, touching the data, touching the data, just by observing just by observing just by observing time. That's what time. That's what time. That's what this conversation is about. What this conversation is about. What this conversation is about. What a roommate on the a roommate on the a roommate on the same same same equipment can equipment can equipment can learn about your LLM learn about your LLM learn about your LLM just from the time just from the time just from the time characteristics and characteristics and characteristics and what you can do about it what you can do about it what you can do about it . To start, . To start, . To start, I'm Branislav Brzak, a I'm Branislav Brzak, a cybersecurity researcher at Stelthium. cybersecurity researcher at Stelthium. At Stelthium, we At Stelthium, we At Stelthium, we specialize in specialize in specialize in ensuring the security ensuring the security ensuring the security and surveillance of and surveillance of and surveillance of environments where environments where environments where AI systems operate. AI systems operate. AI systems operate. My area of ​​interest is My area of ​​interest is low-level low-level low-level security research security research security research related to GPUs. That's what related to GPUs. That's what related to GPUs. That's what this conversation covers this conversation covers . When you share GPUs between . When you share GPUs between . When you share GPUs between tenants, you tenants, you tenants, you expect complete expect complete expect complete isolation, but the isolation, but the isolation, but the isolation provided may not isolation provided may not isolation provided may not be be be sufficient. In the sufficient. In the sufficient. In the case of this attack, case of this attack, case of this attack, the attacker the attacker the attacker doesn't even need doesn't even need doesn't even need special permissions, special permissions, special permissions, such as root access. It such as root access. It such as root access. It also doesn't need any also doesn't need any also doesn't need any traditional traditional traditional exploits. The only thing exploits. The only thing exploits. The only thing it does is it does is it does is measure the operating time of measure the operating time of measure the operating time of parts of the equipment parts of the equipment parts of the equipment that remain that remain that remain common to common to common to tenants. This is a tenants. This is a tenants. This is a demonstration video demonstration video demonstration video of what of what the victim and the victim and attacker control panels look like. On the left we attacker control panels look like. On the left we attacker control panels look like. On the left we have the Quant 3 model have the Quant 3 model have the Quant 3 model performing inference.

performing inference. performing inference. In the case, the attacker In the case, the attacker In the case, the attacker uses a uses a uses a side channel into side channel into side channel into which Quant 3 "leaks" which Quant 3 "leaks" which Quant 3 "leaks" while running. From while running. From while running. From this, it determines this, it determines this, it determines what kind of model it is and what kind of model it is and what kind of model it is and what text it is currently what text it is currently what text it is currently generating. These are just the generating. These are just the generating. These are just the main points, but not the main points, but not the main points, but not the whole story. We whole story. We whole story. We can determine can determine can determine the model, architecture, the model, architecture, the model, architecture, and depth with and depth with and depth with 97% accuracy. We can 97% accuracy. We can 97% accuracy. We can determine the determine the determine the decoding speed with an decoding speed with an decoding speed with an accuracy of 99.3%. We accuracy of 99.3%. We accuracy of 99.3%. We can determine can determine can determine when and how much, when and how much, when and how much, load periods load periods , number of tokens, , number of tokens, , number of tokens, and response length. and response length. and response length. We can't We can't We can't determine exactly what determine exactly what determine exactly what type of work type of work type of work is being done: code, is being done: code, is being done: code, creativity, prose, JSON, creativity, prose, JSON, creativity, prose, JSON, or search. What we or search. What we or search. What we cannot know cannot know cannot know is the prompts themselves is the prompts themselves , the answers, , the answers, , the answers, or the content. Only or the content. Only or the content. Only execution time. And all this on execution time. And all this on execution time. And all this on real H100s real H100s real H100s serving real serving real serving real LLMs. We use LLMs. We use LLMs. We use Nvidia hardware. Nvidia hardware. Nvidia hardware. GPU isolation GPU isolation GPU isolation is achieved using is achieved using Multi-Instance GPU technology, or Multi-Instance GPU technology, or MIG for short. Each MIG for short. Each MIG for short. Each isolated isolated isolated MIG segment has its own physical MIG segment has its own physical MIG segment has its own physical compute units compute units compute units and dedicated VRAM. This and dedicated VRAM. This and dedicated VRAM. This provides hardware provides hardware provides hardware isolation for these isolation for these isolation for these parts. In our parts. In our parts. In our example, the LLM we are example, the LLM we are example, the LLM we are servicing runs servicing runs servicing runs on a segment with three on a segment with three on a segment with three compute compute compute units and 48 GB of VRAM.

units and 48 GB of VRAM. units and 48 GB of VRAM. The attacker is running The attacker is running The attacker is running on only one on only one on only one compute compute compute unit and 10 GB of VRAM. The unit and 10 GB of VRAM. The computing units, computing units, memory capacity, bandwidth quotas, memory capacity, bandwidth quotas, and fault tolerance are isolated. The and fault tolerance are isolated. The L2 cache L2 cache L2 cache is also split for is also split for is also split for each instance. There each instance. There each instance. There is no displacement is no displacement is no displacement between instances here. between instances here. between instances here. So, in that sense, there is So, in that sense, there is So, in that sense, there is nothing in common nothing in common nothing in common . The actual . The actual . The actual leaks are leaks are leaks are memory barriers, or membars, that memory barriers, or membars, that memory barriers, or membars, that operate at the GPU-wide level operate at the GPU-wide level operate at the GPU-wide level . The single . The single . The single instance barrier still instance barrier still instance barrier still affects every affects every affects every L2 partition on the map. Time is the L2 partition on the map. Time is the L2 partition on the map. Time is the only thing that is not only thing that is not only thing that is not isolated. So isolated. So isolated. So when one tenant when one tenant when one tenant is busy, the others is busy, the others is busy, the others work just a little work just a little work just a little slower. The key slower. The key slower. The key point is that the point is that the GPU-wide memory barrier operates in every GPU-wide memory barrier operates in every L2 partition on the card. The L2 partition on the card. The next slide next slide bases the entire attack on bases the entire attack on bases the entire attack on this single fact. this single fact. The vector is that the attacker the attacker the attacker only reads his own memory. only reads his own memory. only reads his own memory. It measures the time of only It measures the time of only It measures the time of only its own readings. its own readings. its own readings. He never He never He never touches the memory of the touches the memory of the touches the memory of the victim. There are three steps. victim. There are three steps. victim. There are three steps. Loading from Loading from Loading from its own small its own small its own small buffer, measuring buffer, measuring buffer, measuring loading time and loading time and loading time and repeating every 5 repeating every 5 repeating every 5 microseconds. This microseconds. This microseconds. This primitive itself leaked primitive itself leaked primitive itself leaked through membar in MIG—not through membar in MIG—not through membar in MIG—not our discovery.

our discovery. our discovery. Credit goes to Credit goes to Credit goes to the researchers behind the researchers behind the researchers behind the article " the article " Behind Bars" on Usenix Security. Behind Bars" on Usenix Security. Behind Bars" on Usenix Security. Many thanks for their Many thanks for their Many thanks for their work. Now work. Now work. Now try to look try to look try to look at the time difference. To at the time difference. To at the time difference. To the naked eye, the naked eye, the naked eye, comparing comparing comparing your tenant's downtime and your tenant's downtime and your tenant's downtime and LLM's maintenance, you LLM's maintenance, you LLM's maintenance, you won't even be able to won't even be able to won't even be able to tell the difference. The median tell the difference. The median tell the difference. The median latency is only latency is only latency is only 3 microseconds higher. However, 3 microseconds higher. However, 3 microseconds higher. However, for an attacker, those 3 for an attacker, those 3 for an attacker, those 3 microseconds are all it microseconds are all it microseconds are all it takes. This is takes. This is takes. This is enough to enough to enough to gather all the gather all the gather all the information information information we have already talked about. Quantri we have already talked about. Quantri we have already talked about. Quantri with 8 billion with 8 billion with 8 billion parameters has been working on the parameters has been working on the parameters has been working on the victim segment victim segment victim segment for for for this entire time. All this entire time. All this entire time. All the data obtained were the data obtained were the data obtained were taken from a real taken from a real taken from a real model. Above— model. Above— GPU activity over time, GPU activity over time, GPU activity over time, one burst per token. one burst per token. one burst per token. Below is the frequency Below is the frequency Below is the frequency spectrum of the attacker's spectrum of the attacker's spectrum of the attacker's own latency trace own latency trace own latency trace . There is a . There is a . There is a fundamental fundamental fundamental frequency of 881 hertz, which is frequency of 881 hertz, which is frequency of 881 hertz, which is simply tokens per simply tokens per simply tokens per second, plus second, plus second, plus harmonics, since harmonics, since harmonics, since each token is a mixture of each token is a mixture of each token is a mixture of individual frequencies, not individual frequencies, not individual frequencies, not one pure one. This is a one pure one. This is a one pure one. This is a real trace, real trace, real trace, one token in the time one token in the time one token in the time domain. The token domain. The token domain. The token shows a total of 39 shows a total of 39 shows a total of 39 peaks. The peaks with peaks. The peaks with peaks. The peaks with green tops green tops are layer challenges, are layer challenges, are layer challenges, while the orange ones are while the orange ones are additional processing additional processing additional processing between layers. In between layers. In between layers. In the following slides, we will the following slides, we will the following slides, we will demonstrate how demonstrate how demonstrate how we determine the we determine the we determine the actual number of actual number of actual number of layers, even with layers, even with layers, even with these these these additional peaks. Here additional peaks. Here additional peaks. Here we look at the we look at the decoding speed. Only decoding speed. Only by measuring by measuring by measuring time did we determine time did we determine time did we determine the speed of tokens the speed of tokens the speed of tokens blindly. So, blindly. So, blindly. So, speed speed speed itself is something that itself is something that itself is something that an attacker can

an attacker can an attacker can accurately recover. But accurately recover. But accurately recover. But speed alone speed alone speed alone is not a clear is not a clear is not a clear reflection. Quantri 8 reflection. Quantri 8 reflection. Quantri 8 billion and Llama 3 8 billion and Llama 3 8 billion and Llama 3 8 billion billion billion differ differ differ by only 1.6% in by only 1.6% in by only 1.6% in token velocity. So just by token velocity. So just by token velocity. So just by the speed you would the speed you would the speed you would think it was the same think it was the same think it was the same model. This model. This model. This narrows the search, narrows the search, narrows the search, but does not provide an exact but does not provide an exact but does not provide an exact answer. We answer. We answer. We need something need something need something more precise, the number of more precise, the number of more precise, the number of layers. This is what the layers. This is what the layers. This is what the next next next few slides are about. We few slides are about. We few slides are about. We use the use the use the same same same convolution and convolution and convolution and averaging method as averaging method as averaging method as before. This time we before. This time we before. This time we run it run it run it once for the model, not once for the model, not once for the model, not once for the token. once for the token. once for the token. The result is a The result is a The result is a number of bits that number of bits that number of bits that almost exactly almost exactly almost exactly matches an matches an matches an integer. The average integer. The average integer. The average deviation from the deviation from the deviation from the integer over 22 integer over 22 integer over 22 measurements is measurements is measurements is only 0.028. This is a only 0.028. This is a only 0.028. This is a hint. If it were hint. If it were hint. If it were just a fixed just a fixed just a fixed hardware artifact hardware artifact hardware artifact divided by different divided by different divided by different token rates, we token rates, we token rates, we would expect fractions, would expect fractions, would expect fractions, not whole numbers. Each not whole numbers. Each not whole numbers. Each count count count corresponds to corresponds to corresponds to the number of layers in the number of layers in the number of layers in the model plus one to three the model plus one to three the model plus one to three additional peaks. Each additional peaks. Each additional peaks. Each model retains model retains model retains its own signal its own signal its own signal fingerprints, combination of fingerprints, combination of fingerprints, combination of tempo and harmonic tempo and harmonic tempo and harmonic structure. To be structure. To be structure. To be clear, this is not a clear, this is not a clear, this is not a trained trained machine learning classifier.

machine learning classifier. There is no There is no There is no training set, no labels training set, no labels , no configuration. , no configuration. , no configuration. It's less than 30 lines of It's less than 30 lines of It's less than 30 lines of code. The method checks code. The method checks code. The method checks each each each candidate depth from 16 to 48. candidate depth from 16 to 48. candidate depth from 16 to 48. It measures It measures It measures how clearly the how clearly the how clearly the expected signature of expected signature of expected signature of that depth that depth that depth appears in the appears in the appears in the actual trace actual trace . Then he chooses the one . Then he chooses the one that fits that fits that fits best. For a best. For a best. For a quantized model quantized model quantized model with 8 billion with 8 billion with 8 billion parameters, 36 layers parameters, 36 layers parameters, 36 layers confidently win by a confidently win by a confidently win by a margin of 4.5 times over the margin of 4.5 times over the margin of 4.5 times over the next candidate next candidate . Because it is a . Because it is a . Because it is a best- best- best- fit test and not a fit test and not a fit test and not a lookup table, it lookup table, it lookup table, it also works with also works with also works with models that have never been models that have never been models that have never been seen before. This seen before. This seen before. This applies to both applies to both applies to both proprietary and proprietary and proprietary and unreleased models. unreleased models. Because he didn't Because he didn't need to need to need to "see" anything in advance "see" anything in advance . And it works surprisingly . And it works surprisingly . And it works surprisingly well. The well. The well. The layer classifier also outputs a layer classifier also outputs a layer classifier also outputs a confidence level. confidence level. confidence level. According to According to According to our testing results, our testing results, our testing results, only two answers with the only two answers with the only two answers with the lowest confidence level lowest confidence level turned out turned out to be incorrect. This to be incorrect. This to be incorrect. This same method also same method also same method also recognizes retrained recognizes retrained recognizes retrained models and installations models and installations models and installations that use the that use the that use the same base model.

same base model. same base model. This is because This is because This is because the architecture and the architecture and the architecture and number of layers number of layers number of layers remain remain remain the same. Let's take the same. Let's take the same. Let's take three three three related models as an example. related models as an example. related models as an example. Llama 3 for 8 billion Llama 3 for 8 billion Llama 3 for 8 billion from Meta has 32 layers. Llama from Meta has 32 layers. Llama from Meta has 32 layers. Llama 3.1 at 8 billion, 3.1 at 8 billion, 3.1 at 8 billion, also from Meta, also from Meta, also from Meta, uses the same uses the same uses the same 32-layer foundation with 32-layer foundation with 32-layer foundation with minor minor minor updates. And R1, still updates. And R1, still updates. And R1, still Llama at 8 billion Llama at 8 billion Llama at 8 billion from DeepSeek, is from DeepSeek, is from DeepSeek, is an improvement on that an improvement on that . The timings per . The timings per . The timings per layer differ layer differ layer differ by only 1.6 by only 1.6 by only 1.6 nanoseconds of nanoseconds of nanoseconds of median latency. median latency. The rest of the signal The rest of the signal looks the same. looks the same. looks the same. Despite this, our Despite this, our Despite this, our classifier is still classifier is still classifier is still able to distinguish between able to distinguish between able to distinguish between them. Now we them. Now we them. Now we will ask a deeper will ask a deeper will ask a deeper question on the same question on the same question on the same channel. It's not what channel. It's not what channel. It's not what model it is, but what it's model it is, but what it's model it is, but what it's doing right now. doing right now. doing right now. We consider We consider We consider four types of four types of four types of workloads: prose, workloads: prose, workloads: prose, code, JSON, and search. We code, JSON, and search. We code, JSON, and search. We make a verdict make a verdict make a verdict every half second every half second every half second based on the last 5 based on the last 5 based on the last 5 seconds of timing. seconds of timing. seconds of timing. Of the 40 half-second Of the 40 half-second Of the 40 half-second verdicts, 31 were verdicts, 31 were verdicts, 31 were correct, and four correct, and four correct, and four full responses were full responses were full responses were correctly correctly correctly identified by a identified by a identified by a majority vote. This is the majority vote. This is the majority vote. This is the same same same classifier from the classifier from the classifier from the previous video, previous video, previous video, including all the including all the including all the errors. It is worth errors. It is worth errors. It is worth noting one condition.

noting one condition. noting one condition. This was measured with This was measured with speculative speculative decoding enabled. This is decoding enabled. This is decoding enabled. This is an option in VLLM, not a an option in VLLM, not a an option in VLLM, not a default setting default setting default setting . Here is . Here is . Here is the mechanism. the mechanism. the mechanism. The predictability The predictability The predictability of the text affects of the text affects of the text affects the number of the number of the number of token drafts accepted at token drafts accepted at token drafts accepted at each step. This each step. This each step. This changes the step execution time changes the step execution time changes the step execution time , and it is this , and it is this , and it is this signal that we record. signal that we record. signal that we record. We tested 11 We tested 11 We tested 11 configurations: dense configurations: dense configurations: dense models, models, models, expert mix, BF16, and FP8. The expert mix, BF16, and FP8. The expert mix, BF16, and FP8. The same same same graph appeared graph appeared graph appeared every time, just with different every time, just with different every time, just with different intensities. intensities. intensities. The spread is 70.2% for GPT 20 The spread is 70.2% for GPT 20 The spread is 70.2% for GPT 20 billion versus 0.9% billion versus 0.9% for GLM 49 billion. for GLM 49 billion. for GLM 49 billion. Then, of the 11 runs, Then, of the 11 runs, Then, of the 11 runs, prose is the fastest, although prose is the fastest, although prose is the fastest, although chance would have given chance would have given chance would have given only one chance out of only one chance out of only one chance out of four. Let's assume four. Let's assume four. Let's assume the attacker already knows the attacker already knows the attacker already knows the model from the previous the model from the previous the model from the previous slides and slides and slides and speculative speculative speculative decoding is decoding is decoding is enabled. It enabled. It enabled. It can also determine what kind of can also determine what kind of can also determine what kind of text text text is generated. Now we is generated. Now we is generated. Now we completely eliminate the completely eliminate the completely eliminate the shared chip from the shared chip from the shared chip from the equation. For MLG, equation. For MLG, equation. For MLG, the attacker must be the attacker must be the attacker must be on the same on the same on the same physical crystal. physical crystal. physical crystal. This attack doesn't This attack doesn't This attack doesn't need that. need that. need that. The setup here is a The setup here is a The setup here is a server with eight H100s server with eight H100s connected via an connected via an connected via an NVLink switch. The victim NVLink switch. The victim NVLink switch. The victim runs tensor runs tensor runs tensor parallelism on GPUs 0 and 1.

parallelism on GPUs 0 and 1. parallelism on GPUs 0 and 1. The attacker The attacker The attacker only needs any other only needs any other only needs any other GPU on the same GPU on the same GPU on the same server, in this case server, in this case server, in this case GPU 2. Peer-to-peer GPU 2. Peer-to-peer GPU 2. Peer-to-peer access via NV access via NV switch is a completely switch is a completely switch is a completely by-product of the feature. by-product of the feature. by-product of the feature. This is a standard This is a standard This is a standard principle of principle of principle of multi-GPU tasks, multi-GPU tasks, multi-GPU tasks, not a vulnerability. MIG not a vulnerability. MIG not a vulnerability. MIG and tensor and tensor and tensor parallelism also parallelism also parallelism also cannot coexist cannot coexist cannot coexist on the same device. on the same device. on the same device. So this is actually a So this is actually a So this is actually a different class of different class of different class of deployment, not deployment, not deployment, not just a repeat of the just a repeat of the just a repeat of the same attack. same attack. The problem occurs on The problem occurs on GPU 2. It monitors GPU 2. It monitors GPU 2. It monitors memory that memory that memory that is actually on GPU 0 is actually on GPU 0 is actually on GPU 0 via NVLink. Measuring via NVLink. Measuring via NVLink. Measuring the time of each transition the time of each transition the time of each transition on the path. The idle on the path. The idle on the path. The idle transition takes transition takes transition takes about 890 nanoseconds about 890 nanoseconds . When the victim starts . When the victim starts . When the victim starts working, this time working, this time working, this time increases to 300 increases to 300 increases to 300 microseconds. These are three microseconds. These are three microseconds. These are three half-second recordings of " half-second recordings of " mirrors". And you mirrors". And you mirrors". And you can even count can even count can even count the tokens by eye. What we the tokens by eye. What we the tokens by eye. What we lose compared lose compared lose compared to MIG is the layer bits to MIG is the layer bits . At this distance, it is . At this distance, it is . At this distance, it is below the noise level.

below the noise level. Therefore, due to NVLink, the Therefore, due to NVLink, the footprints that footprints that footprints that remain are remain are remain are primarily temporal, primarily temporal, primarily temporal, rather than layer count rather than layer count . Here we test . Here we test . Here we test five models on five models on five models on identical identical service settings. Only service settings. Only the model itself reveals the model itself reveals the model itself reveals what it is. This what it is. This what it is. This method gives 79.2% accuracy method gives 79.2% accuracy . It fails exactly . It fails exactly . It fails exactly where one would expect where one would expect where one would expect from temporal analysis. from temporal analysis. from temporal analysis. Llama 3.5 8 billion and R1 Llama 3.5 8 billion and R1 Llama 3.5 8 billion and R1 distill are 8 billion distill are 8 billion distill are 8 billion models that decode models that decode models that decode with a difference of 3 Hz. Therefore, with a difference of 3 Hz. Therefore, with a difference of 3 Hz. Therefore, they merge where they merge where they merge where MIG would easily separate MIG would easily separate MIG would easily separate them by the number of them by the number of them by the number of layers. The conclusion is layers. The conclusion is layers. The conclusion is that the that the signal-to-noise ratio in NVLink is signal-to-noise ratio in NVLink is worse than in MIG, because worse than in MIG, because layer bits are lost. But a worse signal-to-noise ratio is not signal-to-noise ratio is not security. 79% versus 20% of the security. 79% versus 20% of the baseline—this is baseline—this is baseline—this is still a real still a real still a real footprint, not noise. footprint, not noise. First, some good First, some good news. Our own news. Our own news. Our own product at work product at work product at work detected this attack detected this attack detected this attack immediately after immediately after immediately after installation. Stealthy installation. Stealthy installation. Stealthy monitors every monitors every monitors every kernel launch on kernel launch on kernel launch on every GPU, analyzes every GPU, analyzes every GPU, analyzes suspicious suspicious suspicious workloads, and workloads, and workloads, and runs them through a runs them through a runs them through a full full full notification pipeline—something notification pipeline—something notification pipeline—something not every vendor can do not every vendor can do not every vendor can do . That's what . That's what . That's what made this case stand out.

made this case stand out. made this case stand out. One core was One core was One core was restarted about restarted about restarted about 3,000 times, each time 3,000 times, each time 3,000 times, each time accessing the accessing the accessing the same 40 kilobytes of same 40 kilobytes of same 40 kilobytes of memory, instead of memory, instead of memory, instead of moving on to new moving on to new moving on to new data. Only 1 kilobyte of data. Only 1 kilobyte of data. Only 1 kilobyte of data was sent data was sent data was sent back to the CPU in the entire back to the CPU in the entire back to the CPU in the entire time, as the time, as the payload was the payload was the execution time itself, execution time itself, execution time itself, not the result. In not the result. In not the result. In real time, it's real time, it's real time, it's less than 600 less than 600 less than 600 milliseconds from milliseconds from milliseconds from start to finish with start to finish with start to finish with less than 50 less than 50 less than 50 response tokens. We discovered response tokens. We discovered response tokens. We discovered this before this before this before the answer was even the answer was even the answer was even half half half typed. So, typed. So, typed. So, three main conclusions. three main conclusions. three main conclusions. First, placement First, placement is something that operators is something that operators is something that operators actually control, actually control, actually control, so don't put MIG so don't put MIG fragments from an fragments from an fragments from an untrusted untrusted untrusted tenant next to a tenant next to a tenant next to a sensitive model. sensitive model. Second, be aware of Second, be aware of Second, be aware of compromises. MIG compromises. MIG compromises. MIG eliminates the connection between eliminates the connection between eliminates the connection between nodes, which means nodes, which means nodes, which means there is no there is no there is no tensor tensor tensor parallelism on MIG. parallelism on MIG. parallelism on MIG. Sharding over NVLink, Sharding over NVLink, Sharding over NVLink, on the other hand, limits on the other hand, limits on the other hand, limits leaks to only time leaks to only time leaks to only time parameters, not parameters, not parameters, not level data. level data. Third, keep an eye on the Third, keep an eye on the Third, keep an eye on the GPU itself. GPU itself.

GPU itself. Ensure that the Ensure that the Ensure that the host infrastructure host infrastructure host infrastructure actively checks for actively checks for malicious malicious tasks running on the hardware. This is exactly what tasks running on the hardware. This is exactly what tasks running on the hardware. This is exactly what Stealthy does, Stealthy does, Stealthy does, detecting such detecting such detecting such requests directly requests directly requests directly from kernel behavior. from kernel behavior. Sharing Sharing GPUs means that only GPUs means that only GPUs means that only runtime can runtime can runtime can deliver model, deliver model, deliver model, speed, and speed, and speed, and workload. This is workload. This is workload. This is true even for NVLink true even for NVLink , without the need for , without the need for any exploit. Stealthy any exploit. Stealthy any exploit. Stealthy sees this sees this sees this happening in happening in happening in real real real time. Most time. Most time. Most organizations still don't organizations still don't organizations still don't have this level of have this level of have this level of visibility. visibility. visibility. Thanks again to the researchers Thanks again to the researchers Thanks again to the researchers at Behind Bars. This entire at Behind Bars. This entire at Behind Bars. This entire report is based report is based report is based on their work. on their work. on their work. Thank you all for Thank you all for Thank you all for watching.