Can't write to memory after doing malloc - Invalid __global__ write

yugiarora
Sun, 15 Mar 2026 23:35:25 +0000
Compute Sanitizer

I’m trying to malloc on device on multiple threads, then trying to write to one of them, (I’ve also tried writing to all of them, I have taken care of expanding my heap limit, etc.) and for some reason, the write seems to fail with memcheck giving me an error

[MarkAccess] malloc for L2 table returned 0x226d16760
[MarkAccess] malloc for L2 table returned 0x226d178a0
[MarkAccess] malloc for L2 table returned 0x226d189e0
[MarkAccess] malloc for L2 table returned 0x226d19b20
[MarkAccess] malloc for L2 table returned 0x226d15620
[MarkAccess] allocated L2 table for L1 index 1887 at 0x226d16760

========= Invalid __global__ write of size 1 bytes
=========     at 0xc30 in stride_access(int *, int)
=========     by thread (0,0,0) in block (0,0,0)
=========     Address 0x226d16760 is out of bounds
=========     and is inside the nearest allocation at 0x206e00000 of size 536871168 bytes
=========     Saved host backtrace up to driver entry point at kernel launch time
=========     Host Frame:cuLaunchKernel [0x39df25]
=========                in /usr/lib/x86_64-linux-gnu/libcuda.so.1
=========     Host Frame:__cudart819 [0x12dbb]
=========                in /data/./a.out
=========     Host Frame:cudaLaunchKernel [0x700c8]
=========                in /data/./a.out
=========     Host Frame:/usr/local/cuda/bin/../targets/x86_64-linux/include/cuda_runtime.h:215:cudaError cudaLaunchKernel<char>(char const*, dim3, dim3, void**, unsigned long, CUstream_st*) [0xc780]

The address 0x226d15770 is the same address that I’ve just been returned by malloc, I’ve tried tinkering with it for hours and cannot seem to digest why this is happening. The error seems to happen at the memset line. I’ve tried using new_l2[0] = 0 instead, I’ve tried reducing the malloc size, I’ve tried running only 2 kernels, none of these solutions help, even though very clearly, I am trying to write to the memory I have just allocated. Also, my program completes execution when I am not using compute-sanitizer but the output’s not what I am expecting.

Here’s the relevant part of the code

void** new_l2 = (void**)malloc(4096);
            LOG("[MarkAccess] malloc for L2 table returned %p\n", new_l2);
            if (!new_l2) return;
            // Zero-init
            LOG("[MarkAccess] zero-initialising new L2 table at %p\n", new_l2);
            // Atomic CAS: only one thread wins, losers free their allocation
            unsigned long long old = atomicCAS(
                (unsigned long long*)&shadow_l1[l1_idx],
                0ULL,
                (unsigned long long)new_l2
            );
            // Clearing in case of race condition
            if (old != 0ULL){
                free(new_l2);  // another thread already inserted
            }
            else{
                LOG("[MarkAccess] allocated L2 table for L1 index %u at %p\n", l1_idx, new_l2);
                memset(new_l2, 0, 4096); // zero-init the new L2 table
            }

Here’s the whole code:
Full Code (click for more details)
2 posts - 2 participants