I hope I have described my problem clearly now. Every call to this function, the first cudamalloc taking 20ms is making this function less competent.
#Cuda driver api free
At first in this function, it’s a thrust call, but I later found it’s taking excessive time, and then try to add a cudaMalloc and cudafree at the very beginning, and found malloc and free could even taking 20-40ms (then thrust time gets back to normal), but whole function is only taking 50-60ms. Inside this separate function is called in another function in the main program which is in a loop. The actually program I use is I built a separate function, compiled to a static lib and link to the main. Here I am just allocating 4 bytes, seems should not run out of memory and look for another layer.Īnyway, this is just demonstration showing the overhead. I know that is not always possible, but allocating and freeing GPU memory is fairly expensive so should happen infrequently.įWIW, I would not call a 6 millisecond delay “great overhead”. Ideally what you would want is setting up necessary allocations prior to the loop, re-using those allocations throughout the loop, and the freeing allocated memory at the end. Have you actually tried a loop? What timing did you observe for the cudaMalloc() in it? Note that loops that continuously allocate and free memory are not advised from a performance perspective. This is similar to the behavior of malloc() in the C\C++ runtime library: When it runs out of memory it needs to go back to the operating system for the next large chunk, which is an expensive operation time-wise. This should not happen all the time, only occasionally when the runtime allocator runs out of allocated memory and needs to retrieve the next chunk from the next lower layer.
#Cuda driver api software
This looks like CUDA runtime initialization overhead to me, probably the memory allocator inside the CUDA runtime getting an initial chunk of memory from the allocator in the software layer below. If this is in a loop, the overhead is significant. The first cudaMalloc takes excessive time.
#Cuda driver api driver
I also check its context, the runtime context is the same to driver api context, so there is no new context creation involved, I am using the chrono to profile the time.Ī simple illustration can be described below:ĬuCtxGetCurrent(¤t_ctx) //driver current_ctx=0x5641b9aac1a0ĬheckCudaErrors(cuMemAlloc(&d_C1, 4)) //driver api call time: 6.724us, normalĬuCtxGetCurrent(¤t_ctx) //driver current_ctx=0x5641b9aac1a0, same as aboveĬudaMalloc(&d_D, 4) //runtime api call time: 5896.57us, great overheadĬuCtxGetCurrent(¤t_ctx) //runtime current_ctx1=0x5641b9aac1a0 same as aboveĬudaMalloc(&d_E, 4) //runtime call2 time: 10.655us, this time the overhead is normal cudamalloc, is taking excessive time, as this function is called many many times, the excessive overhead is bothering its performance. However when I am profiling the external lib, I found the first call to runtime api call, i.e. I was trying to add a cuda external function call (with a lot of runtime api call) to a program which only involves cuda driver api call, using the separate linkage described in, basically compile them into static lib and link together to the program
0 kommentar(er)
