Miner cudaerror
using handle in GPU high performance programming_ Error macro function to process cudaerror
static void handleerror (cudaerror) returned by each function_ t err,const char *file,int line ) {
if (err != cudaSuccess) {
printf( "% s in %s at line %d\ n", cudaGetErrorString( err ), file, line );< br />exit( EXIT_ FAILURE );< br />}
}
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
t="code" l="cpp">/
#definegetLastCudaError(msg)_getLastCudaError(msg,_FILE_,_LINE_)
inlinevoid__getLastCudaError(constchar*errorMessage,constchar*file,constintline)
{
cudaError_terr=cudaGetLastError();< br />
if(cudaSuccess!= err)
{
fprintf(stderr,"%s(%i):getLastCudaError()CUDAerror:%s:(%d)%s.
",
file,line,errorMessage,(int)err,cudaGetErrorString(err));< br />DEVICE_RESET
exit(EXIT_FAILURE);< br />}
}
#endif
kernel<& <> lt; 1.1>> gt;& gt;();< br />getLastCudaError("ErrorinCalling'kernel'"); pre>
error type: CUDA_ ERROE_ OUT_ OF_ MEMORY
E tensorflow/stream_ executor/cuda/cuda_ driver.cc:924] failed to alloc 17179869184 bytes on host: CUDA_ ERROR_ OUT_ OF_ MEMORY
W ./tensorflow/core/common_ runtime/gpu/pool_ Allocator. H: 195] could not allocate pinned host memory of size: 17179869184
killed
in fact, it is very easy to understand, which roughly means that the GPU size of the server is m
tensorflow, only n (n & lt; M)
in other words, tensorflow tells you that you can't apply for all the resources of GPU, and then quit.
solution:
find the session in the code
Add
config = tf.configproto (allow) before the session definition_ soft_ Placement = true)
# accounting for 70% of GPU resources at most
GPU_ options = tf.GPUOptions(per_ process_ gpu_ memory_ (fraction = 0.7)
# instead of giving tensorflow all GPU resources at first, it will increase
config.gpu on demand_ options.allow_ Growth = true
sess = TF. Session (config = config)
this is no problem
in fact, tensorflow is a greedy tool
even if you use device_ ID specifies that the GPU will also occupy the video memory resources of other GPUs. You must
execute export CUDA before executing the program_ VISIBLE_ Devices = n (n is the visible server number)
to execute Python code. Py will not occupy other GPU resources
recently, before tensorflow, it was Cafe
this week, for three consecutive days, people in the laboratory reported that they occupied too much server resources. It's really tiring. Just use the above method
that is, to execute export CUDA before executing code_ VISIBLE_ Devices = n
only one or a few GPUs can be seen, and other GPUs can't be seen
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
t="code" l="cpp">/
#definegetLastCudaError(msg)_getLastCudaError(msg,_FILE_,_LINE_)
inlinevoid__getLastCudaError(constchar*errorMessage,constchar*file,constintline)
{
cudaError_terr=cudaGetLastError();< br />
if(cudaSuccess!= err)
{
fprintf(stderr,"% s(%i):getLastCudaError()CUDAerror:%s:(%d)%s.
",< br />file,line,errorMessage,(int)err,cudaGetErrorString(err));< br />DEVICE_RESET
exit(EXIT_FAILURE);< br />}
}
#endif
kernel<& <> lt; 1.1>> gt;& gt;();< br />getLastCudaError(" ErrorinCalling kernel'& quot;); pre>