HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of t
阿新 • • 發佈:2018-12-22
今天執行theano程式的時候,遇到了下面的問題:
GRU4Rec git:(master) ✗ python run_rsc15.py Using cuDNN version 6021 on context None Mapped name None to device cuda: GeForce GTX 1080 Ti (0000:03:00.0) start training epoch: 0 Traceback (most recent call last): File "/home/eric/anaconda3/lib/python3.6/site-packages/theano/compile/function_module.py", line 903, in __call__ self.fn() if output_subset is None else\ IndexError: Index out of bounds. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "run_rsc15.py", line 35, in <module> gru.fit(data) File "/home/eric/Documents/Experiments/GRU4Rec/gru4rec.py", line 575, in fit cost = train_function(in_idx, y, len(iters), reset) File "/home/eric/anaconda3/lib/python3.6/site-packages/theano/compile/function_module.py", line 917, in __call__ storage_map=getattr(self.fn, 'storage_map', None)) File "/home/eric/anaconda3/lib/python3.6/site-packages/theano/gof/link.py", line 325, in raise_with_op reraise(exc_type, exc_value, exc_trace) File "/home/eric/anaconda3/lib/python3.6/site-packages/six.py", line 692, in reraise raise value.with_traceback(tb) File "/home/eric/anaconda3/lib/python3.6/site-packages/theano/compile/function_module.py", line 903, in __call__ self.fn() if output_subset is None else\ IndexError: Index out of bounds. Apply node that caused the error: GpuAdvancedSubtensor1(<GpuArrayType<None>(float32, matrix)>, GpuContiguous.0) Toposort index: 29 Inputs types: [GpuArrayType<None>(float32, matrix), GpuArrayType<None>(int64, vector)] Inputs shapes: [(68892, 100), (2080,)] Inputs strides: [(400, 4), (8,)] Inputs values: ['not shown', 'not shown'] Outputs clients: [[GpuElemwise{Composite{((i0 * i1) - (i2 * (i3 / sqrt((i4 + i5 + i6)))))}}[(0, 1)]<gpuarray>(GpuArrayConstant{[[0.3]]}, GpuAdvancedSubtensor1.0, GpuArrayConstant{[[0.05]]}, GpuDot22.0, GpuArrayConstant{[[1.e-06]]}, GpuAdvancedSubtensor1.0, GpuElemwise{sqr,no_inplace}.0)]] Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer): File "run_rsc15.py", line 35, in <module> gru.fit(data) File "/home/eric/Documents/Experiments/GRU4Rec/gru4rec.py", line 532, in fit updates = self.RMSprop(cost, params, full_params, sparams, sidxs) File "/home/eric/Documents/Experiments/GRU4Rec/gru4rec.py", line 408, in RMSprop vs = velocity[sample_idx] File "/home/eric/anaconda3/lib/python3.6/site-packages/theano/gpuarray/type.py", line 675, in __getitem__ return _operators.__getitem__(self, *args) File "run_rsc15.py", line 35, in <module> gru.fit(data) File "/home/eric/Documents/Experiments/GRU4Rec/gru4rec.py", line 532, in fit updates = self.RMSprop(cost, params, full_params, sparams, sidxs) File "/home/eric/Documents/Experiments/GRU4Rec/gru4rec.py", line 408, in RMSprop vs = velocity[sample_idx] File "/home/eric/anaconda3/lib/python3.6/site-packages/theano/gpuarray/type.py", line 675, in __getitem__ return _operators.__getitem__(self, *args) HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
我開始的theano的版本為:
Theano 1.0.2+2.gc449c8699
然後我把版本降低到0.9之後,就執行正常了:
conda install theano=0.9
程式正常跑起來的日誌:
GRU4Rec git:(master) ✗ python run_rsc15.py /home/eric/anaconda3/lib/python3.6/site-packages/theano/gpuarray/dnn.py:135: UserWarning: Your cuDNN version is more recent than Theano. If you encounter problems, try updating Theano or downgrading cuDNN to version 5.1. warnings.warn("Your cuDNN version is more recent than " Using cuDNN version 6021 on context None Mapped name None to device cuda: GeForce GTX 1080 Ti (0000:03:00.0) start training epoch: 0 Epoch0 loss: 0.515304 epoch: 1 Epoch1 loss: 0.356552 epoch: 2 Epoch2 loss: 0.273438 epoch: 3 Epoch3 loss: 0.232888 epoch: 4 Epoch4 loss: 0.211045 epoch: 5 Epoch5 loss: 0.198076 epoch: 6 Epoch6 loss: 0.189473 epoch: 7 Epoch7 loss: 0.183521 epoch: 8 Epoch8 loss: 0.179129 epoch: 9 Epoch9 loss: 0.175793 epoch: 10 Epoch10 loss: 0.173115 epoch: 11 Epoch11 loss: 0.170898 epoch: 12 Epoch12 loss: 0.169162