• 檢查

a = torch.tensor(...)
a.dtype # float32
a.type() # torch.FloatTensor
a.device

Data type

• float/ double/ half

  • Error: Expected Dtype Float but found double

  https://github.com/pytorch/pytorch/issues/2138

• 發生問題處

  • 卷積層
  • backward的時候

• 解決

  • .float() vs .double()

    • .float() ⇒ torch.FloatTensor ⇒ dtype = float32
    • .double() ⇒ torch.DoubleTensor ⇒ dtype = float64
      • 比較準，但比較慢算量比較大

  • in dataset

    X = torch.from_numpy(X).float()
    y = torch.from_numpy(y).float()
    

  • in train function

    model.float()
    

Device Type

• Error: tow devices cuda and cpu

• 出現問題處

  • 模型

  • 新加的模型層

  • 資料

  • save dict完以後

    • 要重新 .to(device)

  • torch.load ⇒ optimizer裡面的 param_groups

    • key: model params
    • value: optimizer params
      • eg. adam 的 running mean

    checkpoint = torch.load(PATH, map_location=opt.device)
    optimizer = checkpoint['optimizer_state_dict']
    for param_group in optimizer.param_groups:
        for K, V in param_group.items():
            if isinstance(v, torch.Tensor):
                print(k)
                print(v.type())
                print(v.device)
    
    for K, V in optimizer.state.items():
        for k, v in V.items():
            if isinstance(v, torch.Tensor):
                # V[k] = v.to(opt.device)
                print(V[k].device, V[k].type())
    

• 解決

  • .cuda() / to(device)
  • .cpu() / to(device)

• torch.cuda.FloatTensor vs torch.FloatTensor

Data Value

• black image values == 0 無法學習
• float() 精度太小

Loss Function

1. CrossEntropyLoss, NLLLoss
   • y_true 要是標籤的 index
     • [0.1, 0.6, 0.3] , [2]
     • [0.1, 0.2, 0.7] , [3]
2. BCELoss
   • y_true 只能是 0 or 1