Wsrtjtyk

In pytorch, to update the model, should I use optimizer.step() or model.step() ?

Here is a example snippet:

import torch

import torch nn

class SomeNeuralNet(nn.Module):

    def __init__(self,hs,es,dropout):

          SomeNeuralNet(ClaimRecognizer, self).__init__()

          # Some initialization here

    def forward(x):

          # forward propagation here



model = SomeNeuralNet(es,hs,dp)

optimizer = optim.Adam(model.parameters())

loss_function = nn.NLLLoss()

for epoch in N:

   for x in data:

      logp = model(x)

      loss = loss_function(logp,gold_outs)

      loss.backward()

       # Which one I should call ? Optimizer.step() or model.step() or both ?

       optimizer.step()

       model.step()

To make the gradient descent step, you normally use just optimizer.step().
Here is also an example taken from the documentation (same link at bottom), what it looks like in general:

for input, target in dataset:

    optimizer.zero_grad()

    output = model(input)

    loss = loss_fn(output, target)

    loss.backward()

    optimizer.step()

I don't know where you got this model.step()? Does did you try it?

If your model really possesses some kind of step()-functionality, it probably does something different.

But unless you define something extra, your model gets its functions from nn.Module and this does not have step function!

See this example from the the Pytorch Documentation:

import torch.nn as nn

import torch.nn.functional as F



class Model(nn.Module):

    def __init__(self):

        super(Model, self).__init__()

        self.conv1 = nn.Conv2d(1, 20, 5)

        self.conv2 = nn.Conv2d(20, 20, 5)



    def forward(self, x):

        x = F.relu(self.conv1(x))

        return F.relu(self.conv2(x))



model = Model()

model.step()

Trying to call step() result in an AttributeError:

---------------------------------------------------------------------------

AttributeError                            Traceback (most recent call last)

<ipython-input-41-b032813f7eda> in <module>

     13 

     14 model = Model()

---> 15 model.step()



~/miniconda3/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py in __getattr__(self, name)

    530                 return modules[name]

    531         raise AttributeError("'{}' object has no attribute '{}'".format(

--> 532             type(self).__name__, name))

    533 

    534     def __setattr__(self, name, value):



AttributeError: 'Model' object has no attribute 'step'

To sum it up, normally your model should not have a step() function, optimizer.step() is the way to go if you want to do the optimization step.

See also here:
https://pytorch.org/docs/stable/optim.html#taking-an-optimization-step