# 前向操作符重载自动微分实现

## 前向操作符重载自动微分实现

### 前向自动微分原理

• 分解程序为一系列已知微分规则的基础表达式的组合
• 根据已知微分规则给出各基础表达式的微分结果
• 根据基础表达式间的数据依赖关系，使用链式法则将微分结果组合完成程序的微分结果

• 分解程序为一系列已知微分规则的基础表达式组合，并使用高级语言的重载操作
• 在重载运算操作的过程中，根据已知微分规则给出各基础表达式的微分结果
• 根据基础表达式间的数据依赖关系，使用链式法则将微分结果组合完成程序的微分结果

### 具体实现

``````import numpy as np
``````

``````class ADTangent:

# 自变量 x，对自变量进行求导得到的 dx
def __init__(self, x, dx):
self.x = x
self.dx = dx

# 重载 str 是为了方便打印的时候，看到输入的值和求导后的值
def __str__(self):
context = f'value:{self.x:.4f}, grad:{self.dx}'
return context
``````

``````    def __add__(self, other):
if isinstance(other, ADTangent):
x = self.x + other.x
dx = self.dx + other.dx
elif isinstance(other, float):
x = self.x + other
dx = self.dx
else:
return NotImplementedError
return ADTangent(x, dx)
``````

``````    def __sub__(self, other):
if isinstance(other, ADTangent):
x = self.x - other.x
dx = self.dx - other.dx
elif isinstance(other, float):
x = self.x - other
ex = self.dx
else:
return NotImplementedError
return ADTangent(x, dx)

def __mul__(self, other):
if isinstance(other, ADTangent):
x = self.x * other.x
dx = self.x * other.dx + self.dx * other.x
elif isinstance(other, float):
x = self.x * other
dx = self.dx * other
else:
return NotImplementedError
return ADTangent(x, dx)

def log(self):
x = np.log(self.x)
dx = 1 / self.x * self.dx
return ADTangent(x, dx)

def sin(self):
x = np.sin(self.x)
dx = self.dx * np.cos(self.x)
return ADTangent(x, dx)
``````

``````x = ADTangent(x=2., dx=1)
y = ADTangent(x=5., dx=0)
f = ADTangent.log(x) + x * y - ADTangent.sin(y)
print(f)

value:11.6521, grad:5.5
``````

Pytroch 对公式1的自动微分结果：

``````import torch
from torch.autograd import Variable

x = Variable(torch.Tensor([2.]), requires_grad=True)
y = Variable(torch.Tensor([5.]), requires_grad=True)
f = torch.log(x) + x * y - torch.sin(y)
f.backward()

print(f)
print(x.grad)
print(y.grad)
``````

``````tensor([11.6521], grad_fn=<SubBackward0>)
tensor([5.5000])
tensor([1.7163])
``````

MindSpore 对公式1的自动微分结果：

``````import numpy as np
import mindspore.nn as nn
from mindspore import Parameter, Tensor

class Fun(nn.Cell):
def __init__(self):
super(Fun, self).__init__()

def construct(self, x, y):
f = ops.log(x) + x * y - ops.sin(y)
return f

x = Tensor(np.array([2.], np.float32))
y = Tensor(np.array([5.], np.float32))
f = Fun()(x, y)

grad_all = ops.GradOperation()
grad = grad_all(Fun())(x, y)

print(f)
print(grad[0])

``````

``````[11.65207]
5.5``````
原文作者：ZOMI酱酱
原文地址: https://www.cnblogs.com/ZOMI/p/16314760.html
本文转自网络文章，转载此文章仅为分享知识，如有侵权，请联系博主进行删除。