elasticai.creator.nn.quantized_grads.base_modules.linear
#
Module Contents#
Classes#
A linear layer. The weights and bias are fake quantized during initialization. Make sure that math_ops is a module where all needed tensors are part of it, so they can be moved to the same device. Make sure that weight_quantization and bias_quantization are modules that implement the forward function. If you want to quantize during initialization or only apply quantized updates make sure to use a quantized optimizer and implement the right_inverse method for your module. |
API#
- class elasticai.creator.nn.quantized_grads.base_modules.linear.Linear(math_ops: torch.nn.Module, in_features: int, out_features: int, weight_quantization: torch.nn.Module, bias: bool, bias_quantization: torch.nn.Module = None, device: Any = None, dtype: Any = None)[source]#
Bases:
torch.nn.Linear
A linear layer. The weights and bias are fake quantized during initialization. Make sure that math_ops is a module where all needed tensors are part of it, so they can be moved to the same device. Make sure that weight_quantization and bias_quantization are modules that implement the forward function. If you want to quantize during initialization or only apply quantized updates make sure to use a quantized optimizer and implement the right_inverse method for your module.
Initialization
Initialize internal Module state, shared by both nn.Module and ScriptModule.