elasticai.creator.nn.quantized_grads.base_modules.linear#

Module Contents#

Classes#

Linear

A linear layer. The weights and bias are fake quantized during initialization. Make sure that math_ops is a module where all needed tensors are part of it, so they can be moved to the same device. Make sure that weight_quantization and bias_quantization are modules that implement the forward function. If you want to quantize during initialization or only apply quantized updates make sure to use a quantized optimizer and implement the right_inverse method for your module.

API#

class elasticai.creator.nn.quantized_grads.base_modules.linear.Linear(math_ops: torch.nn.Module, in_features: int, out_features: int, weight_quantization: torch.nn.Module, bias: bool, bias_quantization: torch.nn.Module = None, device: Any = None, dtype: Any = None)[source]#

Bases: torch.nn.Linear

A linear layer. The weights and bias are fake quantized during initialization. Make sure that math_ops is a module where all needed tensors are part of it, so they can be moved to the same device. Make sure that weight_quantization and bias_quantization are modules that implement the forward function. If you want to quantize during initialization or only apply quantized updates make sure to use a quantized optimizer and implement the right_inverse method for your module.

Initialization

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x: torch.Tensor) torch.Tensor[source]#