navigation links

activations module

Activation functions Element-wise transformations without trainable parameters

class BiLU(alpha=1.0, delta=0.9)

Bases: torch.nn.modules.module.Module

Bijective linear unit f(x) = alpha + delta*sign(x)

Initializes internal Module state, shared by both nn.Module and ScriptModule.

forward(input)

Output of the BiLU activation function

training: bool

class NormBiTanh(alpha=0.3)

Bases: torch.nn.modules.module.Module

Bijective Normalized Tanh layer f(x) = alpha*Tanh + (1-alpha)*x activation

Initializes internal Module state, shared by both nn.Module and ScriptModule.

forward(input)

Output of the NormBiTanh activation function

training: bool