afnio.cognitive.parameter#

Classes

Parameter([data, role, requires_grad])

A subclass of Variable that represents learnable parameters (similar to nn.Parameter).
class afnio.cognitive.parameter.Parameter(data=None, role=None, requires_grad=True)[source]#

Bases: Variable

A subclass of Variable that represents learnable parameters (similar to nn.Parameter). These parameters are typically text-based, learnable weights, embeddings, etc.

append_grad(gradient)#

Appends a gradient value to the list .grad for this variable.

backward(gradient=None, retain_graph=None, create_graph=False, inputs=None)#

Computes the gradient of current variable wrt graph leaves.

The graph is differentiated using the chain rule. If the variable is non-scalar (i.e. its data has more than one element) and requires gradient, the function additionally requires specifying a gradient. It should be a variable with data of matching type and shape, that represents the gradient of the differentiated function w.r.t. self.

This function accumulates gradients in the leaves - you might need to zero .grad attributes or set them to None before calling it.

Note

When inputs are provided, each input must be a leaf variable. If any input is not a leaf, a RuntimeError is raised.

Parameters:

  • gradient (Variable, optional) – The gradient of the function being differentiated w.r.t. self. This argument can be omitted if self is a scalar.

  • retain_graph (bool, optional) – If False, the graph used to compute the grads will be freed. Setting this to True retains the graph, allowing for additional backward calls on the same graph, useful for example for multi-task learning where you have multiple losses. However, retaining the graph is not needed in nearly all cases and can be worked around in a much more efficient way. Defaults to the value of create_graph.

  • create_graph (bool, optional) – If True, graph of the derivative will be constructed, allowing to compute higher order derivative products. Defaults to False.

  • inputs (sequence of Variable, optional) – Inputs w.r.t. which the gradient will be accumulated into .grad. All other variables will be ignored. If not provided, the gradient is accumulated into all the leaf Variables that were used to compute the variables.

copy_(src)#

Copies the data from the source Variable into this Variable.

Parameters:

src (Variable) – The source Variable to copy from.

Returns:

The current Variable with updated data, role and requires_grad.

Return type:

self

Raises:

  • TypeError – If the source is not a Variable.

  • ValueError – If the source data type does not match the target data type.

property data#
detach()#

Returns a new Variable, detached from the computation graph. This new Variable will not have a grad_fn and will not track gradients.

property grad: Variable | None#
property grad_fn: Node | None#
is_floating_point()#

Checks if the Variable’s data contains floating-point values.

Returns:

True if the data is a floating-point type (either scalar or

all elements in a list/tuple are floating-point).

Return type:

bool

is_leaf: bool#

All Variables that have requires_grad which is False will be leaf Variables by convention.

For Variables that have requires_grad which is True, they will be leaf Variables if they were created by the user. This means that they are not the result of an operation and so grad_fn is None.

Only leaf Variables will have their grad populated during a call to backward(). To get grad populated for non-leaf Variables, you can use retain_grad().

Example:

>>> a = hf.Variable("abc", requires_grad=True)
>>> a.is_leaf
True
>>> b = hf.Variable("abc", requires_grad=True).upper()
>>> b.is_leaf
False
# b was created by the operation that converts all string characters to uppercase
>>> c = hf.Variable("abc", requires_grad=True) + "def"
>>> c.is_leaf
False
# c was created by the addition operation
>>> d = hf.Variable("abc").upper()
>>> d.is_leaf
True
# d does not require gradients and so has no operation creating it (that is tracked by the autodiff engine)
>>> e = hf.Variable("abc").upper().requires_grad_()
>>> e.is_leaf
True
# e requires gradients and has no operations creating it
property output_nr: int#
requires_grad: bool#
requires_grad_(mode=True)#

Change if autodiff should record operations on this variable: sets this variable’s requires_grad attribute in-place. Returns this variable.

requires_grad_()’s main use case is to tell autodiff to begin recording operations on a Variable variable. If variable has requires_grad=False (because it was obtained through a DataLoader, or required preprocessing or initialization), variable.requires_grad_() makes it so that autodiff will begin to record operations on variable.

Parameters:

requires_grad (bool) – If autodiff should record operations on this variable. Default: True.

Example

>>> # Initialize with requires_grad=False for data preprocessing
>>> x = hf.Variable(data="abc", role="input")
>>> x = preprocess(x)  # Preprocess without gradient tracking
>>> x
variable(abc, role=input, requires_grad=False)

>>> # Now enable requires_grad for backpropagation
>>> x.requires_grad_()
>>> output = model(x)
>>> output.backward()  # Backpropagation through `x`
>>> x.grad
variable(ABC, role=input, requires_grad=True)
retain_grad()#

Enable gradient retention for non-leaf variables.

to(dtype=None)#

Cast the data of the Variable to the specified dtype.

Parameters:

dtype (Optional[type]) – The target type to cast the data (e.g., float, int, str).

Returns:

A new Variable with data cast to the target dtype.

Return type:

Variable

variable_id: Optional[str]#