AvgPool

AvgPool  // Average Pooling operation

Description

Average pooling windows its input and produces an average for each window.

Inputs

Name Element Type Shape Notes
data Any \((N,C,d_1,\ldots,d_n)\) \(n>0, d_i>0\)

Attributes

Name Type Notes
w Shape[n] Window shape. \(w_i\le d_i\)
s Strides[n] Window strides.
p Shape[n] Padding below.
q Shape[n] Padding above.
i Boolean Include padding in average.

Outputs

Name Element Type Shape
output Any \((N,C,d'_1,\ldots,d'_n)\)

Average pooling takes as its input, a batch tensor data of shape \((N,C,d_1,\ldots,d_n)\), where where \(N\) is the batch size, and \(C > 0\) is the number of channels (sometimes called features). The dimensions \((d_1,\ldots,d_n)\) correspond to the shape of an \(n\)-dimensional data item in a batch. For example, where \(n=2\), the data may represent a two-dimensional image. It also takes four attributes:

  1. window shape,
  2. window movement strides, (optional)
  3. padding below, (optional)
  4. padding above, (optional)
  5. include padding in average

The shape of output is \((N,C,d'_1,\ldots,d'_n)\), where \(d'_n = \lceil \frac{p_i + d_i + q_i - w_i + 1}{s_i} \rceil\).

Informal definition: If \(\textit{i}\) is \(\textit{true}\), then averages are computed as though the padding region contained regular elements of value zero. If \(\textit{i}\) is \(\textit{false}\), then averages are computed using only the non-padding tensor elements that are present in each window.

Example: Consider two instances of this operator with the following attributes: \(\textit{w} = (2,2)\), \(\textit{s} = (1,1)\), \(\textit{p} = (1,1)\), and (in one instance) \(\textit{i} = false\) or (in the other instance) \(\textit{i} = true\).

Consider how those two operator instances would handle this input tensor:

\[\begin{split}T_\textit{in} = \begin{bmatrix} 1 & 3 & 5 & \ldots \\ 7 & 11 & 13 & \ldots \\ 17 & 19 & 23 & \ldots \\ \vdots & \vdots & \vdots & \ddots \end{bmatrix}\end{split}\]

Applying the padding indicated by the value of \(\textit{p}\), we have the padded image of \(T_\textit{in}\) as follows:

\[\begin{split}T_\textit{in,padded} = \begin{bmatrix} (0) & (0) & (0) & (0) & \ldots \\ (0) & 1 & 3 & 5 & \ldots \\ (0) & 7 & 11 & 13 & \ldots \\ (0) & 17 & 19 & 23 & \ldots \\ (0) & \vdots & \vdots & \vdots & \ddots \end{bmatrix}\end{split}\]

Now consider how the two variations of this example’s AvgPool operator will compute the “average” value of the top-left window, which contains exactly the elements:

\[\begin{split}\begin{bmatrix} (0) & (0) \\ (0) & 1 \end{bmatrix}\end{split}\]

If \(\textit{i} = false\), then the operator simply ignores the padding elements. It therefore computes the average of the single-element set \(\{ 1 \}\), yielding \(1.0\).

If \(\textit{i} = true\), then the operator computes the average of the set \(\{ 0, 0, 0, 1\}\), yielding 0.25.

Note: This operator is ill-defined when both of the following conditions hold: (1) \(\textit{i} = false\), and (2) the operator’s other attribute values indicate that at least one window will contain only padding elements.

Formal definition: In the absence of padding, given an input data batch tensor \(T_\textit{in}\), the output tensor is defined by the equation

\[T_\textit{out}[a,c,i_1,\ldots,i_n] = \frac{\sum_{j_1 = s_1 i_1, \ldots, j_n = s_n i_n}^{j_1 = s_1 i_1 + w_1 - 1, \ldots, j_n = s_n i_n + w_n - 1} T_\textit{in}[a,c,j_1,\ldots,j_n]}{\prod_{i=1}^n{w_n}}\]

In the presence of padding, we do not always want to divide by a reciprocal equal to the number of elements in the window, since some of the output points are determined by a window that is partly hanging beyond the edge of the tensor. In this case we can define the output

In this case we can define the output via a few intermediate steps.

First define the sum tensor \(T_\textit{sum}\), with shape \((N,C,d'_1,\ldots,d'_n)\), as follows.

\[T_\textit{sum}[a,c,i_1,\ldots,i_n] = \frac{\sum_{j_1 = s_1 i_1, \ldots, j_n = s_n i_n}^{j_1 = s_1 i_1 + w_1 - 1, \ldots, j_n = s_n i_n + w_n - 1} \textit{val}[a,c,j_1,\ldots,j_n]}{\prod_{i=1}^n{w_n}}\]

where

\[\begin{split}\textit{val}[a,c,j_1,\ldots,j_n] = \begin{cases} T_\textit{in}[a,c,j_1,\ldots,j_n]&\text{if for all } k, p_k \le j_k < p_k + d_k\\ 0&\text{otherwise}. \end{cases}\end{split}\]

Second, define the divisor tensor \(T_\textit{div}\), with shape \((N,C,d'_1,\ldots,d'_n)\), as follows.

\[T_\textit{div}[a,c,i_1,\ldots,i_n] = \frac{\sum_{j_1 = s_1 i_1, \ldots, j_n = s_n i_n}^{j_1 = s_1 i_1 + w_1 - 1, \ldots, j_n = s_n i_n + w_n - 1} \textit{val}[a,c,j_1,\ldots,j_n]}{\prod_{i=1}^n{w_n}}\]

where

\[\begin{split}\textit{val}[a,c,j_1,\ldots,j_n] = \begin{cases} 1&\text{if for all }k, p_k \le j_k < p_k + d_k\\ 0&\text{otherwise}. \end{cases}\end{split}\]

Finally, define \(T_\textit{out}\) as the result of elementwise dividing \(T_\textit{sum}\) by \(T_\textit{div}\). Note that at positions where \(T_\textit{div}\) is zero, values may be infinity or nan. (This corresponds to a condition where the pooling window is completely out of bounds, encompassing no valid values.)

Backprop

C++ Interface

class AvgPool : public ngraph::op::Op

Batched average pooling operation, with optional padding and window stride.

Public Functions

const NodeTypeInfo &get_type_info() const

Returns the NodeTypeInfo for the node’s class. During transition to type_info, returns a dummy type_info for Node if the class has not been updated yet.

AvgPool()

Constructs a batched average pooling operation.

AvgPool(const Output<Node> &arg, const Shape &window_shape, const Strides &window_movement_strides, const Shape &padding_below, const Shape &padding_above, bool include_padding_in_avg_computation, const PadType &pad_type, bool ceil_mode)

Constructs a batched average pooling operation.

Parameters
  • arg: The output producing the input data batch tensor.[d1, dn]
  • window_shape: The window shape.[n]
  • window_movement_strides: The window movement strides.[n]
  • padding_below: The below-padding shape.[n]
  • padding_above: The above-padding shape.[n]
  • include_padding_in_avg_computation: If true then averages include padding elements, each treated as the number zero. If false, padding elements are entirely ignored when computing averages.
  • pad_type: Padding type to use for additional padded dimensions
  • ceil_mode: Whether to use ceiling while computing output shape.

AvgPool(const Output<Node> &arg, const Shape &window_shape, const Strides &window_movement_strides, const Shape &padding_below, const Shape &padding_above, bool include_padding_in_avg_computation, const PadType &pad_type)

Constructs a batched average pooling operation.

Parameters
  • arg: The output producing the input data batch tensor.[d1, dn]
  • window_shape: The window shape.[n]
  • window_movement_strides: The window movement strides.[n]
  • padding_below: The below-padding shape.[n]
  • padding_above: The above-padding shape.[n]
  • include_padding_in_avg_computation: If true then averages include padding elements, each treated as the number zero. If false, padding elements are entirely ignored when computing averages.
  • pad_type: Padding type to use for additional padded dimensions

AvgPool(const Output<Node> &arg, const Shape &window_shape, const Strides &window_movement_strides, const Shape &padding_below, const Shape &padding_above, bool include_padding_in_avg_computation = false)

Constructs a batched average pooling operation.

Parameters
  • arg: The output producing the input data batch tensor.[d1, dn]
  • window_shape: The window shape.[n]
  • window_movement_strides: The window movement strides.[n]
  • padding_below: The below-padding shape.[n]
  • padding_above: The above-padding shape.[n]
  • include_padding_in_avg_computation: If true then averages include padding elements, each treated as the number zero. If false, padding elements are entirely ignored when computing averages.

AvgPool(const Output<Node> &arg, const Shape &window_shape, const Strides &window_movement_strides)

Constructs a batched, unpadded average pooling operation (i.e., all padding shapes are set to 0).

Parameters
  • arg: The output producing the input data batch tensor.[d1, ..., dn]
  • window_shape: The window shape.[n]
  • window_movement_strides: The window movement strides.[n]

AvgPool(const Output<Node> &arg, const Shape &window_shape)

Constructs an unstrided batched convolution operation (i.e., all window movement strides are 1 and all padding shapes are set to 0).

Parameters
  • arg: The output producing the input data batch tensor.[d1, ..., dn]
  • window_shape: The window shape.[n]

void validate_and_infer_types()

Throws if the node is invalid.

const Shape &get_window_shape() const

Return
The window shape.

const Strides &get_window_movement_strides() const

Return
The window movement strides.

const Shape &get_padding_below() const

Return
The below-padding shape.

const Shape &get_padding_above() const

Return
The above-padding shape.

const op::PadType &get_pad_type() const

Return
The pad type for pooling.

shared_ptr<Node> get_default_value() const

Return
The default value for AvgPool.