Dynamic Shapes

For an example on how to use dynamic shapes, see the Scenario Two: Known Partial Shape documentation.

Runtime Error Checking

Static type-checking in the presence of dynamic shapes will make optimistic assumptions about things like shape mismatches. For example, if an elementwise op is provided inputs of shapes (2,?) and (?,5), the type checker will proceed under the assumption that the user is not going to pass tensors with inconsistent shape at runtime, and therefore infer an output shape of (2,5). That means that shape mismatches can now occur at runtime.

PartialShape, Dimension, and Rank Classes

Partial shape information is expressed via the PartialShape, Dimension, and Rank classes.

Note

Rank is an alias for Dimension, used when the value represents the number of axes in a shape, rather than the size of one dimension in a shape.

class PartialShape

Class representing a shape that may be partially or totally dynamic.

XXX: THIS CLASS IS EXPERIMENTAL AND THE ENTIRE DESIGN IS SUBJECT TO CHANGE.

A PartialShape may have:

  • Dynamic rank. (Informal notation: ?)
  • Static rank, but dynamic dimensions on some or all axes. (Informal notation examples: {1,2,?,4}, {?,?,?})
  • Static rank, and static dimensions on all axes. (Informal notation examples: {1,2,3,4}, {6}, {})

Public Functions

PartialShape(std::initializer_list<Dimension> init)

Constructs a shape with static rank from an initializer list of Dimension.

Examples:

Parameters
  • init: The Dimension values for the constructed shape.

PartialShape s{2,3,4};                     // rank=3, all dimensions static
PartialShape s{};                          // rank=0
PartialShape s{2,Dimension::dynamic(),3};  // rank=3, dimension 1 dynamic

PartialShape(const std::vector<Dimension> &dimensions)

Constructs a PartialShape with static rank from a vector of Dimension.

Parameters
  • dimensions: The Dimension values for the constructed shape.

PartialShape()

Constructs a static PartialShape with zero rank (the shape of a scalar).

PartialShape(const Shape &shape)

Constructs a static PartialShape from a Shape.

Parameters

bool is_static() const

Check if this shape is static.

A shape is considered static if it has static rank, and all dimensions of the shape are static.

Return
true if this shape is static, else false.

bool is_dynamic() const

Check if this shape is dynamic.

A shape is considered static if it has static rank, and all dimensions of the shape are static.

Return
false if this shape is static, else true.

Rank rank() const

Get the rank of the shape.

Return
The rank of the shape. This will be Rank::dynamic() if the rank of the shape is dynamic.

bool compatible(const PartialShape &s) const

Check whether this shape is compatible with the argument, i.e., whether it is possible to merge them.

Two shapes are compatible if

  • one or both of them has dynamic rank, or
  • both shapes have dynamic and equal rank, and their dimensions are elementwise compatible (see Dimension::compatible()).
Return
true if this shape is compatible with s, else false.
Parameters
  • s: The shape to be checked for compatibility with this shape.

bool same_scheme(const PartialShape &s) const

Check whether this shape represents the same scheme as the argument.

Two shapes

s1 and s2 represent the same scheme if
  • they both have dynamic rank, or
  • they both have static and equal rank r, and for every i from 0 to r-1, s1[i] represents the same scheme as s2[i] (see Dimension::same_scheme()).
Return
true if this shape represents the same scheme as s, else false.
Parameters
  • s: The shape whose scheme is being compared with this shape.

bool relaxes(const PartialShape &s) const

Check whether this shape is a relaxation of the argument.

Intuitively, a

PartialShape s1 is said to relax s2 (or is a relaxation of s2) if it is “more permissive” than s2. In other words, s1 is a relaxation of s2 if anything you can form by plugging things into the dynamic dimensions of s2 is also something you can form by plugging things into the dynamic dimensions of s1, but not necessarily the other way around.
Return
true if this shape relaxes s, else false.
Parameters
  • s: The shape which is being compared against this shape.

s1.relaxes(s2) is equivalent to s2.refines(s1).

Formally, PartialShape s1 is said to relax PartialShape s2 if:

  • s1 has dynamic rank, or
  • s1 and s2 both have static rank r, and for every i from 0 to r-1, either s1[i] is dynamic, or s1[i] == s2[i].

bool refines(const PartialShape &s) const

Check whether this shape is a refinement of the argument.

Intuitively, a

PartialShape s1 is said to relax s2 (or is a relaxation of s2) if it is “less permissive” than s2. In other words, s1 is a relaxation of s2 if anything you can form by plugging things into the dynamic dimensions of s1 is also something you can form by plugging things into the dynamic dimensions of s2, but not necessarily the other way around.
Return
true if this shape refines s, else false.
Parameters
  • s: The shape which is being compared against this shape.

s1.refines(s2) is equivalent to s2.relaxes(s1).

Formally, PartialShape s1 is said to refine PartialShape s2 if:

  • s2 has dynamic rank, or
  • s1 and s2 both have static rank r, and for every i from 0 to r-1, either s2[i] is dynamic, or s1[i] == s2[i].

bool merge_rank(Rank r)

Checks that this shape’s rank is compatible with r, and, if this shape’s rank is dynamic and r is static, updates this shape to have a rank of r with dimensions all dynamic.

Return
true if this shape’s rank is compatible with r, else false.

Shape to_shape() const

Convert a static PartialShape to a Shape.

Return
A new Shape s where s[i] = size_t((*this)[i]).
Exceptions

bool all_non_negative() const

Returns true if all static dimensions of the tensor are non-negative, else false.

const Dimension &operator[](size_t i) const

Index operator for PartialShape.

Return
A reference to the ith Dimension of this shape.
Parameters
  • i: The index of the dimension being selected.

Dimension &operator[](size_t i)

Index operator for PartialShape.

Return
A reference to the ith Dimension of this shape.
Parameters
  • i: The index of the dimension being selected.

operator std::vector<Dimension>() const

Returns a vector of the dimensions. This has no meaning if dynamic.

Public Static Functions

PartialShape dynamic(Rank r = Rank::dynamic())

Construct a PartialShape with the given rank and all dimensions (if any) dynamic.

Return
A PartialShape with the given rank, and all dimensions (if any) dynamic.

bool merge_into(PartialShape &dst, const PartialShape &src)

Try to merge one shape into another.

Merges

src into dst, returning true on success and false on failure. If false is returned, the effect on dst is unspecified.
Return
true if merging succeeds, else false.
Parameters
  • dst: The shape that src will be merged into.
  • src: The shape that will be merged into dst.

To merge two partial shapes s1 and s2 is to find the most permissive partial shape s that is no more permissive than s1 or s2, if s exists. For example:

merge(?,?) -> ?
merge(?,{?,?}) -> {?,?}
merge({?,?},{?,?}) -> {?,?}
merge({1,2,3,4},?) -> {1,2,3,4}
merge({1,2},{1,?}) -> {1,2}
merge({1,2,?,?},{1,?,3,?}) -> {1,2,3,?}
merge({1,2,3},{1,2,3}) -> {1,2,3}

merge({1,?},{2,?}) fails [dimension 0 constraints are inconsistent]
merge({?,?},{?,?,?}) fails [ranks are inconsistent]

This function (merge_into) performs the “merge” operation described above on dst and src, but overwrites dst with the result and returns true if merging is successful; if merging is unsuccessful, the function returns false and may make unspecified changes to dst.

bool broadcast_merge_into(PartialShape &dst, const PartialShape &src, const op::AutoBroadcastSpec &autob)

Try to merge one shape into another along with implicit broadcasting.

class Dimension

Class representing a dimension, which may be dynamic (undetermined until runtime), in a shape or shape-like object.

Static dimensions may be implicitly converted from int64_t. A dynamic dimension is constructed with Dimension() or Dimension::dynamic().

XXX: THIS CLASS IS NOT IN USE YET AND THE ENTIRE DESIGN IS SUBJECT TO CHANGE.

Public Functions

Dimension(int64_t dimension)

Construct a static dimension.

Parameters
Exceptions

Dimension()

Construct a dynamic dimension.

bool is_static() const

Check whether this dimension is static.

Return
true if the dimension is static, else false.

bool is_dynamic() const

Check whether this dimension is dynamic.

Return
false if the dimension is static, else true.

operator int64_t() const

Convert this dimension to int64_t. This dimension must be static.

Exceptions
  • std::invalid_argument: If this dimension is dynamic.

operator size_t() const

Convert this dimension to size_t. This dimension must be static and non-negative.

Exceptions
  • std::invalid_argument: If this dimension is dynamic or negative.

bool same_scheme(const Dimension &dim) const

Check whether this dimension represents the same scheme as the argument (both dynamic, or equal).

Return
true if this dimension and dim are both dynamic, or if they are both static and equal; otherwise, false.
Parameters
  • dim: The other dimension to compare this dimension to.

bool compatible(const Dimension &d) const

Check whether this dimension is capable of being merged with the argument dimension.

Two dimensions are considered compatible if it is possible to merge them. (See

Dimension::merge.)
Return
true if this dimension is compatible with d, else false.
Parameters
  • d: The dimension to compare this dimension with.

bool relaxes(const Dimension &d) const

Check whether this dimension is a relaxation of the argument.

A dimension

d1 relaxes (or is a relaxation of) d2 if d1 and d2 are static and equal, or d1 is dynamic.
Return
true if this dimension relaxes d, else false.
Parameters
  • d: The dimension to compare this dimension with.

d1.relaxes(d2) is equivalent to d2.refines(d1).

bool refines(const Dimension &d) const

Check whether this dimension is a refinement of the argument.

A dimension

d2 refines (or is a refinement of) d1 if d1 and d2 are static and equal, or d2 is dynamic.
Return
true if this dimension relaxes d, else false.
Parameters
  • d: The dimension to compare this dimension with.

d1.refines(d2) is equivalent to d2.relaxes(d1).

Dimension operator+(const Dimension &dim) const

Addition operator for Dimension.

Return
Dimension::dynamic() if either of *this or dim is dynamic; else, a static dimension with value int64_t(*this)+in64_t(dim).
Parameters
  • dim: Right operand for addition.

Dimension operator-(const Dimension &dim) const

Subtraction operator for Dimension.

Return
Dimension::dynamic() if either of *this or dim is dynamic; else, a static dimension with value int64_t(*this)-int64_t(dim).
Parameters
  • dim: Right operand for subtraction.

Dimension operator*(const Dimension &dim) const

Multiplication operator for Dimension.

Return
0 if either of *this or dim is static and 0; else, Dimension::dynamic() if either of *this or dim is dynamic; else, a static dimension with value int64_t(*this)*int64_t(dim).
Parameters
  • dim: Right operand for multiplicaiton.

Dimension &operator+=(const Dimension &dim)

Add-into operator for Dimension.

Return
A reference to *this, after updating *this to the value *this + dim.
Parameters
  • dim: Right operand for addition.

Dimension &operator*=(const Dimension &dim)

Multiply-into operator for Dimension.

Return
A reference to *this, after updating *this to the value *this * dim.
Parameters
  • dim: Right operand for multiplication.

Public Static Functions

bool merge(Dimension &dst, const Dimension d1, const Dimension d2)

Try to merge two Dimension objects together.

  • If d1 is dynamic, writes d2 to dst and returns true.
  • If d2 is dynamic, writes d1 to dst and returns true.
  • If d1 and d2 are static and equal, writes d1 to dst and returns true.
  • If d1 and d2 are both static and unequal, leaves dst unchanged and returns false.
Return
true if merging succeeds, else false.
Parameters
  • dst: Reference to write the merged Dimension into.
  • d1: First dimension to merge.
  • d2: Second dimension to merge.

bool broadcast_merge(Dimension &dst, const Dimension d1, const Dimension d2)

Try to merge two Dimension objects together with implicit broadcasting of unit-sized dimension to non unit-sized dimension.

static Dimension dynamic()

Create a dynamic dimension.

Return
A dynamic dimension.

Public Static Attributes

const int64_t s_dynamic_val = {(std::numeric_limits<int64_t>::max())}

Constant for the value used internally to represent a dynamic dimension.