Creation Of Array Of Arrays Fails, When First Size Of First Dimension Matches

I stumpled upon some behaviour of the np.array function I do not understand. This code works the way I expect it: arr1 = np.zeros((3,2)) arr2 = np.zeros((2,2)) np.array([arr1,arr2

Solution 1:

As noted in numpy array 1.9.2 getting ValueError: could not broadcast input array from shape (4,2) into shape (4)

there have been changes in how object arrays are created. Where possible np.array tries to create multidimensional array. Creating an object array has been the fallback choice, to be used when the inputs don't match in shape. And even then it can be unpredictable.

The surest way is to make an empty object array of the right size and insert the objects

In [242]: a=np.zeros((2,3));b=np.ones((2,2))

In [243]: arr=np.zeros((2,), object)
In [244]: arr[0]=a; arr[1]=b

In [245]: arr
Out[245]: 
array([array([[ 0.,  0.,  0.],
       [ 0.,  0.,  0.]]),
       array([[ 1.,  1.],
       [ 1.,  1.]])], dtype=object)
In [246]: arr[:]=[a,b]           # also works

You've already seen that np.array([a.T, b]) works. If I turn the arrays into nested lists I get a 2d array of flat lists; 2 and 3 element lists are the lowest level of incompatible objects.

In [250]: np.array([a.tolist(),b.tolist()])
Out[250]: 
array([[[0.0, 0.0, 0.0], [0.0, 0.0, 0.0]],
       [[1.0, 1.0], [1.0, 1.0]]], dtype=object)

In [251]: _.shape
Out[251]: (2, 2)

np.array is compiled, so it would take some digging to decode its logic. My earlier digging indicates that the made some changes to speed up the creation of an array from arrays (more like concatenate). This problem may be a side effect of that change.

The take home message is: if you want to create an object array of arrays, don't count on np.array to do the job right. It's designed primarily to make multidimensional arrays of scalars.

For example if the arrays match in size:

In [252]: np.array([a,a]).shape
Out[252]: (2, 2, 3)

I have to use arr[...]=[a,a] if I want a 2 element object array.