even though we perceive our universe as expanding from a 'spot'
That is a common misconception. In fact, the universe is _not_ expanding
from any particular spot.
Imagine ants living on a perfectly spherical balloon. If you expand the
balloon-universe (perhaps by heating the air inside) the universe gets
bigger. The ants do not get bigger. Every ant is carried away from every
other ant by of the expansion. The velocity, for any pair of ants, is
proportional to the distance between them.
The ants do not know and do not care that their D=2 universe is embedded in
our D=3 universe.
So it is with us. We observe that _every_ spot in our universe is
expanding away from every other spot, with a velocity proportional to
distance. We do not know, and do not care, whether our universe is
embedded in a higher-dimensional space.
===
Also note that the observation of expansion does not, by itself, tell the
ants anything about the size of their universe. They could be living on a
balloon that is one mile in diameter, or ten miles, or a trillion miles. A
10% expansion looks the same in each case.
So it is with us. It may well be that our universe is infinite in all
directions, and always has been -- yet still it is expanding. No problem.