There is indeed a problem in mixing both ideas, as I see it.
I'd say that the conception of mass "as the amount of substance which can
not be changed without taken somehting away, or adding" may be a
misconception*. Namely, as I move faster and faster, relative to someone
else, I see her/him getting fater and fater, and she/he is not eating that
much. It is the inertia which is increasing.
Moreover the idea of 'amount of substance' is misleading because if we
think of an object as the sum of fundamental particles, thinking about the
'amount of substance' of the object seems to suggest that we're thinking
about the number of fundamental particles that compose it. This, however,
can only delay the question to describing the mass of fundamental particle
for which the definition of mass as "the amount of substance which cannot be
changed without taken somehting away, or adding", is meaningless due to the
inseparability of the particle.
james