Based Pointers

One of the more curious C/C++ extensions I have come across is from Microsoft in the form of __based pointers. Unlike a traditional pointer, a based (on) pointer represents a memory address relative to another.

void *base;
void __based(base) *pointer;

If the above example were to be compiled targeting x86, pointer would store a 32-bit integer representing an offset from base. How can we make this more portable?

template<typename T, void *&B>
class based_pointer
{
	intptr_t offset;

public:
	based_pointer<T, B>(T *ptr)
	{
		this->offset = (ptrdiff_t)ptr - (ptrdiff_t)B;
	}

	operator T* ()
	{
		return (T*)((ptrdiff_t)B + (ptrdiff_t)offset);
	}
};