It's expensive only if you think within the current design constraints of Pin.

To me, logic dictates that whatever information you have when you emit the instrumented code is information you can store and make accessible later. Thus, with some space overhead, Pin could provide information on demand with zero time overhead for the common case, and with an overhead that's proportional to the amount of information needed if information is in fact accessed.  (I would tolerate some inaccuracy - for instance, I wouldn't insist that you report the values of registers that are dead in the original code and which the instrumented code thus doesn't maintain, etc.)

Robert is correct that there's an common vs. uncommon question when at least some of the states is accessed most of the time, but clever design of data structures might be able to ameliorate that. It's clear that what Pin currently does - inlining dozens of instructions to pass the state to an outlined function - is very inexpensive and led to the 3x slowdown. ("complex inlinable functions" notwithstanding.)

On a related note, some tasks are extremely difficult, or even impossible, to implement in Pin because you don't have access to the compiler: a simple gen/kill analysis for each trace, for instance. And doing it dynamically at runtime kills you for the reasons listed above. That at least should not be an issue with DynamoRIO because it provides source for its instrumentation framework.

 - Godmar