How to switch to torch.double for all simulations?

[maicoldubbio]

Hi all!

For some simulations with exponentials, we run into small number that are below float32.epsilon and thus need to switch to full precision. So what I do is: analog_model.to(device, dtype=torch.double)

However I found out this won't change underlying InferenceRPUConfig computations (I still get, say, 'g_prog' in FP32).

So I looked up: rpu_config_test.runtime.data_type = RPUDataType.DOUBLE
but then in throws
aihwkit.exceptions.ConfigError: Cannot find requested data_type 'double' in rpu_base module.

How can I fix it? Does this get all computations ins FP64?

[maljoras-sony]

Hi @edminge,
double support is only experimental. You need to compile the aihwkit again with -DRPU_USE_DOUBLE=ON flag to enable it (see ). However, we never had any issues using float as base type, in particular considering that the weights are analog and typically scaled with some digital scale, so the dynamic range of double makes not much sense when simulating noisy conductances.

aihwkit/CMakeLists.txt

Line 21 in 39581ec

option(RPU_USE_DOUBLE "EXPERIMENTAL: Build DOUBLE support" OFF)

[maicoldubbio]

Hi @maljoras-sony, thanks for your answer. Now the error makes sense.

Yeah I see, but I think it could be relevant in cases like:

def apply_drift_noise_to_conductances(self, g_prog):
...
g_final = f(g_prog)

and eventually, some g_final end up below 1e-6 (e.g., high-temperature drift for long times or very low temperature activation) .