Matrix memory cache

This notebook demonstrates the usage of the in-memory precomputed interpolation weights cache. Please note this is only available for the precomputed backend (for both the high-level and and array-level regrid() methods). By default this cache is enabled, for details see In-memory caching for precomputed weights.

import numpy as np

import earthkit.geo as ekg

# create input data array for an O1280 grid
in_data = np.ones(6599680)


# helper method for regrid
def _run(n=10):
    for _ in range(n):
        ekg.grids.array.regrid(in_data, in_grid={"grid": "O1280"}, out_grid={"grid": [0.1, 0.1]}, backend="precomputed")

In the examples below we will change the configuration multiple times. First we ensure all the changes are temporary and no options are saved into the configuration file. We also reset the configuration to the defaults.

ekg.config.autosave = False
ekg.config.reset()

The “off” cache policy

We can control the in-memory weights cache via the Configuration. In this example we turn off the in-memory cache.

ekg.config.set(regrid_precomputed_weights_memory_cache_policy="off")

%time _run()

CPU times: user 5.97 s, sys: 619 ms, total: 6.59 s
Wall time: 6.74 s

ekg.grids.utils.precomputed_memory_cache_info()

CacheInfo(hits=0, misses=0, maxsize=0, currsize=0, count=0, policy='off')

The “lru” (default) cache policy

Now we will use the default memory cache policy, which is “lru”. For details see: LRU cache policy. Notice the roughly x9 speed-up we achieved with the caching.

ekg.config.set(regrid_precomputed_weights_memory_cache_policy="lru")

%time _run()

CPU times: user 498 ms, sys: 70.8 ms, total: 569 ms
Wall time: 573 ms

ekg.grids.utils.precomputed_memory_cache_info()

CacheInfo(hits=9, misses=1, maxsize=524288000, currsize=259170724, count=1, policy='lru')

Clearing the cache

We can clear the cache with clear_precomputed_memory_cache().

ekg.grids.utils.clear_precomputed_memory_cache()
ekg.grids.utils.precomputed_memory_cache_info()

CacheInfo(hits=0, misses=0, maxsize=524288000, currsize=0, count=0, policy='lru')