Omoide Cache introduction, quick and easy caching in Python

In this article I’d like to introduce and show how to use a library that I developed - Omoide Cache (GitHub / PyPI). This is a robust, highly tunable and easy-to-integrate in-memory cache solution written in pure Python, with no dependencies. It is designed to be a method level cache, wrapping around a single class method, using method call arguments as cache key and storing its return value.

Hopefully some of you will find this useful, and maybe even integrate Omoide Cache in your own projects. Without further adieu - let’s get started!

Cache via decorator

First I’ll demonstrate how easy it is to get started with Omoide Cache. Integrating it into your code will literally add 2 more lines - an import statement and a decorator over the method that you want to cache. But first things first - let’s assume we have some sort of service, that has some time-consuming method, and we’d like to speed it up. As a quick example let’s generate a list of popular books, completely random, and put a time delay before it, let’s say 3 seconds. Here’s a quick code snippet with this example:

1import time
2from typing import List
3from simplestr import gen_str_repr_eq
4from omoide_cache import omoide_cache, RefreshMode
5from tekleo_common_utils import UtilsId, UtilsTime, UtilsRandom
6
7
8utils_id = UtilsId()
9utils_time = UtilsTime()
10utils_random = UtilsRandom()
11
12
13@gen_str_repr_eq
14class Book:
15    title: str
16    author: str
17    year: int
18    country: str
19
20    def __init__(self, title: str, author: str, year: int, country: str):
21        self.title = title
22        self.author = author
23        self.year = year
24        self.country = country
25
26
27def time_now_str() -> str:
28    return utils_time.format_timestamp_ms(utils_time.get_timestamp_ms_now(), date_format='%d.%m.%Y %H:%M:%S')
29
30
31def random_book() -> Book:
32    return Book(
33        utils_random.get_random_docker_name().replace('_', ' ').title(),
34        utils_random.get_random_full_name(),
35        utils_random.get_random_year(),
36        utils_random.get_random_country()
37    )
38
39
40class BookService:
41    def get_popular_books(self) -> List[Book]:
42        time.sleep(3.0)
43        books = [random_book() for i in range(0, 10)]
44        print("BookService.get_popular_books(): Generated at " + time_now_str() + ", first book is " + str(books[0]))
45        return books
46
47
48service = BookService()
49
50
51# Try 1
52books = service.get_popular_books()
53print("Obtained at " + time_now_str() + ", first book is " + str(books[0]) + '\n')
54
55# Try 2
56books = service.get_popular_books()
57print("Obtained at " + time_now_str() + ", first book is " + str(books[0]) + '\n')
58
59# Try 3
60books = service.get_popular_books()
61print("Obtained at " + time_now_str() + ", first book is " + str(books[0]) + '\n')

When we call this service 3 times it will generate a new list each time and there will be an expected delay as follows:

1BookService.get_popular_books(): Generated at 03.07.2022 19:15:44, first book is Book{title=Thirsty Feynman, author=Michelle Garcia, year=1945, country=Sudan}
2Obtained at 03.07.2022 19:15:44, first book is Book{title=Thirsty Feynman, author=Michelle Garcia, year=1945, country=Sudan}
3
4BookService.get_popular_books(): Generated at 03.07.2022 19:15:47, first book is Book{title=Sharp Spence, author=Brooke Smith, year=1996, country=Congo}
5Obtained at 03.07.2022 19:15:47, first book is Book{title=Sharp Spence, author=Brooke Smith, year=1996, country=Congo}
6
7BookService.get_popular_books(): Generated at 03.07.2022 19:15:50, first book is Book{title=Lucid Lovelace, author=Brandon Swanson, year=1943, country=Netherlands}
8Obtained at 03.07.2022 19:15:50, first book is Book{title=Lucid Lovelace, author=Brandon Swanson, year=1943, country=Netherlands}

Now let’s add our cache to it! It is very simple at this point, we just drop a decorator on top of our costly method:

1class BookService:
2    @omoide_cache()
3    def get_popular_books(self) -> List[Book]:
4        ...

And when we run it this time - only the first call will trigger the real method, and the remaining 2 calls will reuse the cached value:

1BookService.get_popular_books(): Generated at 03.07.2022 19:15:26, first book is Book{title=Brave Clarke, author=Xavier Walker, year=1982, country=Cambodia}
2Obtained at 03.07.2022 19:15:26, first book is Book{title=Brave Clarke, author=Xavier Walker, year=1982, country=Cambodia}
3
4Obtained at 03.07.2022 19:15:26, first book is Book{title=Brave Clarke, author=Xavier Walker, year=1982, country=Cambodia}
5
6Obtained at 03.07.2022 19:15:26, first book is Book{title=Brave Clarke, author=Xavier Walker, year=1982, country=Cambodia}

Cache with expiry

Obviously most of the times you don’t want to cache the method’s return value indefinitely. You’d want it to be recomputed at some point, and we provide several ways to achieve this behavior. The most basic approach is expiry - the cache will invalidate stored value after a certain amount of time has passed since it was last computed. To set this up just add a parameter to the decorator:

1class BookService:
2    @omoide_cache(expire_by_computed_duration_s=5.0)
3    def get_popular_books(self) -> List[Book]:
4        ...

Then if we try to call this service the following chain of events will happen:

1# Try 1 - will generate initial results
2books = service.get_popular_books()
3print("Obtained at " + time_now_str() + ", first book is " + str(books[0]) + '\n')
4
5# Try 2 - will return cached initial results
6books = service.get_popular_books()
7print("Obtained at " + time_now_str() + ", first book is " + str(books[0]) + '\n')
8
9# Delay
10print('Sleeping...\n')
11time.sleep(6.0)
12
13# Try 3 - will return cached initial results & mark them as expired
14books = service.get_popular_books()
15print("Obtained at " + time_now_str() + ", first book is " + str(books[0]) + '\n')
16
17# Try 4 - will generate new results
18books = service.get_popular_books()
19print("Obtained at " + time_now_str() + ", first book is " + str(books[0]) + '\n')

1BookService.get_popular_books(): Generated at 03.07.2022 21:14:58, first book is Book{title=Silly Pare, author=Megan Mcclure, year=1946, country=Guinea-Bissau}
2Obtained at 03.07.2022 21:14:58, first book is Book{title=Silly Pare, author=Megan Mcclure, year=1946, country=Guinea-Bissau}
3
4Obtained at 03.07.2022 21:14:58, first book is Book{title=Silly Pare, author=Megan Mcclure, year=1946, country=Guinea-Bissau}
5
6Sleeping...
7
8Obtained at 03.07.2022 21:15:04, first book is Book{title=Silly Pare, author=Megan Mcclure, year=1946, country=Guinea-Bissau}
9
10BookService.get_popular_books(): Generated at 03.07.2022 21:15:07, first book is Book{title=Elated Ardinghelli, author=Emma Higgins, year=2008, country=Chad}
11Obtained at 03.07.2022 21:15:07, first book is Book{title=Elated Ardinghelli, author=Emma Higgins, year=2008, country=Chad}

Cache with async refresh

You probably noticed the issue with previous approach? When the value is recomputed the client who calls our service has to wait. The refresh is performed in sync with the service’s clients. But what if we can’t afford these delays? What if we want a seamless experience for the service’s client? Well, here comes the async refresh - it will recompute the return value in a background thread. Same as before, just add parameters to the decorator:

1class BookService:
2    @omoide_cache(refresh_duration_s=10.0, refresh_mode=RefreshMode.INDEPENDENT, refresh_period_s=2.0)
3    def get_popular_books(self) -> List[Book]:
4        time.sleep(0.5)
5        ...

In this example we tell the cache to asynchronously check itself every 2 seconds, and recompute values that were last computed more than 10 seconds ago. And the cache’s behavior becomes:

1# Try 1 - will generate initial results
2books = service.get_popular_books()
3print("Obtained at " + time_now_str() + ", first book is " + str(books[0]) + '\n')
4
5# Delay, refresh will continue to run in its own thread, checking every 2 seconds, and refreshing results older than 10 seconds
6print('Sleeping...\n')
7time.sleep(40.0)
8
9# Try 2 - will return last cached results
10books = service.get_popular_books()
11print("Obtained at " + time_now_str() + ", first book is " + str(books[0]) + '\n')

1BookService.get_popular_books(): Generated at 03.07.2022 21:47:06, first book is Book{title=Determined Bohr, author=Wendy Harmon, year=1915, country=Puerto Rico}
2Obtained at 03.07.2022 21:47:06, first book is Book{title=Determined Bohr, author=Wendy Harmon, year=1915, country=Puerto Rico}
3
4Sleeping...
5
6BookService.get_popular_books(): Generated at 03.07.2022 21:47:18, first book is Book{title=Distracted Archimedes, author=Michael Thompson, year=1951, country=Moldova}
7BookService.get_popular_books(): Generated at 03.07.2022 21:47:29, first book is Book{title=Amazing Aryabhata, author=Jessica Wiley, year=1993, country=Israel}
8BookService.get_popular_books(): Generated at 03.07.2022 21:47:39, first book is Book{title=Zealous Minsky, author=Mr. James Lewis, year=1996, country=Burundi}
9Obtained at 03.07.2022 21:47:46, first book is Book{title=Zealous Minsky, author=Mr. James Lewis, year=1996, country=Burundi}

These were a few basic examples on how you can use Omoide Cache to speed up your Python services, I hope this tutorial was useful, and you will be able to add Omoide Cache to your project without any issues.

GitHub repo with source code for this tutorial

In case you’d like to check my other work or contact me:

Omoide Cache introduction, quick and easy caching in Python

Cache via decorator

Cache with expiry

Cache with async refresh

More articles from TekLeo

Ping and SYN flood attacks with Python and Scapy

Stress testing websites with HTTP flood attack