Dev Guide¶

Table of contents

ObjectPool Class

The core module of the object pool creation library and contains all the necessary classes.

ObjectPool Class ¶

class object_pool.pool.ObjectPool(klass, max_capacity=20, min_init=3, max_reusable=20, expires=600, lazy=False, pre_check=False, post_check=True, cloning=False)¶

This is singleton object pool class. It creates pool, checks expiry and validation of the resource.

Parameters:

base_klass –

class for which pool will be created.

In the below example, Pool will be created for Browser class and used.

class Browser:
    def __init__(self):
        self.browser = self.__class__.__create_connection()

    @staticmethod
    def __create_connection():
        obj = "connection_object"
        return obj

    def do_work(self):
        return True

    def clean_up(self, **stats):
        print("close connection object")

    def check_invalid(self, **stats):
        '''Returns True if resource is valid, otherwise False'''
        return False

lazy – by default, resources are created when initiated. lazy option will skip resource creation on init and will create when the pool item is requested.
min_init –
minimum resources will be created while initiating.

Note

When lazy=True is set, min option is not respected.
max_capacity –
Maximum capacity of the pool. Pool will be created with min_init capacity. But it go grow up to max_capacity.
Note
- max_capacity is not a hard constraint to the pool. When the client request for a resource, and no resources are available for client, new resource will be created and provided to the client. But this extra resource will not be queue, it will be cleaned up without performing any validation.
- pool is implemented using Queue. But maxsize is not provided to handle max_capacity. This is a implementation choice.
- As creation and cleaning up of additional resource performed when pool gets full, This will slow down the program. This is a cue to check bottleneck on client processing.
max_reusable – maximum number of times resource can be reused. Once this exceeds, respective resource will be destroyed and new resource will be created. By default, 20 is set. You can disable this by setting to 0.
expires –
resource expiration in seconds.

Note

Example: expires=600, Resource will be only alive for 10 minutes from the creation.
pre_check – resource validation is performed before requesting the resource. This is disabled by default.
post_check –
resource expiration checked after resource is being used. This is the default option.

Note

Base class should define clean_up method when expiry is set or max_reusable is set or check_valid method is defined in the class. If the clean up such as closing database connection or closing browser are not performed, those process will run in the background and cause performance issue in the system.

cloning –

reserved resource will be created to create new resource, in case of resource expiration. Cloning is disabled by default. You can enable by passing cloning=True.

Note

Reserved resource will be created even lazy=True option provided to reduce the resource creation time.

>>> class Browser: # Objective pool class
...     def __init__(self):
...         self._browser = "connected!"
...
...     def do_work(self):
...         return "job done!"
...
...     def clean_up(self, **stats):
...         print("stats contains resource stats")
>>> # default ConnectionPool options
>>> connection_pool = ObjectPool(Browser, max_capacity=20, min_init=3, max_reusable=20,
...             expires=600, lazy=False, pre_check=False, post_check=True, cloning=False)

get()¶

Creates contextmanager instance and returns resource and stats

>>> class Connection:
...     def __init__(self):
...         self._conn = "connected!"
...
...     def do_work(self):
...         print("job done!")
...
>>> pool = ObjectPool(Connection, min_init=3)
>>>
>>> with pool.get() as (resource, resource_stats):
...     resource.do_work()
job done!

get_pool_size()¶

Returns the size of the pool (queue).

Any operation on the queue results in different output time to time as the resources are removed from queue and added back.

>>> pool = ObjectPool(Connection, min_init=3)
>>> print(pool.get_pool_size())
3

static pool_exists(klass)¶

Return True if the pool is already created, False otherwise.

>>> pool = ObjectPool(Connection, min_init=3)
>>> ObjectPool.pool_exists(Connection)
True
>>> ObjectPool.pool_exists(DummyConnection)
False

destroy()¶

Removes pool from the registry and performs clean up.

When the pool is destroyed and cleanup_func is provided or class has clean_up method defined while creating, respective clean up method is called to clean up on the object.

Example: When the connection pool is destroyed, all the connection objects in the pool will be closed if the cleanup method is provided when creating the pool.

>>> class Connection:
...     def __init__(self):
...         self._conn = "connected!"
...
...     def do_work(self):
...         return "job done!"
...
...     def clean_up(self, **resource_stats):
...         return "cleanup performed!"
...
>>> pool = ObjectPool(Connection, min_init=1)
>>> pool.destroy()
cleanup performed!

is_pool_full()¶

Return True if the pool is full, False otherwise.

>>> pool = ObjectPool(Connection, min_init=3, max_capacity=3)
>>> pool.is_pool_full()
True

Note

This method will always return False when max_capacity=0, As the pool grow unlimited.

Dev Guide¶

ObjectPool Class¶

ObjectPool Class ¶