Python MongoEngine Document class with dynamic collection name¶

Normally, a MongoEngine Document class corresponds to a single collection. But sometimes, we want a Document class to represent multiple collections. For example, we may want to split a big collection into multiple ones separated by dates, that is, one collection for one day's data.

General idea¶

With the help of MongoEngine's switch_collection function, we can dynamically change the collection name of a Document class. Everytime before we use a Document class, we use switch_collection to set the appropriate collection name.

Code Example¶

Assume that we use multiple collections to save logs. Collection names are logs-2023-02-23, logs-2023-02-24, logs-2023-02-25 and so on. These collections share the same Document class.

In the code below, we use switch_collection to change the collection name of the Log Document class.

import mongoengine as me
from mongoengine.context_managers import switch_collection

me.connect('test')

# define the document
class Log(me.Document):
    content = me.StringField(require=True)

# define an example function that uses the Document class defined above 
def save_log(content):
    Log(content=content).save()

# this log will be saved to the 'log' collection
save_log('log example 001')

# this log will be saved to the 'logs-2023-02-23' collection
with switch_collection(Log, 'logs-2023-02-23'):
    save_log('log log example 002')

# this log will be saved to the 'logs-2023-02-24' collection
with switch_collection(Log, 'logs-2023-02-24'):
    save_log('log log example 003')

In the code above, collection names are hardcoded.
Here's a more practical code example, the collection name is dynamically built:

from datetime import datetime

today = datetime.now().strftime('%Y-%m-%d')
with switch_collection(Log, today):
    save_log('log log example 004')