RediSearch in Django(Rest framework)

RediSearch is an innovative NoSQL extension that augments the capabilities of Redis, a powerful in-memory data store. It empowers developers to execute rapid, high-performance searches and advanced indexing on extensive datasets within Redis. With its rich features including full-text search, secondary indexing, and geospatial search, RediSearch enables real-time retrieval of pertinent information. Seamlessly integrated into Redis, it offers auto-suggestion, auto-completion, and numeric filtering. This versatile solution is particularly well-suited for applications requiring swift and efficient search operations, eliminating the need for standalone search engines.

Django is the most powerful web framework of Python. But how to use that RediSearch in Django? It's EASY.

To use you have to install Redis in your environment using PIP.

pip install redis        

But redis database is similar to NoSQL. We need to use Redis in structural way like Pydantic where we can use write our pydantic model with fields. So that we need to use Redis-om instant of Redis. So install the following package redis-om which use pydantic for object relational model.

pip install redis-om        

So you installed redis in your project but what about your server because redis runs on a special port like django server. To perform the RediSearch, you need to install redis-stack-server in your machine or docker because redis-server do not support search. By the way, redis-server and redis-stack-server are different from each other. Redis-stack-server is a different type of server.

In windows, you need to install WSL because redis donot support wind😐ws. After installing the WSL(ubuntu), open the ubuntu shell/terminal and follow the linux installation process.

In linux, please make sure that your linux is supported by Redis-stack-server because only limited version is supported. Read more: Install Redis Stack on Linux | Redis

If your linux version is ok, now install the redis-stack-server from your shell/terminal and run the following shell commands.

curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gp

sudo chmod 644 /usr/share/keyrings/redis-archive-keyring.gpg

echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/redis.list

sudo apt-get update

sudo apt-get install redis-stack-serverg        

OR, if you are facing a problem, please use docker by running a simple container

docker run -d --name redis-stack-server -p 6379:6379 redis/redis-stack-server:latest        

Now time to start our redis-stack-server. Write following script to run your server.

N.B. In windows, every time you start your computer, you need to run the following command in your ubuntu shell.

sudo service redis-stack-server start        

Your machine setup is COMPLETED.

NOW, time to code.

In django, i created two models named Employee and Category where Category is a foreignKey of Employee

import uuid

from django.contrib.auth import get_user_model
from django.db import models

from common.models import BaseModelWithUID
from users.choices import EmployeeDepartment

class BaseModelWithUID(models.Model)
    class Meta:
        abstract = True


    uid = models.UUIDField(
        default=uuid.uuid4, editable=False, unique=True, db_index=True
    )
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)

User = get_user_model()

class Category(BaseModelWithUID):
    name = models.CharField(max_length=255)
    def __str__(self):
        return f"Name: {self.name}"

class Employee(BaseModelWithUID):
    name = models.CharField(max_length=300)
    department = models.CharField(choices=EmployeeDepartment.choices, max_length=25)
    category = models.ForeignKey(
        Category, on_delete=models.CASCADE, null=True, blank=True
    )
    def __str__(self):
        return f"Name: {self.name}"d
        

As redis-om doc, we need to create a pydantic data which is needed to be saved to redis database.

/redisio/users/schemas.py

from redis_om import JsonModel, Field, EmbeddedJsonModel, Migrator

class CategoryPydantic(EmbeddedJsonModel):
    uid: str = Field(index=False)
    name: str = Field(index=False)

class EmployeePydantic(JsonModel):
    uid: str = Field(index=False)
    name: str = Field(index=True, full_text_search=True)
    department: str = Field(index=False)
    category = CategoryPydantic

Migrator().run()        

We use EmployeePydantic which inherit the JsonModel which is responsible to save this pydantic data to redis database as json format and EmbeddedJsonModel is the relational json data which is connected to EmployeePydantic model's category field and i only need Employee name to search the text. that's why i declare this as index=True and to enable field search, we have to add full_text_search=True.

So, how to add an employee to redis? While we are creating the Employee instance, we saved that instance to redis as well.

But we need to save json data, so need to send the serialize json data into the create function. So i need a serializer which will convert the python data into json data by default.

class PrivateCategorySerializer(serializers.ModelSerializer):
    class Meta:
        model = Category
        fields = ("uid", "name")

class PrivateEmployeeListSerializer(serializers.ModelSerializer):
    category = PrivateCategorySerializer(read_only=True)
    category_uid = serializers.SlugRelatedField(
        write_only=True, queryset=Category.objects.filter(), slug_field="uid"
    )

    class Meta:
        model = Employee
        fields = (
            "uid",
            "name",
            "department",
            "category",
            "category_uid",
        )
        read_only_fields = ("category",)

    def create(self, validated_data):
        validated_data["category"] = validated_data.pop("category_uid")
        employee: Employee = super().create(validated_data)

        # a saving instance to redis
        employee_redis_search = EmployeeRedisServices()
        employee_redis_search.create(instance=employee)
        return employee        

This will convert our Employee instance into json format.

class EmployeeRedisServices
    def create(self, instance: Employee = None, serialize_data: dict = None):
        serialized_data = serialize_data
        # check if instance, make this instance to serialized data (json)
        if instance:
            from users.rest.serializers.employees import PrivateEmployeeListSerializer
            serialized_data = PrivateEmployeeListSerializer(instance).data


        # if saved to json, saved to redis using pydantic data. Same as Python Pydantic.
        new_employee_to_redis_save = EmployeePydantic(
            uid=serialized_data.get("uid"),
            name=serialized_data.get("name"),
            department=serialized_data.get("uid"),
            category=serialized_data.get("category"),
        )
        new_employee_to_redis_save.save():        

in create function, if i get the instance, we will convert this instance to json using serializers. and save that serialized dict formatted data into pydantic model and save that instance into redis. This will save the formatted data into redis as json format.

N.B. if you want to redis data visual, you may download RedisInsight v2 for windows.

It time to search the a employee by name

class EmployeeRedisServices:
    def search(self, name) -> List[dict]:
        # search employee name
        # use * for find name startswith, * means all character will be accepted after searching name
        # redis by default case in-sensitive.
        employee_from_redis = EmployeePydantic.find(EmployeePydantic.name % f"{name}*")


        # push that searching data into python queue
        response_data = deque(maxlen=employee_from_redis.count())
        for employee in employee_from_redis.all():
            response_data.appendleft(employee.dict(exclude={"pk"}))
        return list(response_data)        

using %, we can search the data which is startswith.

N.B. RediSearch is case-insensitive by default. So whatever you want to search, you can use capital or small character.

EmployeePydantic.find will filter your data redis-om. use % to search the data. use * to ignore rest of the character if exists the pydantic field's value.

But how is my views looks like:

class PrivateEmployeeList(ListCreateAPIView):
    permission_classes = (IsAuthenticated,)
    serializer_class = PrivateEmployeeListSerializer

    def get_queryset(self):
        return None


    def get(self, request, *args, **kwargs):
        search = self.request.query_params.get("search", "")
        employee_redis_search = EmployeeRedisServices()

        if not search:
            return Response(data=employee_redis_search.all())

        return Response(data=employee_redis_search.search(name=search))
        

We all search from redis-database and successfully working.

Wanna see that project?

Feel free to ask questions.