Django Q object - How to make many / multiple, complex, and dynamic queries

What are Django Q objects??

First of all, remember that every time you query the database, your Python is being converted into the SQL language of your choice (postgresql, sql, sqlite, etc.). And whenever you want to obj.filter() your results, you "AND" everything together in your SQL, meaning you limit yourself on what you can search for.

"X" and "Y" requires both variables for the statement to be True, but "X" or "Y" requires only one of them to be True.

Q() objects allow you to make complex queries on your model fields for multiple terms with an "OR" operator.

So if you have a bunch of books with a book.author attribute, and you want to search those attributes with the words "poe" OR "tolkien": Q() objects allow you to do that. Q() objects are also much faster to write, and require less code than writing a raw function to filter everything yourself.

What is request.GET, and how do you use it for search fields / parameters??

This is a dictionary like object. It lets you check for any keys in it like you would a normal Python dictionary. It is not a real Python dictionary, but it works exactly the same, so don't worry about it. With request.GET, you can add a form field to let users search for multiple terms at a time. I will show how in the example that follows.

An example with Python's reduce function, and functools' OR operator

        <!-- your html template -- >

        <form action="?search/" method="GET">
          <input type="text" name="search">     *****1*****
          <input type="submit" value="search">
        </form>



        # models.py

        class BlogPost(models.Model):
            title = models.CharField(...)
            body = models.TextField(....)
            user = models.ForeignKey(User....)
            .....



            # views.py

            from django.db.models import Q
            from django.shortcuts import render
            import operator
            from functools import reduce
            ...


            ...
            ...
            search = request.GET.get('search', None)  # *****2*****
            if search:      # *****3*****
                    multiple_lookups = reduce(operator.or_, (     # *****5*****
                          Q(title__icontains=term) |
                          Q(body__icontains=term) |
                          Q(user__username__icontains=term) for term in search.split()    # *****4*****
                          )
                        )
                    these_blogs = BlogPost.objects.filter(multiple_lookups)    # *****6*****
                    return render(request, 'your_app/your_template.html', {'these_blogs': these_blogs})
            ....
            ....


      

Explanation

1. We start with creating the form on an html template. Notice how I have a ?search/ for the GET action, as well as name="search" for the input. Remember that anything after the "?" is an OPTIONAL url parameter, and without it you will get errors. Now, if you used this form and searched for steak and yogurt with a space between the two... You would see website.com/some_url?search=steak+yogurt in the url. Also note that this form's action goes to the current url this is at, so there's no need to make another urlpattern and function.
2. Here, we get that search value. If you print(request.GET) in your view with the steak and yogurt parameters, you're going to get <QueryDict: {'search': ['steak yogurt']}>.
3. This line is why we don't need to make a separate function and url pattern. If the request.GET has a truthy value, we alter the results based on that.
4. After splitting our list (remember that the default is an empty space), we loop through each term we're looking for from multiple fields to construct the Q objects. Here, we will be searching a BlogPost's title, body, and username fields to see if the field contains a case insensitive version of the term searched for (steak or Steak, yogurt or Yogurt). Check any other field lookups you'd like to use, but this one is common. We also haven't queried the database yet, we are only builiding the queries we're about to make. What you will get if you print this is (OR: ('title__icontains', 'steak'), ('body__icontains', 'steak'), ('user__username__icontains', 'steak'), ('title__icontains', 'yogurt'), ('body__icontains', 'yogurt'), ('user__username__icontains', 'yogurt')). And notice how a pipe | is used as an "OR" operator. On qwerty keyboards, you make that symbol with shift + backslash. It's above the enter / return key.
5. This reduce() function is from functools. It takes the operator.or_ function, and all the Q() objects after it. Without these you're going to get errors, and you'll have to write a lot more code. Just know that this enables us to use the "OR" operator for all of our terms.
6. Now we filter our BlogPost objects, and give the blogs with those terms.

And you're done! You can check out an example of it on my forum app page. I hope this helps someone.

If anyone ever needs a website or some web apps - maybe the next Patreon alternative or Facebook - feel free to contact me!

--Kevin