How to use MeiliSearch with React

MeiliSearch is an open-source search engine written in Rust that provides a fast and relevant search experience to your users: it can be considered an alternative to Algolia, Elasticsearch, or Solr.

In this tutorial, we will build a search engine for our blog posts using MeiliSearch and React: I will show you how I added Meilisearch to this blog to search through my blog posts.

NB: We assume you already have a working React application and MDX files for your blog posts.

The final result (which you can test live) is the following:

Installing and running Meilisearch using Docker

To install MeiliSearch, you can use Docker or others. I will use Docker in this tutorial.

First, we want to add the Docker configuration to our project. To do so, let's add the file docker-compose.yml at the root of our project:

docker-compose.yml

version: "3.9"
services:
 meilisearch:
   image: "getmeili/meilisearch:latest"
   restart: unless-stopped
   ports:
     - "127.0.0.1:7700:7700"
   volumes:
     - /var/local/meilisearch:/data.ms

The, run the command docker-compose up to start MeiliSearch.

Creating a MeiliSearch client

Now that MeiliSearch is running, we want to create a client to interact with it.

Setting the MeiliSearch URL and API key

To create a client, we need to set the URL and the API key of our MeiliSearch instance. To do so, we can use environment variables.

In Next.js, it is as simple as creating a .env.local file at the root of our project and setting the following values:

MEILISEARCH_HOST=http://localhost:7700
MEILISEARCH_API_KEY=

When you need to run your application in production, you can set the correct values for these variables using your CI or service provider (such as Vercel).

Installing the MeiliSearch client for JavaScript

To install the MeiliSearch client for JavaScript, run the following command:

npm i meilisearch

Creating the client

Now that we have installed the MeiliSearch client, we can create a function to create a client to interact with our MeiliSearch instance.

meilisearch-client.ts

import { MeiliSearch } from 'meilisearch';
function getMeiliSearchClient() {
 const host = process.env.MEILISEARCH_HOST;
 const apiKey = process.env.MEILISEARCH_API_KEY;
 if (!host) {
   throw new Error('MEILISEARCH_HOST is not defined');
 }
 return new MeiliSearch({ host, apiKey });
}
export default getMeiliSearchClient;

Et voilà! When you need to use the MeiliSearch client, you can import the getMeiliSearchClient function and call it to get a client.

const client = getMeiliSearchClient();

Indexing our blog posts

Now that we have a client, we want to index our blog posts in MeiliSearch.

In this guide, we will use a very simple way to index our blog posts: we check if the files are diffed and only index the diffed files.

The first time you run the indexing, you will need to index all your blog posts.

Creating a shared interface for the indexed documents

We want to create a shared interface for your indexed documents, so we can use it across our application. This interface will be used to type the documents we index in MeiliSearch.

interface IndexedDocument {
  id: string;
  fullPath: string;
  content: string;
  title: string;
  path: string;
  collection: string;
  tags: string[];
  description: string;
  link: string;
  image?: string;
}
export default IndexedDocument;

Iterating over the blog posts to index them

To index our blog posts, we need to iterate over them. To do so, we will use a simple function that, given a directory, will recursively iterate over all the directories and find all the files with the extension .mdx.

import { loadEnvConfig } from '@next/env';
import { lstat, readdir } from 'fs/promises';
import { join } from 'path';
// load environment variables using Next.js
loadEnvConfig('.', process.env.NODE_ENV !== 'production');
async function getDocumentsFromDirectory(
  directory: string,
  tags: string[],
  documents: IndexedDocument[],
  mapper = (doc: IndexedDocument) => doc
) {
  // read the directory
  const files = await readdir(directory);
  // iterate over the files in the directory
  for (const file of files) {
    // get the path of the file
    const fullPath = join(directory, file);
    const stats = await lstat(fullPath);
    // if the file is a directory, we want to recursively iterate over it
    if (stats.isDirectory()) {
      await getDocumentsFromDirectory(fullPath, tags, documents, mapper);
    } else {
      // if the file is an MDX file, we want to extract the metadata
      if (stats.isFile() && file.endsWith(MDX_EXTENSION)) {
        // extract the metadata from the MDX file
        const matter = readFrontMatter(fullPath);
        // if something went wrong, we want to skip this file
        if (!matter) {
          continue;
        }
        // collect the metadata of the file and add it to the documents
        const content = matter.content;
        const data = matter.data;
        const collection = data.collection?.split('.json')[0];
        const path = file?.split('.mdx')[0];
        // if you have an ID for your documents, use that one
        // we don't, so we generate one from the full path
        const id = Buffer.from(fullPath).toString('hex');
        // add the document to the documents
        // we use the mapper function to transform the document if needed
        documents.push(
          mapper({
            id,
            fullPath,
            content,
            title: data?.title,
            description: data?.description,
            path,
            image: data.image,
            collection,
            tags,
            link: '',
          })
        );
      }
    }
  }
  return documents;
}

Let's quickly take a look at the function readFrontMatter, which allows us to read the metadata of an MDX file.

We use the package gray-matter to read the metadata of an MDX file:

import matter from 'gray-matter';
export function readFrontMatter(fullPath: string) {
  try {
    const fileContents = readFileSync(fullPath, 'utf8');
    return matter(fileContents);
  } catch {
    return;
  }
}

Indexing the blog posts

Now that we have a function to get all the documents from a directory, we can use it to index our blog posts:

const MDX_EXTENSION = `.mdx`;
const INDEX = 'content';
// index documents when the script is run directly
void indexAll();
// define the function to index the files
async function indexAll() {
  const client = getMeiliSearchClient();
  // we will collect all the documents in this array
  const documents: IndexedDocument[] = [];
  // get the documents from the blog posts directory
  // in this case, the directory is _posts
  await getDocumentsFromDirectory(
    '_posts',
    ['blog', 'posts'],
    documents,
    (document) => {
      // we want to add the link to the document
      return {
        ...document,
        link: `/blog/${document.collection}/${document.path}`,
      };
    }
  );
  const index = client.index(INDEX);
  try {
    console.log(`[Search Engine] Indexing ${documents.length} documents...`);
    // index the documents or update them if they already exist
    await index.updateDocuments(documents);
    console.error(`[MeiliSearch] Tasks created successfully`);
  } catch (error) {
    console.error(`[MeiliSearch] Failed to index documents`);
    console.error(error);
  }
}

If you have other collections of documents in different directories (as it is the case in this blog), you can simply add a call to getDocumentsFromDirectory for each directory.

The documents array will get populated with all the documents from all the directories.

Additionally, to make a property "filterable", you need to add it to the filterableAttributes:

client.index(INDEX).updateFilterableAttributes(['tags']);

🎉 Congratulations! You have now indexed your blog posts in MeiliSearch!

Executing the script

Now that we have a script to index our blog posts, we want to execute it when we build our application. To do so, let's add a script to npm and use tsx to execute it:

{
  "scripts": {
    "index-docs": "npx tsx indexer.ts"
  }
}

To run it, use the following command:

npm run index-docs

Indexing only the changed documents

If you have a lot of documents, you might not want to index all of them every time you build your application.

Therefore, after indexing your documents for the first time, we can use a slightly smarter way to detect which documents have changed and only index those. For example, we can use git to retrieve the list of changed files, and filter out all the files that have not changed.

Let's create a Javascript function to retrieve the list of files that have been changed using a git command:

function getChangedFiles() {
  const diffOutput = execSync("git diff --name-only -- '***.mdx' ");
  return diffOutput.toString().split('\n').filter(Boolean);
}

The above will check what .mdx files have changed and will return a list of file paths.

Now, we can use this function to only index the changed files, and we can update our function above to check that the file changed before adding it to the documents array.

At the top of the file, we add the changed files to the changedFiles constant:

const changedFiles = getChangedFiles();

Within the getDocumentsFromDirectory function, we add a check to see if the document is within the changedFiles array:

// only index changed files
const fileDidChange = changedFiles.includes(fullPath);
if (!fileDidChange) {
  continue;
}

Now, we only index the files that have changed! 🎉

Adding a Search Bar to our blog

Now that we have indexed our blog posts, we want to add a search bar to our blog so that users can search for blog posts.

Creating an API handler to search for blog posts

Since we need to access Meilisearch from the frontend, we need to create an API handler that can securely connect with the search engine.

The API endpoint will be /api/search:

The request accepts two query parameters: q for the search query and tags for the tags to filter on.
We use Zod to validate the query parameters
We return an array as a list of blog posts that match the search query and the tags

pages/api/search.ts

import { NextApiRequest, NextApiResponse } from 'next';
import { z } from 'zod';
import getMeiliSearchClient from '~/core/meilisearch-client';
async function searchHandler(
 req: NextApiRequest,
 res: NextApiResponse
) {
 // cache the response for 1 week
 res.setHeader(
   'Cache-Control',
   'public, s-maxage=604800, stale-while-revalidate=59'
 );
 const client = getMeiliSearchClient();
 // validate and extract the query from the request
 const { q, tags } = getBodySchema().parse(req.query);
 try {
   // get tags from the query
   const tagsFilter = (tags ?? '')
     .split(',')
     .filter(Boolean)
     .map((tag) => {
       return `tags = ${tag}`;
     });
   // search for the query in the index
   const { hits } = await client.index('content').search(q, {
     filter: [tagsFilter],
   });
   // return the results
   return res.json(hits);
 } catch (error) {
   // if something went wrong, return an empty array
   console.error(error);
   res.json([]);
 }
}
export default searchHandler;
function getBodySchema() {
 return z.object({
   q: z.string().min(1),
   tags: z.string().optional(),
 });
}

Creating a search bar component

Let's create the UI to search for blog posts. We will create a search bar in the header of the blog that allows users to search for the documents we indexed.

The component below uses APIs and components from the Makerkit UI kit: please treat these as implementation details and feel free to use your own components and APIs.

components/SearchBar.tsx

import React, {
 FormEventHandler,
 useCallback,
 useEffect,
 useMemo,
 useState,
} from 'react';
import Link from 'next/link';
import Image from 'next/image';
import { Popover, Transition } from '@headlessui/react';
import { SpringSpinner } from 'react-epic-spinners';
import {
 debounceTime,
 Subject,
 distinctUntilChanged,
 switchMap,
 tap,
} from 'rxjs';
import { useApiRequest } from '~/core/hooks/use-api';
import TextField from '~/core/ui/TextField';
import If from '~/core/ui/If';
function SearchBar({
 tags,
}: React.PropsWithChildren<{
 tags?: string[];
}>) {
 const [results, setResults] = useState<Maybe<Array<IndexedDocument>>>();
 const [searchRequest, { loading }] = useSearch();
 const subject$ = useMemo(() => new Subject<string>(), []);
 const [open, setOpen] = useState(false);
 const onInput: FormEventHandler<HTMLInputElement> = useCallback(
   (e) => {
     subject$.next(e.currentTarget.value);
   },
   [subject$]
 );
 useEffect(() => {
   const subscription = subject$
     .pipe(
       debounceTime(500),
       distinctUntilChanged(),
       switchMap((value) => {
         if (!value) {
           return Promise.resolve(undefined);
         }
         return searchRequest({
           q: value,
           tags: tags ?? [],
         });
       }),
       tap((results) => {
         setOpen(!!results);
       })
     )
     .subscribe(setResults);
   return () => subscription.unsubscribe();
 }, [searchRequest, subject$, tags]);
 return (
   <div className={'w-full'}>
     <TextField className={'flex justify-between'}>
       <TextField.Input
         className={
           'rounded-full !border-transparent !bg-black-400' +
           ' text-base font-normal'
         }
         onInput={onInput}
         onBlur={() => {
           setTimeout(() => {
             setOpen(false);
           }, 150);
         }}
         onFocus={() => {
           if (results?.length) {
             setOpen(true);
           }
         }}
         placeholder={'Search...'}
       >
         <If condition={loading}>
           <div className={'absolute right-4 top-2 text-primary-500'}>
             <SpringSpinner color={'currentColor'} size={24} />
           </div>
         </If>
       </TextField.Input>
     </TextField>
     <If condition={open}>
       <Popover className="relative top-1 z-10 rounded-sm">
         <Transition
           show={open}
           appear={open}
           enter="transition ease-out duration-200"
           enterFrom="opacity-0 translate-y-1"
           enterTo="opacity-100 translate-y-0"
           leave="transition ease-in duration-150"
           leaveFrom="opacity-100 translate-y-0"
           leaveTo="opacity-0 translate-y-1"
         >
           <Popover.Panel
             static
             className="PopoverPanel max-h-[400px] overflow-y-auto bg-black-300"
           >
             <div className="overflow-hidden rounded-bl-md rounded-br-md">
               <div className="relative flex flex-col space-y-1 bg-white p-2 dark:bg-black-300">
                 <If condition={results && !results.length}>
                   <p>
                     Sorry, we couldn&apos;t find any results for your search
                   </p>
                 </If>
                 {results?.map((result, index) => (
                   <Link
                     prefetch={false}
                     key={index}
                     href={result.link}
                     className={
                       'rounded-md bg-black-300 p-3 dark:hover:bg-black-200' +
                       ' transition-all'
                     }
                   >
                     <div className={'flex items-center space-x-8'}>
                       <If condition={result.image}>
                         {(image) => (
                           <img
                             src={image}
                             width={128}
                             height={128}
                             alt={`Image for ${result.title}`}
                           />
                         )}
                       </If>
                       <div className={'flex flex-col space-y-1'}>
                         <p>
                           <span className={'font-semibold text-white'}>
                             {result.title}
                           </span>
                         </p>
                         <p className={'text-sm text-gray-400'}>
                           {result.description}
                         </p>
                       </div>
                     </div>
                   </Link>
                 ))}
               </div>
             </div>
           </Popover.Panel>
         </Transition>
       </Popover>
     </If>
   </div>
 );
}
function useSearch() {
 return useApiRequest<
   IndexedDocument[],
   {
     q: string;
     tags?: string[];
   }
 >('/api/search', 'GET');
}
export default SearchBar;

Okay, let's break down what's happening here.

The Search API Request Hook

Let's start with the useSearch hook.

function useSearch() {
  return useApiRequest<
    IndexedDocument[],
    {
      q: string;
      tags?: string[];
    }
  >('/api/search', 'GET');
}

Nothing special here. The hook useApiRequest simply wraps a fetch call around a hook with a reducer to handle the loading state.

In your application, you're likely using SWR or React Query. Simply use their API to execute a fetch call to the API handler /api/search with a GET request.

Debouncing the Search Input Field with RxJS

One important thing to note is that we're debouncing the search input field using RxJs, which is a library for reactive programming that makes it a fairly trivial job.

Basically, we create a subject that we can use to emit values when the input changes. We then use the debounceTime operator to only emit values every 500 milliseconds, and switchMap to cancel any previous requests that are on-going when a new event is emitted.

Here we wrap a Subject with a useMemo hook to ensure that the subject is the same between renders:

const subject$ = useMemo(() => new Subject<string>(), []);

Then, we subscribe to the subject and use the useEffect hook to ensure that it is only run the first time the component is rendered:

useEffect(() => {
    const subscription = subject$
      .pipe(
        // debounce the request every 500ms
        debounceTime(500),
        distinctUntilChanged(),
        switchMap((value) => {
          if (!value) {
            return Promise.resolve(undefined);
          }
          // execute call
          return searchRequest({
            q: value,
            tags: tags ?? [],
          });
        }),
        tap((results) => {
          // only open the results list if there are results
          setOpen(!!results);
        })
      )
      .subscribe(
        // set the results
        setResults
      );
    return () => subscription.unsubscribe();
  }, [searchRequest, subject$, tags]);

RxJS can be intimidating at first, but learning it is well worth it.

Emitting text field changes to the Subject

Now that we have the subject, we can emit values to it when the input field changes:

const onInput: FormEventHandler<HTMLInputElement> = useCallback(
  (e) => {
    subject$.next(e.currentTarget.value);
  },
  [subject$]
);

Then, you can add the onInput handler to the input field:

<TextField.Input
  className={
    'rounded-full !border-transparent !bg-black-400' +
    ' text-base font-normal'
  }
  onInput={onInput}
  onBlur={() => {
    setTimeout(() => {
      setOpen(false);
    }, 150);
  }}
  onFocus={() => {
    if (results?.length) {
      setOpen(true);
    }
  }}
  placeholder={'Search...'}
>
  <If condition={loading}>
    Loading...
  </If>
</TextField.Input>

I also use the onBlur and onFocus handlers to open and close the results panel. The rest of the UI really is just a matter of displaying the results to the user and linking to the appropriate page.

Conclusion

In this tutorial, I've shown you how to index your blog posts in Meilisearch and how to create a React Search Bar component to query and display the results.

I hope you enjoyed this tutorial and that you'll be able to use Meilisearch successfully in your projects 🚀

If you have any questions, feel free to reach out to me on Twitter or Discord.