M. Niyazi Alpay
M. Niyazi Alpay
M. Niyazi Alpay

I've been interested in computer systems since a very young age, and I've been programming since 2005. I have knowledge in PHP, MySQL, Python, MongoDB, and Linux.

 

about.me/Cryptograph

  • admin@niyazi.org

Laravel - Meilisearch and MongoDB Integration

Laravel - Meilisearch and MongoDB Integration

In my previous writings, I talked about integrating Laravel with both Meilisearch and MongoDB separately. In this piece, I'll discuss how to use all three together.

As I mentioned in my previous posts, we're integrating MongoDB and Meilisearch. Everything seems to be working fine: content I add is also created on the Meilisearch side, it gets updated when I update it, and it gets deleted in Meilisearch when I delete it. However, there's one issue: I'm not getting any results when I search.

When I examined the database query executed after the search operation in the debugbar output, I found that the issue stemmed from the search query executed on the MongoDB side.

Meilisearch executes the following sample query for search in a project developed with MySQL:

select * from `products` where `products`.`id` in (2, 1)

According to the search in the table named products, it calls the contents with id values 1 and 2, but the table name is specified again in the where condition. This is where the event starts on the MongoDB side :)

On the MongoDB side, the query is run as follows.

products.find({"products._id": {"$in":["656d9ac0ab082026280db1a4","656d9a78ab082026280db193"]}},{"typeMap":{"root":"array","document":"array"}})

The field named "products._id" does not exist. While in SQL, we can write queries in the format of "table.column_name", in MongoDB, we cannot write queries in this way.

In the /vendor/laravel/scout/src/Searchable.php file, it can be seen that the queries come from Laravel's own query builder structure. Unfortunately, we cannot directly modify files in this directory. Therefore, we will make edits by taking a copy of the Searchable.php file and calling it to the models we want to define as searchable.

We copy the /vendor/laravel/scout/src/Searchable.php file to the app directory under the Traits directory. You can place it anywhere under the app directory, but I prefer this approach to maintain the file structure. After copying, we change the content of the file as follows.

<?php namespace AppTraits;

use Illuminate\Database\Eloquent\Builder as EloquentBuilder;
use Illuminate\Database\Eloquent\SoftDeletes;
use Illuminate\Support\Collection as BaseCollection;
use Illuminate\Database\Eloquent\Collection;
use Laravel\Scout\Builder;
use Laravel\Scout\EngineManager;
use Laravel\Scout\ModelObserver;
use Laravel\Scout\Scout;
use Laravel\Scout\SearchableScope;

trait Searchable
{
    /**
     * Additional metadata attributes managed by Scout.
     *
     * @var array
     */
    protected array $scoutMetadata = [];

    /**
     * Boot the trait.
     *
     * @return void
     */
    public static function bootSearchable(): void
    {
        static::addGlobalScope(new SearchableScope);
        static::observe(new ModelObserver);
        (new static)->registerSearchableMacros();
    }

    /**
     * Register the searchable macros.
     *
     * @return void
     */
    public function registerSearchableMacros(): void
    {
        $self = $this;
        BaseCollection::macro('searchable', function () use ($self) {
            $self->queueMakeSearchable($this);
        });
        BaseCollection::macro('unsearchable', function () use ($self) {
            $self->queueRemoveFromSearch($this);
        });
    }

    /**
     * Dispatch the job to make the given models searchable.
     *
     * @param Collection $models * @return void
     */
    public function queueMakeSearchable($models)
    {
        if ($models->isEmpty()) {
            return;
        }
        if (!config('scout.queue')) {
            return $models->first()->makeSearchableUsing($models)->first()->searchableUsing()->update($models);
        }
        dispatch((new Scout::$makeSearchableJob($models))->onQueue($models->first()->syncWithSearchUsingQueue())->onConnection($models->first()->syncWithSearchUsing()));
    }

    /**
     * Dispatch the job to make the given models unsearchable.
     *
     * @param Collection $models
     * @return void
     */
    public function queueRemoveFromSearch($models)
    {
        if ($models->isEmpty()) {
            return;
        }
        if (!config('scout.queue')) {
            return $models->first()->searchableUsing()->delete($models);
        }
        dispatch(new Scout::$removeFromSearchJob($models))->onQueue($models->first()->syncWithSearchUsingQueue())->onConnection($models->first()->syncWithSearchUsing());
    }

    /**
     * Determine if the model should be searchable.
     *
     * @return bool
     */
    public function shouldBeSearchable()
    {
        return true;
    }

    /**
     * When updating a model, this method determines if we should update the search index.
     *
     * @return bool
     */
    public function searchIndexShouldBeUpdated()
    {
        return true;
    }

    /**
     * Perform a search against the model's indexed data.
     *
     * @param string $query * @param  Closure  $callback      * @return Builder
     */
    public static function search($query = '', $callback = null)
    {
        return app(Builder::class, ['model' => new static, 'query' => $query, 'callback' => $callback, 'softDelete' => static::usesSoftDelete() && config('scout.soft_delete', false)]);
    }

    /**
     * Make all instances of the model searchable.
     *
     * @param int $chunk * @return void
     */
    public static function makeAllSearchable($chunk = null)
    {
        $self = new static;
        $softDelete = static::usesSoftDelete() && config('scout.soft_delete', false);
        $self->newQuery()->when(true, function ($query) use ($self) {
            $self->makeAllSearchableUsing($query);
        })->when($softDelete, function ($query) {
            $query->withTrashed();
        })->orderBy($self->getScoutKeyName())->searchable($chunk);
    }

    /**
     * Modify the collection of models being made searchable.
     *
     * @param Illuminate\Support\Collection $models
     * @return Illuminate\Support\Collection
     */
    public function makeSearchableUsing(BaseCollection $models)
    {
        return $models;
    }

    /**
     * Modify the query used to retrieve models when making all of the models searchable.
     *
     * @param Illuminate\Database\Eloquent\Builder $query
     * @return Illuminate\Database\Eloquent\Builder
     */
    protected function makeAllSearchableUsing(EloquentBuilder $query)
    {
        return $query;
    }

    /**
     * Make the given model instance searchable.
     *
     * @return void
     */
    public function searchable()
    {
        $this->newCollection([$this])->searchable();
    }

    /**      * Remove all instances of the model from the search index.      *      * @return void */
    public static function removeAllFromSearch()
    {
        $self = new static;
        $self->searchableUsing()->flush($self);
    }

    /**
     * Remove the given model instance from the search index.
     *
     * @return void
     */
    public function unsearchable()
    {
        $this->newCollection([$this])->unsearchable();
    }

    /**
     * Determine if the model existed in the search index prior to an update.
     *
     * @return bool
     */
    public function wasSearchableBeforeUpdate()
    {
        return true;
    }

    /**
     * Determine if the model existed in the search index prior to deletion.
     *
     * @return bool
     */
    public function wasSearchableBeforeDelete()
    {
        return true;
    }

    /**
     * Get the requested models from an array of object IDs.
     *
     * @param Builder $builder
     * @param  array  $ids
     * @return mixed
     */
    public function getScoutModelsByIds(Builder $builder, array $ids)
    {
        return $this->queryScoutModelsByIds($builder, $ids)->get();
    }

    /**
     * Get a query builder for retrieving the requested models from an array of object IDs.
     *
     * @param Builder $builder
     * @param  array  $ids
     * @return mixed
     */
    public function queryScoutModelsByIds(Builder $builder, array $ids)
    {
        $query = static::usesSoftDelete() ? $this->withTrashed() : $this->newQuery();
        if ($builder->queryCallback) {
            call_user_func($builder->queryCallback, $query);
        }
        $whereIn = in_array($this->getScoutKeyType(), ['int', 'integer']) ? 'whereIntegerInRaw' : 'whereIn';
        return $query->{$whereIn}($this->getScoutKeyName(), $ids);
    }

    /**      * Enable search syncing for this model.      *      * @return void */
    public static function enableSearchSyncing()
    {
        ModelObserver::enableSyncingFor(get_called_class());
    }

    /**
     * Disable search syncing for this model.
     *
     * @return void 
     */
    public static function disableSearchSyncing()
    {
        ModelObserver::disableSyncingFor(get_called_class());
    }

    /**
     * Temporarily disable search syncing for the given callback.
     *
     * @param callable $callback
     * @return mixed 
     */
    public static function withoutSyncingToSearch($callback)
    {
        static::disableSearchSyncing();
        try {
            return $callback();
        } finally {
            static::enableSearchSyncing();
        }
    }

    /**
     * Get the index name for the model.
     * 
     * @return string 
     */
    public function searchableAs()
    {
        return config('scout.prefix') . $this->getTable();
    }

    /**
     * Get the indexable data array for the model.
     * 
     * @return array 
     */
    public function toSearchableArray()
    {
        return $this->toArray();
    }

    /**
     * Get the Scout engine for the model.
     * 
     * @return mixed 
     */
    public function searchableUsing()
    {
        return app(EngineManager::class)->engine();
    }

    /**
     * Get the queue connection that should be used when syncing.
     * 
     * @return string 
     */
    public function syncWithSearchUsing()
    {
        return config('scout.queue.connection') ?: config('queue.default');
    }

    /** 
     * Get the queue that should be used with syncing. 
     * 
     * @return string 
     */
    public function syncWithSearchUsingQueue()
    {
        return config('scout.queue.queue');
    }

    /**
     * Sync the soft deleted status for this model into the metadata. 
     * 
     * @return $this 
     */
    public function pushSoftDeleteMetadata()
    {
        return $this->withScoutMetadata('__soft_deleted', $this->trashed() ? 1 : 0);
    }

    /** 
     * Get all Scout related metadata.
     * 
     * @return array 
     */
    public function scoutMetadata()
    {
        return $this->scoutMetadata;
    }

    /**
     * Set a Scout related metadata.
     * 
     * @param string $key 
     * @param  mixed  $value 
     * @return $this 
     */
    public function withScoutMetadata($key, $value)
    {
        $this->scoutMetadata[$key] = $value;
        return $this;
    }

    /** 
     * Get the value used to index the model. 
     * @return mixed 
     */
    public function getScoutKey()
    {
        return $this->getKey();
    }

    /** 
     * Get the auto-incrementing key type for querying models.   
     * 
     * @return string 
     */
    public function getScoutKeyType()
    {
        return $this->getKeyType();
    }

    /**
     * Get the key name used to index the model.   
     *  
     * @return mixed 
     */
    public function getScoutKeyName()
    {
        return $this->getKeyName();
    }

    /**
     * Determine if the current class should use soft deletes with searching.
     * 
     * @return bool 
     */
    protected static function usesSoftDelete()
    {
        return in_array(SoftDeletes::class, class_uses_recursive(get_called_class()));
    }
}

We change the line that says '$self->qualifyColumn($self->getScoutKeyName())' to '$self->getScoutKeyName()', and we change the line that says '$this->qualifyColumn($this->getScoutKeyName()), $ids' to '$this->getScoutKeyName(), $ids'. Since we moved it to a different directory, we define the namespace as 'namespace AppTraits;'. Due to the change in namespace and directory, many functions are not working, so we call them into the file using 'use' to make them work.

use Illuminate\Database\Eloquent\Collection;

use Laravel\Scout\Builder;
use Laravel\Scout\EngineManager;
use Laravel\Scout\ModelObserver;
use Laravel\Scout\Scout;

use Laravel\Scout\SearchableScope;

In essence, what we're doing is disabling the qualifyColumn functions. Because it's this function that generates the query in the format of 'table_name.column_name'.

We call the file we created under app/traits to the model we want to define as searchable. You can see an example model below.

<?php
namespace App\Models;

use App\Traits\Searchable;
use Illuminate\Database\Eloquent\Factories\HasFactory;
use MongoDB\Laravel\Eloquent\Model;

class Blog extends Model {
          use HasFactory;
          use Searchable;

          protected $collection = 'blogs';

          protected $fillable = [
                    'title', 
                    'slug',
                    'body',
                    'image',
                    'user_id',
          ];
} 

Muhammed Niyazi ALPAY - Cryptograph

Senior Software Developer & Senior Linux System Administrator

Meraklı

PHP MySQL MongoDB Python Linux Cyber Security

You may also want to read these

2 comments

  • GromNaN
    GromNaN
    05 Nov 2024 11:52

    Thank you for sharing your issues and solutions. I read your article with interest, and here are a few comments: - You don't need to duplicate the whole trait when you want to update a single method. You can use a trait in an other trait. This will help you for maintenance if you don't duplicate code from the vendor. And this would highlight the code that is actually different from the vendor. - But, you don't need to make this changes. As of mongodb/laravel-mongodb version 4.1.0, the method "qualifyColumn" doesn't change the field name. Laravel Scout is supported out of the box with MongoDB document models.

  •  Cryptograph
    Cryptograph
    05 Nov 2024 11:55

    Thanks your comment

Leave a comment too

Your email address will not be published. Required fields are marked *