Pagination showing too many page numbers

I have a query that gets all the post from custom taxonomy. I have modified the query that it only shows products that has a custom field value in_stock = “yes”

I do this by adding meta_query to all the queries the site makes and it works well. The problem is that the pagination does not exclude those posts. At first it was not a problem but now that I have thousands of pages it starts to add up in the last pages. So the more posts are in the query, more empty pages are going to be in the end of pagination.

For example if I have 50 posts / page and total of 50 pages -> there might be posts with in_stock value “yes” only for 48 pages and the rest are with the value “no” and they are not in the loop (and not showing like they should) but they seem to be counted in pagination.