Count the number of files with yesterday’s date in ls’s output

I would like to count the number of files with yesterday’s date in ls’s output.
I tried:

yday=$(date --date="yesterday" +"%b%Oe")
ls . -ltr | grep $yday | wc -l

The problem is that files have dates with two spaces between the month and days less than 10, for example:

Jan  6
   ^^---- two spaces

but my command creates a date with only one space:

yday=$(date --date="yesterday" +"%b%Oe")

Jan 6
   ^----- one space

How can I modify my command to solve this problem?