When I schedule my casting sessions, I keep a couple of important things in mind.
The first person to come to a casting session is at a disadvantage. No matter how dialed in I am, there are still adjustments to make, so they're auditioning while I'm settling in, tweaking the blocking, setting up my papers, and figuring out exactly how I'm going to get what I'm looking for.
The last people to come to a casting session are at an advantage. I'm looking for The One, and if I haven't found him or her by then, I'm hopeful that I will.
I'm also more confident in my approach. I've seen what's working and what actors aren't responding to. And I have a better sense of how the scene is going to work. (That is, unless I'm running way behind schedule and have a plane to catch, in which case I want to be done and out of there.)
These things may not be true for you, but they are for me. And as much as I may try to compensate for my weaknesses, I'm smart enough to know that it's an uphill battle.
So I work with them.
If there's someone I hold out a lot of hope for, I schedule him or her to come in toward the end. If I'm seeing someone as a favor, or if an actor is pretty clearly not right for the part but I want to give him or her a shot, I'll have him or her come in toward the beginning.
No, it's not fair. Fair would be doing the opposite. Tilting the playing field to favor the ones that I don't expect to do well and to put the ones I get a good feeling from at more of a disadvantage. The thing is, after running hundreds of casting sessions I've seen patterns emerge, and while I can't tell going in who's going to get the part, I can usually tell who's going to be a contender for it.
Besides, I'm open to being surprised.
It happens, too. Often enough for me to know better than to check out at any point. Or to rely too heavily on the people I schedule toward the end to come in and knock it out of the park.