You can set the listbox's SelectMode property to None, but this would also disallow you being able to programmatically select any index in that listbox too.
It would still be easy enough to write coding that will search your second listbox and return the index you need to use without every actually selecting the item.